RGMIL: Guide Your Multiple-Instance Learning Model with Regressor
Authors: Zhaolong Du
Shasha Mao
Yimeng Zhang
Shuiping Gou
Licheng Jiao
Lin Xiong
Venue: NeurIPS 2023 poster (2023)
Keywords: Video Analysis
Multiple-Instance Learning
Representation learning
TL;DR: Introduce a  regressor-guided method to learn discriminative instance-level representations in multiple-instance learning

Abstract: In video analysis, an important challenge is insufficient annotated data due to the rare occurrence of the critical patterns, and we need to provide discriminative frame-level representation with limited annotation in some applications. Multiple Instance Learning (MIL) is suitable for this scenario. However, many MIL models paid attention to analyzing the relationships between instance representations and aggregating them, but neglecting the critical information from the MIL problem itself, which causes difficultly achieving ideal instance-level performance compared with the supervised model.
To address this issue, we propose the $\textbf{\textit{Regressor-Guided MIL network} (RGMIL)}$, which effectively produces discriminative instance-level representations in a general multi-classification scenario. In the proposed method, we make full use of the $\textit{regressor}$ through our newly introduced $\textit{aggregator}$, $\textbf{\textit{Regressor-Guided Pooling} (RGP)}$. RGP focuses on simulating the correct inference process of humans while facing similar problems without introducing new parameters, and the MIL problem can be accurately described through the critical information from the $\textit{regressor}$ in our method. 
In experiments, RGP shows dominance on more than 20 MIL benchmark datasets, with the average bag-level classification accuracy close to 1. 
We also perform a series of comprehensive experiments on the MMNIST dataset. Experimental results illustrate that our $\textit{aggregator}$ outperforms existing methods under different challenging circumstances. Instance-level predictions are even possible under the guidance of RGP information table in a long sequence. RGMIL also presents comparable instance-level performance with S-O-T-A supervised models in complicated applications. Statistical results demonstrate the assumption that a MIL model can compete with a supervised model at the instance level, as long as a structure that accurately describes the MIL problem is provided. The codes are available on $\url{https://github.com/LMBDA-design/RGMIL}$.

RGMIL: Guide Your Multiple-Instance Learning Model with Regressor
Zhaolong Du', Shasha Mao'; Yimeng Zhang', Shuiping Gou', Licheng Jiao', Lin Xiong Xidian University, "SenseTime.
Abstract
In video analysis, an important challenge is insufficient annotated data due to the occurrence of the critical patterns, and need provide discriminative frame-level representation with limited annotation in some applications. Multiple Instance Learning (MIL) suitable for this scenario. However, many MIL models paid attention to analyzing the relationships between instance representations and aggregating them, but neglecting the critical information from the MIL problem itself, which causes difficultly achieving ideal instance- performance compared with the supervised model. To address this issue, we propose the Regressor- Guided MIL network (RGMIL), which effectively produces discriminative instance- representations in multi-classification scenario. the proposed method, we make full use of the regressor through our newly introduced aggregator, Regressor- Guided Pooling (RGP). RGP focuses on simulating the correct inference process of humans while facing similar problems without introducing new parameters, and the MIL problem can be accurately described through the critical information from the regressor in our method. In experiments, RGP shows dominance more than 20 MIL benchmark datasets, with the average bag-level classification accuracy close to 1. We also perform series of comprehensive experiments the MMNIST dataset. Experimental results illustrate that our aggregator outperforms existing methods under different challenging cireumstances. Instance-level predictions possible under the guidance of RGP information table a long sequence. RGMIL also presents comparable instance-level performance with S-O-T-A supervised models in complicated applications. Statistical results demonstrate the assumption that MIL model can compete with a supervised model the instance level, long as a structure that accurately describes the MIL problem is provided. The codes are available on ttttps://gitee httpss///gttbbb.ccm/BBBAddeedd comp/L/BBBA-eesigg//RR/
Introduction
Multiple-instanee learning (MIL) 3 33 is commonly used for the binary classification tasks in which instances lack individual labels and are grouped into bags. In MIL, a bag is labeled as positive if it contains at least one positive instance, otherwise, it is negative. Its goal is to learn the patterns that characterize positive bags. The models MIL. have been widely applied various applications,
such anomaly detection [6
13
pathology diagnosis (2 4. etc.
At present, many deep MIL models adopt attention baased modules [22 to solve the MIL problems, which aggregate instance-level representations in various ways. Such approaches work during the stage of aggregating instance-level representations and perform well when the given instance-level representations are with good discriminability. However, in some applications, such facial pain estimation, we face scenario where there are no discriminative instance-level representations or the instance-level representations need to be explicitly learned. For them, simply learning the aggregation
"Corresponding author. email: : ssmaodxidian enu.
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

process is far from enough when the instance-level performance is important. Most of the current methods fail achieve satisfying instance-level performance in this case.
To address this issue, we propose a new model of multiple-instance learning based on the guidance of regressor, named Sngressor-Guider MII. network (RGMIL)), which achieves promising instance- level performance. Compared with existing methods, our contributions are listed as follows:
RGMIL can address the general multi-class classification problem in MIL scenarios by learning discriminative instance-level features. RGMIL accurately describes the MIL problem through the newly introduced aggregator, Pegressor-Guided Pooling (RGP). With accurate description of the MIL problem, we can fully transfer the learning process into the direct learning of instance-level representations, without the requirement for increasingly complicated aggregation processes.
RGMIL demonstrates outstanding performance and expected functionality on public datasets, showcasing the essential role of RGP. Indicators show that RGP performs well under different challenging circumstances, and instance-level predictions are even possible with the guidance of RGP information in long sequence. Experimental results validate that the regressor guidance possibly brings similar instance-level performance to a supervised model.
Excepted the simulated data (MMNIST), the effectiveness of RGMIL is also validated on real pain estimation dataset (UNBC). Specifically, from the perspective of pain estimation, RGMII. can be as effective supervised learning models.
2 Related Works
2.1 The General Models of MII.
In the classical case of Multi-Instance Learning (MIL), binary-classification problem is built and solved based on bags, and each bag is formed by several instances. In general, a bag is defined as X [x1. x2, Xx), and its label is formulated by
max {yx), & [1.
Each bag includes t instances (X, E Pxt) and one label (Y, E (0, recognize what type of instances-level representation in the bag X, could make the bag label Y; be one (positive). Classical MIL also assumes that there are neither ordering nor dependency of instances within bag. There are two main architectures in the frameworks of MIL, shown as follows.
1)). The task of MIL is to
Architecture I: Instance-level Backbone Regressor + Aggregator
In this architecture, the MIL model generally takes an instance-level backbone to obtain the instance- level representations (fss [fi. ..., fi) which involves each instance bag (X = [x,,,.,,e] where f; is corresponding to x, f; E RR. i E [], Then, the instance-level representations (fs) are fed into the regressor to predict the score or possibility (p.) of categories for each instance. Finally, the aggregator is used to aggregate the predicted scores (P = [p1...., Pr]) and obtain the final bag-score Pbag Suppose p express the aggregation function, the aggregator is formulated as Pbag (P). and should follow the
inniutttiooiinarianttt restriction:
p (P) p (PT),
where T expresses arbitrary permutation matrix with appropriate dimension. In this architecture, the max pooling and average pooling are two most widely used aggregators. The max pooling (MXP) takes the highest value in produce Phag, while the average pooling takes the average value in P. and both of them comply with Equation (2).
Architecture 2.: Instance-level Backbone Aggregator Regressor
In this architecture, the MIL model produces the bag-level representation (F) directly via a back- taggegggro structure. For example, in image classification problem, a 2D-CNN backbone is used to get instance-level representation matrix fs E Reet and then aggregator p is applied to get the bag-level representation F = (fs). The function p still satisfy Equation (2). Then, each bag is predicted by the regressor based on the bag-level representation F E R In this architecture, the most widely used aggregators are coantion-based pooling (ABP) [2]. gated attention-basee pooling

---

(G-ABP) [1], and their variations [5 30 [32]. Compared with Architecture 1, Wang et al. 277 advocates Architecture 2 which produce the bag-level representation F, since they often perform better in terms of the bag-level classification.
2.2 Attention-based Pooling Series
At present, Attention-Based Pooling (ABP) and its variations are widely adopted in many mainstream methods of MIL. Generally, ABP is formulated as
exp { w T tanh (Vig)
F axfe as 2jm1 exp (wT tanh (Vr)))
where w E R' and V E X are both parameters to be trained. fg R is the kth instance representation in a bag. ABP provides the bag representation by weighted sum of representations of different instances. Moreover, many variations of ABP has been proposed, such as Dual-Stream MIL((221) [TS], OTFD-MIL((022 (30, (AMMI-(202) 32 Dual-Stream MII. [55] (DSP) used MXP and an MXP-guided ABP as two streams to aggregate representations and obtained the S-O-T-A level performance on MIL benchmarks. DTFD-MIL [30] divided bag into several pscudo-bags when the bag size was really large and applied ABP every pseudo-bag twice in two-stage manner. They still follow the main idea of ABP: weights should rely on representations.
In many applications, such as the whole slide image based pathology diagnosis [9] there is no strong requirement for sufficiently discriminative instance-level representations. The model is provided with bags with the bag length over large, and is asked to make accurate bag-level predictions. Most of the current models like ABP variations introduced above would only focus learning of aggregation process or limit the analysis process between instances given the instance-level representations fixed.
2.3 Pain or No Pain
In some real-world applications, it is excepted to obtain the accurate instance-level prediction or representation, such as pain estimation. Most studies of pain estimation based on facial expression 6 18 rely on the Facial Action Coding System (FACS) (5), which is rating system that distinguishes 44 facial movements, called Action Units (AU). Painful expressions are associated with some of these AUs. Earlier, Prkachin et al. [19] developed the Prkachin and Solomon Pain Intensity (PSPI) index based on these AUs, which is a 77-point scale reflecting the degree of pain, where indicates no pain and 16 indicates extreme pain. Whereas, due to the imbalanced distribution of the PSPI index, researchers redefined the pain intensity based on the PSPI index. In general, the pain intensity are divided into four categories based PSPI: no pain (0), mild pain (1 to 2), severe pain (3 to 5). and extreme pain (the above 5). At present, there are some existing MIL algorithms, such as MI-DORF [IR but it is difficult for a pure MIL algorithm provide ideal instance-level performance when compared with supervised deep models.
3 The Proposed Method
3.1 Assumptions
In this paper, we design the model (RGML) with the guidance of two assumptions shown follows.
Assumption 1 It is realistic for a MIL model achieve the performance similar to a supervised model at the instance level, as long as it has a well-designed architecture.
Compared to the When facing MIL problems, we start from the Architecture 2 given in section 2.1
fully supervised model, a new component aggregator added. Here we would not focus on whether the data is in the form of bags or instances. From the perspective of bags, it is still fully supervised. But, is simply another model with an additional component, and the problem to be solved not even ambiguous. When MIL problem is treated in this way, there is reason to make this assumption.
Assumption 2 Performance bottleneck of the current MIL model is the aggregator function p.
3

---

RGP
predec
RGP-N
Uneor Regresser
Backbone
Arbbary
Pesture Extsctor
Fratule Eatrctor
eyti (2
Regrosorr
Input Video
Backbone
Linear Regressor
RGP1
prddet
100
Model Overveew
Regnassor
(ta) Procedures the rth Branch
Figure I: Overview of Regressor-Guided mulltiple-instncee Learning Network (RGMIL)
Compared to the supervised learning, the available information obtained by MIL is limited. Therefore, the instance-level performance limit of MIL should be equivalent to the fully supervised mode. Two main architectures introduced above differ from the fully supervised ones only on the presence of the aggregator. We argue that the inability of MIL to achieve the performance of the fully supervised mode is due to the limitation of the learning stage of the new component (aggregator).
So what makes a good aggregator? We assume that we could describe the key differences between MIL problem and supervised problem in well enough, we could get an accurate MIL model. Recall what CNN does. Early in computer vision, representations were provided as fixed to train pure regressor. CNN took a step forward by its accurate description, which makes it possible to train the backbone + regressor at the same time, and get even better performance. The main task the new component would also be to provide accurate description.
3.2 Introduction of RGMIL
RGMIL is proposed for multi-classification scenario, shown in Fig It is achieved by N branches, each of which learns to distinguish one corresponding non-negative class (with non-zero label). Thus N-branch RGMIL is suitable to solve an (N + 1)-class classification problem. To solve the pain estimation problem, it requires only a sequence of frames and the highest pain intensity level (0 to N, discrete) of the sequence as input.
During each training step, a bag composed of t images is fed into the model, denoted as X [x1, X2, .2....,... x,], X E R Dxt, with bag label Ybgg [0, N] which indicates the max instance label in the bag. We convert Y Yang to special vector Y, similar to but not the same as one-hot encoding:
YERM, Y, = { 1, iff Dag = 0, otherwise
(i (1,
Then, the backbone will produce the instance-level representations fs [fr fg ...... f.] gxx corresponding to each image in this bag. The representations fs are fed into all the branches at the same time. For each branch B we have corresponding linear regressor with parameters W; E gex2 and b; E 2i E [1, NJ.. Parameters in both the RGP and the regressor will be NOT involved in the gradient descent. To be clear, since goal is produce discriminative instance-level representations, only the backbone component (instance- level feature extractor) needs to be trained. For the RGP and regressor components, they are just means of describing the MIL problem. They describe the MII, problem through specific structures, and as long as the structure is clear enough to provide accurate description, would be sufficient.
In the branch B', given the instance-leved representation matrix fs E Rext the RGP works as follows:
H fs
P
W H + by: (k 1,t R(x2)
(5)
P P T :p = P3 (p: R²)
(6)
p Elpl
(w E R' -
Var(p) +e
exp(W>)
R 0
(āt E (0,
(8)
Σラ= exp(w,)

---

F
Ci H (Fi R9)
Instance classification logits provided in Equation 5 Difference values are then calculated to indicate the importance of instances in Equation 6 Normalization like Equation 7 is adopted to prevent the gradient vanishing Equation Use Equation @
and 9 to get the final bag representation Fí. In a MIL problem. we obey the inference rule that if there are positive instance-level patterns, the bag is positive. That is to say, there consistency between the discrimination of critical instance- level patterns and the bag representation. We argue that this is a key element to describe a MIL problem, and we use a shared regressor to express this consistency between them. summary. the RGP asks for the current result of the regressor, and always tries to aggregate the instance representations based on the current judgments made by the regressor, then dynamically adjusts Much like what humans would do.
current judgments in the training process
Return to the process of forward propagation. For the branch B', given the bag representation Fi E I the corresponding regressor parameters will be used again to produce the prediction Z E IR² which will be used calculate the loss L in this branch(i E [I, formulated as follows:
Z
Wi F + b (Z E R22)
(10)
N L where L' 1-1
exp( (itxr) exp 21) texp 23))
log
iff
> Y bag
L
(11)
otherwise
The loss value L' on the corresponding branch B' is calculated as shown above. For a MIL problem, the bag label Y bag the maximum instance label in the bag. Based on this, we give the vector Y that describes the presence of instances with different labels in this bag. Y; =1= 0 means no instance with label presented in this bag, while Y;
means presence. Since bag is the maximum instance label, some bits of Y would be reliable. For Y, where i E (Y bag. NJ. it gives the correct ground truth, while for Y; where E [1, Y bag 1 it is not reliable. So for the branch B where i [1, bag - 1]. is meaningless to calculate the loss since it may contain false information. We set the loss on these branches to 0, as shown in Equation [1] For other reliable branches, we calculate the cross-entropy loss respectively. The total loss is the sum of the loss values on all branches.
During test, for the branch Bi, after we get the output Zi, we can get the indicator vector Y:
Ý e ERR; Y, = argmax { | 1; (k E (1,2})
(12)
k
The branch B' finally produce a number of 0 or I based on Zi. When get the number of in the branch B' that indicates the presence of the corresponding label i, we say the model predicted that there are instances with the label in the bag. Hence, the indicator vector Y describes whether the bag contains different labels. And at last,
Y argmax Y Y = 1;ke [1
k
(13)
When different branches output I, we take the highest branch, because we need to know the maximum
label in the bag shown in Equation 13 Above equations produce the final result Y, Y N).
4 Experiments and Analyses
4.1 Bag-Level Performance: Evaluation on Benchmarks
With the branch number N 1, five classic MIL benchmark datasets 1 are used to evaluate the bag-level performance, and each provides the samples in form of bags, where the bag size t varies widely, ranging from to 1044. Experimental results are shown in Table [ test for the model under different conditions. From Table [] it is obviously seen that our method (RGMIL) obtains the highest accuracies for five datasets, especially for FOX dataset. More benchmark performances are shown in Section B of our supplementary.
a comprehensive
5

---

Methods
MUSKT
MUSK2
FOX
TIGER
ELEPHANT
MI-Net(2018)
0.887 :: 0.041
0.859
0.046
0.622 4: 0.038
0.830 + 0.032
0.862 + 0.034
ABMIL((2018)
0.892 0.040
0.858
0.615 0.043
0.839 0.022
0.868 0.022
Galedd-BMM1.22011
0.900 + 0.050
0.863 0.042
0.603 + 0.029
0.845 + 0.018
0.857 + 0.027
OP-MINN(2018)
0.907 = 0.036
0.926 = 0.043
0.655 + 0.052
0.897 + 0.028
0.894 + 0.030
Non-Local(2018)
0.921 = 0.017
0.910 0.009
0.703 + 0.035
0.837 + 0.013
0.876 + 0.011
Asymmetric Non- (2c19(2019)
0.912 0.009
0.822 0.084
0.643
0.733 0.068
0.883 0.014
Dual-Stream 12L(2021)*
0.932 0.023
0.930 0.020
0.729 0.018
0,869 +: 0.008
0.925 + 0.007
BDR()022)
0.926 + 0.079
0.905 + 0.092
0.629 + 0.110
0.869 + 0.066
0.908 + 0.054
(AMII122025
0.933 + 0.065
0.910
0.085
0.685 + 0.093
0.894 + 0.070
0.915 + 0.058
RGMIL
0.968 0.060
0.985
0.039
0.954 0.048
0.951 0.045
0.965 1 0.032
Table I: The classification accuracy (mean + std) of RGMIL and compared methods on five benchmark datasets f*for S-O-T-A method). The results are the average of five independent experiments with a
10-fold cross-validation. Statistics collected from MI-Net(27], ABMIL[[2], Gated-ABMIL [12], DP-
MINN 29], Non-Local Asymmetric Non-Local DSMIL 15] [DR[1]] and GAMIL [3
4.2 A Challenging Dataset for Demonstration
Since pain estimation is generally regarded as the classification problems with four classes, we need to set the branch N to 3. It indicates that construct a 3-branch RGMIL to meet the upcoming challenges. Due to the extremely imbalance of label distribution, the pain data is not really a clear and flexible enough choice to present demonstration. Thus, we constructed flexible MIL dataset based on MNIST [14] and denoted it as MMNIST. In MMNIST, samples are presented in form of bags. A bag is composed of grayscale images (1,28,23 28) selected from the MNIST dataset. With N = we take the images with label 0 to to form MMNIST bags. The bag label is assigned based on the label with the largest count. The number of training bags for each bag label is approximately 1000. The labels in the bag are uniformly distributed. For the test, we use a fixed number of 10000 test images.
4.3 Evaluation Methods on the Aggregator p
We introduce the error rate o, and the distance y to describe the aggregator p to figure out how good the aggregator describes a MIL problem.
To explain the indicators, we still need to start from classical MIL. binary-classificttinn scenario. Given the instance representations fs- [f, f2..... fr] E Rext, the aggregator p. the classifier with linear transformation A and the sigmoid function sigm(...) as the activate function, with the total N bags with the bag label Y E [0, 1] in the dataset. Suppose it be currently in the training phase, the training phase is equivalent to a process that we try to find the solution to the system of N equations, each with the following formula:
sigm (fpss) Y; Y (0,1)
(14)
Here, we assume that aggregator P can be represented in special formula:
p (ss)
Go; s.t. Σ oz oz = 1; 1; on 2 0
(15)
k=1
Then, we can transform the Equation 14 into the following formula:
{
+00; if the given bag label is oo; otherwise
as (Aft)
(16)
The detailed explanation of this transform are given in Section A of our supplementary. Then, we consider the training process when we apply the gradient-descent algorithm here. Suppose the corresponding weight a of the kcth instance in a positive bag be larger than others', the absolute value of T would be larger than others'. Here think the instance produces large gradient. 82
We recognize that Af, is the classification score of the corresponding kth instance. What's more, its instance label is consistent with the bag label, we call it a critical instance. Given Qi greater than O, the gradient-descent algorithm would just push the parameters Af, to the right direction to produce correct instance-level prediction. The overall gradient of the parameters in the network will be the sum of different gradients produced by different instances in this bag. In conclusion, if an instance in bag has large corresponding weight and it is critical instance, then Equation 16 would be

---

Aggregators
10/1
16/1
32/1
48/1
64/1
512/1
F/ul)
MXP
93.47
88.71
81.17
73.02
68.11
65.37
ABP
92.18
89.69
90.84
86.59
89.47
42.21
G-ABP
91.47
89.13
87.47
84.38
86.93
36.99
97.75
DSP
58.54
58.12
58.47
56.13
52.43
42.23
RGP
98.25
97.55
96.37
95.12
94.50
87.28
Table 2: Instance-level Average Test Accuracies for the first 30 Training Epochs on MMNIST. The
first row contains the different modes (M/N), and 1/1(Full) expresses the fully supervised case.
(al (101)
(0 (801))
(c) (5121)
68 (5122))
69 y (5121)
Figure 2: Detailed metric fluctuations of Table under different modes (M/N),
likely to produce a right gradient-descent direction to achieve good performance on instance-level prediction, and that is what we want. If the gradient-descent direction of the equation is beneficial for the instance-level prediction, we call a right equation, otherwise an erroneous equation. The error rate 6 is the percentage of erroneous equations the total number N of equations. & describes the degree of deviation during the training phase. When ø > 0, gradient-descent accumulates errors approximate Φ by following steps: In a positive bag,
in the wrong direction. In practice, calculate the values of a for critical instances and take the average as 31,, and the average value of for non-critical instances is denoted 32. If 31 is smaller than (2,, we consider this to be erroneous equation. We approximate ø by taking the ratio of the number of erroneous equations and the total number of positive bags, and & evaluated once for every training epoch. For fully supervised model, it is easy to imagine that its & is O from the beginning of train phase, since every equation would produce a right gradient-descet é is supposed to be close enough to 0, to be similar enough to fully supervised model.
direction. We argue that with good aggregator p.
Another indicator is the distance Y. Still imagine what human will do when facing the MIL problem. To predict whether the bag is positive, we usually make decisions only based on the several critical instances in this bag. A well-designed p should simulate this characteristic. ? is metric that measures the difference in weights between critical instances a positive bag, which describes how similar it is to human decision-making making.. In practice, calculate 7 as follows: For every positive bag, we calculate the difference between the maximum and minimum weights a of the corresponding critical instances, and take the average of these differences for all positive bags, denoted as y- ? is also evaluated once for every training epoch. For the max pooling, it is easy to imagine that its large. We argue that with a good
distance y could reach but do not want the distance aggregator is supposed to be close enough to 0, to be similar enough to how we make decisions.
Notice that and 7 introduced to describe the training process for demonstration, thus all evaluated on the training set. Instance labels are known just when we evaluate á and 7,
4.4 Instance-Level Evaluation on MMNIST: Why RGP Works?
When an experiment with the training bag size (M) and the testing bag size (N) is implemented, we define it the M/N mode. We give comprehensive experiments in different modes test the aggregators in Table 2 For some detail fluctuations see Fig. 2 Results show that RGP shows dominance in different scenarios. As shown in Fig. 2 (d) and (e) under mode 512/1, with the faster convergence of o to 0 and the closer value of 7 to O, RGP exhibits a working logic that is sufficiently similar to human thinking process and supervised algorithm. For RGP, it represents the consistency between instances and bags through specific structure, so that the learning process can be easily transferred to instance level. Attention-based thoughts are valuable, but simply applying more complicated attention modules to analyze the problem, as done in ABP series, may not lead to breakthrough results on instance level because indicators show that they struggle to catch the points. Another critical fact we found is that if we switch training bag size to 1, which would bring fully
7

---

0
3
2
2
2
3
Branch
0.0151
0.031
0.4981
0.1183
0,0220
0A8557
043365
0.0001
0.313
0.1497
0.0132
0,0577
0.0841
0.3062
0.0185
0.1959
0.1977
0.0299
0.069
0.0360
0,0375
0.1889
0.0479
0.0889
0.2001
Table 3: RGP Information Table (64/10) on MMNIST. The first row contains the 10 images of the
test bag. Indicator vector Y is the model output. The remaining contents are all the weights of corresponding instances from all the branches.
supervised scenario, the performance is lower than RGP in 10/1 mode. This clearly tells that RGP is not performance bottleneck, it would enhance the model when the bag size is relatively small. With the guidance of regressor, RGP describes the MIL problem accurately, which make it similar enough to a supervised case to produce discriminative instance-level representations from backbone. We can also say that, when the number of data recordsican be in form of bags or instances) given fixed, MIL model with RGP could even outperform a fully supervised one under some condition. That verifies our Assumption 1.
We also provide a special experiment under mode 64/10, with the RGP information table presented in Table 3 We see that not only we get a correct indicator vector Y on bag level, we can also directly provide instance-level predictions by combining weight-related information with the model output Y: try to take the maximum value in each column.
RGP strictly follows the expectation we gave it, which is a strong constraint inspired by the inference rule. For positive bag, according to the way RGP works, when the kth instance representation with a high classification score inside the bag MUST have a larger value of weight or. Referring to Equation 16 that explains 6 and y, the absolute value of its partial derivative 82 JA. during the training process would be larger. The gradient descent algorithm will increase the classification score, which promotes the increase of its weight or: because it explicitly depends on the classification score in RGP. The gradient descent process produces instance-level representations with large weights and high classification scores in positive bags. For negative bags, the training process will lower the classification scores of all instance representations inside it, but since there no positive instances in it, the weights of the instance representations would be random. The gradient descent process will uniformly decrease the classification scores of all instances inside the negative bag. In the end of training process, RGMIL will converge to a solution where all the training bags are predicted correctly. In this final state, there is a huge difference in instance weights, and instance representations in positive bags with high classification scores will have significantly larger weights. This final solution we get from RGMIL would comply with our inference rule above. In a MIL problem, any solution that meets our inference rule is legal.
in the training process,
Since we have verified that the final convergence state of RGMIL is a legal solution to the MIL problem, the convergence process implies an improvement in the model's ability to distinguish critical instances. But still, the speed of convergence is crucial to improve instance-level performance. We must accelerate the speed at which the model generates discriminative instance-level representations as much as possible, because when the loss value reaches or approaches 0, the model wouldn't 't be trained anymore, and too slow a speed can stop the model from reaching the final solution. With low y value, RGP is able to capture more critical instances, which brings quicker improvement in instance-level discriminative ability. Based on this, RGMIL could correct previous erroncous inferences to reduce the & value quickly and thus avoid the accumulation of errors on instance level. That leads a huge improvement on instance-level performance when compared with other methods. After three epochs of training, RGMIL successfully lowered the ø-value to 0 in the fourth epoch and maintained it till the end. Multiple experiments have shown that RGMIL can always adjust its inference to lower the v-value to 0 within the first seven training epochs in 512/1 mode.
4.5 Ablation Study of RGP
As shown above, we have provided the derivation of the feasibility of RGP using sigmoid as the activation function, which we refer to as the original version. The derivation built on the case where there is no trouble in numerical computation or gradient descent. However, in the practical implementation, we still need to explore more practical way. Here, we still refer to the formula of the RGP forward propagation process in the ith branch (B') mentioned in previous section. In
8

---

this implement, we take the difference by using Equation 6 meanwhile using Equation 7 to achieve the normalization for preventing gradient vanishing. Without the normalization equation, if we take the value of the second bit of the softmax output instead of the difference, it would be similar to the original version, which we refer to as the baseline version. The experimental results for different versions are shown in T Obviously, it is seen that the performance based the different and the normalization superior to others. Without the training tricks from the equations, it is difficult to reduce 7-value to low level.
Aggregators RGP (B) RGP (B + N)
10/1
32/1 1
64/1 1
512/1
94.60
82.64
62.75
66.54
95.31
93.92
90.88
72.70
RGP (B + N D)
98.25
96.37
94.50
87.28
Table 4: Instance-level Average Test Accuracies for first 30 Training Epochs on MMNIST, where 'B'
expresses Baseline Version, 'N' expresses Normalization Formula, "D' expresses Difference Formula.
4.6 Comparison with Supervised Models in Pain Estimation
In this part, pain estimation is considered as real application to validate the performance of RGMIL, where the benchmark pain dataset (UNBC-McMaster Shoulder Pain dataset [17]. UNBC) is used. The UNBC dataset contains 200 video sequences with 48398 frames from 25 subjects, and all frames well-annotated. To compare with the existing supervised models, we follow the same experimental settings with the S-O-T-A model MSRAN (1) 2 using 25-fold cross-validation with save-one-subject-on
strategy and evaluating by four metrics. In experiment, all images are resized as (3, 224, 224), and the training bag size is set as 64. An average of about 6000 bags is used as training data during each fold validation, and ResNet18 { is used as backbone. Experimental results are shown in Table 5 From Table 5 it is seen that RGMIL. does gain comparable performance to supervised methods. It indicates that current design is effective enough to be practical without causing any performance degradation compared to supervised ones for pain estimation. More details about metries and settings are given Section C of supplementary.
Methods
MAE $
MSE 4
PCC f
ICC ft
Deep Pain(2017)
0.50
0.74
0.78
0.45
DSHF(2018)
0.94
0.68
DBR(2018)
0.69
0.81
Multistream CNN(2019)
0.47
0.53
0.70
0.55
(202))(2021)
0.40
0.46
0.78
0.63
LIAN(2021)
0.45
0.66
0.81
0.61
RGMIL
0.31
0.42
0.77
0.62
Table 5: Instance-level Performance Comparison with Supervised Models (":S-O-T-A model) UNBC pain dataset, where all results are the mean of 25-fold cross-validation and four metrics are used: MAE, MSE, PCC and ICC. In this experiment, we take consecutive 64 frames in video as a training bag. To increase the amount of data, we used sliding window approach with step size of 8 produce training bags. Statistics collected from Deep Pain [21), DSHF 25 DBR 23
Multistream CNN 10), MSRAN 22, LIAN 28
4.7 Discussion for More General Scenarios and Limitations
To validate the performance of RGMIL on the general multi-class bag-level classification problems, we test our model on SIVAL (20) dataset, where SIVAL consists of 25 classes of complex objects photographed in different environments and each class contains 60 images. In this experiment, each image segmented into approximately 30 segments, and each segment represented by 30-dimensional feature vector that encodes information. The segments are labeled as containing the object or the background. We separately select 1, 3 and 10 classes as positive classes, and randomly sample from the other classes as the negative class. We set up three Linear+ReLU blocks as feature extractors. The test scenario involves image classification problems ranging from binary to 1-llass classification. Here, each image is treated as bag. Considering that the classes of SIVAL do not have any ordering relationship, all bits of the output indicator vector are reliable, thus all branches are involved to the computation of loss function. Continuing with the current version's 's approach, the

---

model's output during test is still obtained by selecting the branch with the highest position among all branches with an output of 1. Table shows the bag-level performance on SIVAL dataset [20]. From T it is seen that RGP still obtains competitive results on the general bag-level problem.
# of positive classes
3
10
MXP
94.87
89.08
60.95
ABP
97.05
93.93
75.71
G-ABP
97. 97.31
94.28
81.33
DSP
96.79
89.60
75.91
RGP
97.31
94.40
80.19
Table 6: Bag-level Results on the general multi-class image classification problem (the SIVAL dataset), where 1, and 10 express three different multi-classification problems in which 1, and 10 classes are separately used as positive classes, respectively. The shown result is the average of 10-time test accuracy for each model after training convergence.
Moreover, we also validate RGMIL on the general multi-class instance-level classification problems. In this experiment, we still use the 3-branch RGMIL, and adopt different approach in constructing the data: the images from classes (ranging from 0 to 6) in MMNIST are considered as the negative instances, like the background class, and the images from classes (7, 8 and 9) in MMNIST are as three different positive instances. Table shows the instance-level performance of the constructed four classes classification on MMNIST. Unlike the original MMNIST series, the convergence speed of different methods varies much. Thus, we present the average of 10-time test accuracy for each model after convergence or training after 100 epochs. Each positive class and the negative class account for approximately 25% of the data. Within one positive bag, the number of images of positive class is approximately 10%. The processing of loss value and output is the same as the SIVAL experiment it is also seen that instance- level results are promising in a more general case.
above. From Table 7
Aggregators
10/1
16/1
32/1 1
64/1
256/1
512/1
MXP
76.09
71.78
57.49
58.15
38.63
32.48
ABP
71.76
75.17
71.81
68.33
69.09
62.43
G-ABP
73.12
75.04
72.21
64.58
69.20
60.56
DSP
62.81
69.29
63.75
67.04
59.77
52.91
RGP
88.79
85.73
85.02
83.19
82.50
81.46
Table 7: Instance-level Results on general 4-class problems constructed based MMNIST. The first row contains the different modes (M/N). The shown result is the average of 10-time test accuracy for each model after convergence or 100 epochs.
Although RGMIL is suitable to solve the general multi multiicclassificaiin problem in which class de- pendencies or ordering need not be considered, the feasibility of training on instance-level with an excessively large bag-length still need to be ensured. As shown in Table 1, it is seen that the instance-periee perence-prfoommnce decays with increasing bag lengths. Obviously, when the bag length is 512, the accuracy is lowest and reduced by 1 % than the minimum bag length. In addition, there is memory limitation to handle the excessively large bags when the feature extractor is involved in training.
5 Conclusion
In this paper, we propose new vision on MIL with a practical multi-classification MIL. model (RGMIL). RGMIL. extends the classical binary to a multi-classsfication scenario via optimizing several reliable equations simultaneously. More importantly, key component (RGP) is introduced in the MIL model. We argue that the difference between a supervised problem and MIL is deterministic and clear: we may not need an over-complicated black box simulate the difference. An accurate description exists. We describe the characteristic of MIL problems through the structure of RGP, and only the parameters in the feature extractor (backbone) need to be trained to provide the discriminative instance-level representation. This is method that has good interpretability and conforms to the true inference rule of the MIL problem. We also validate RGMIL real-world application the UNBC dataset. Suffice it to say that it is possible for MIL model to achieve even better instance-leve performance than some supervised models. We expect RGP to become an important component of future MIL network architectures and to provide some inspiration.
10

---

Acknowledgments
This work is supported in part by the State Key Program of National Natural Science of China under Grant 62234010, the National Natural Science Foundation of China under Grant 61806154, the China Postdoctoral Science Foundation Funded Project under Grant 2019M653565, and the Fundamental Research Funds for the Central Universities No.ZYTS23060.
References
[1] Stuart Andrews, Ioannis Tsochantaridis, and Thomas Hofmann. Support vector machines for multiple-instance learning. Advances in neural information processing systems, 15, 2002.
[2] Shaoxing Cui, Dong Huang, Yue Ni, and Xiaoyi Feng. Multi-scale regional attention networks for pain estimation. In 2021 13th International Conference on Bioinformatics and Biomedical Technology, pages 1-8, 2021.
[3] Thomas G Dietterich, Richard H Lathrop, and Tomás Lozano-Pérez Solving the multiple instance problem with axis-parallel rectangles. Artificial intelligence, 89(1-2)331-7, 1997.
[4] Michael Gadermayr and Maximilian Tschuchnig. Multiple instance learning for digital pathology: A review on the state-of-the-ar,, limitations & future potential. arXiv preprint 20Xi::2200.04445,, 2022.
[5] Behnood Gholami, Wassim Haddad, and Allen R Tannenbaum. Relevance vector machine learning for neonate pain intensity assessment using digital imaging. IEEE Transactions on biomedical engineering, 576))114571446, 2010.
[6] Teena Hassan, Dominík SeuB, Johannes Wollenberg, Katharina Weitz, Miriam Kunz, Ste- fan Lautenbacher, Jens-Uwe Garbas, and Ute Schmid. Automatic detection of pain from facial expressions: survey. IEEE transactions pattern analysis and machine intelligence, 1315):1815-1831, 2019.
[7] Teena Hassan, Dominik SeuB, Johannes Wollenberg, Katharina Weitz, Miriam Kunz, Stefan Lautenbacher, Jens-Uwe Garbas, and Ute Schmid. Automatic detection of pain from facial expressions: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 43(6):1815-183) 2021.
[8] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and paltern recognition, pages 770-778, 2016.
[9] Lei He, L Rodney Long, Sameer Antani, and George R Thoma. Histology image analysis for carcinoma detection and grading. Computer methods and programs in biomedicine, 107(3):538- 556, 2012.
[10] Dong Huang, Zhaoqiang Xia, Lei Li. Kunwei Wang, and Xiaoyi Feng. Pain awin-ewareness mul- tistream convolutional neural network for pain estimation. Journal of Electronic Imaging, 28(4):043008 -00000000000000088,.. 2019.
[11] Shiluo Huang, Zheng Liu, Wei Jin, and Ying Mu. Bag dissimilarity regularized learning. Pattern Recognition, 126:108583, 2022.
[12] Maximilian Ilse, Jakub Tomezak, and Max Welling, Attention- based deep multiple instance learning. In International conference on machine learning, pages 2127-2136. PMLR, 2018.
[13] Ammar Mansoor Kamoona, Amirali Khodadadian Gostar, Alíreza Bab- Hadiashar, and Reza Hoseinnezhad. Multiple instance-based video anomaly detection using deep temporal encoding- decoding. Expert Systems with Applications, 214:1 19079, 2023.
[14) Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(1 86(11):2278-233 1998.

---

[15] Bin Li, Yin Li, and Kevin W Eliceiri. Dual-stream multiple instance learning network for whole slide image classification with self-supervised contrastive learning. In Proceedings of the IEEF/CVF conference computer vision and pattern recognition, pages 14318-14328, 2021.
[16] Yang Liu, Jing Liu, Xiaoguang Zhu, Donglai Wei, Xiaohong Huang, and Liang Song. Learning task-specific representation for video anomaly detection with spatial-temporal attention. In ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 2190-2194. IEEE, 2022.
[17] Patrick Lucey, Jeffrey F. Cohn, Kenneth M. Prkachin, Patricia E. Solomon, Sien Chew, and Iain Matthews. Painful monitoring: Automatic pain monitoring using the unbe-memaster shoulder pain expression archive database. Image and Vision Computing, 30(3):197-205 2012. Best of Automatic Face and Gesture Recognition 2011.
[18] Shasha Mao, Guanghui Shi, Licheng Jiao, Shuiping Gou, Yangyang Li, Lin Xiong, and Boxin Shi. Label distribution amendment with emotional semantic correlations for facial expression recognition, 2021.
[19] Kenneth M Prkachin and Patricia E Solomon. The structure, reliability and validity of pain expression: Evidence from patients with shoulder pain. Pain, 139(2):267-274 2008.
[20] Rouhollah Rahmani, Sally A Goldman, Hui Zhang, John Krettek, and Jason Fritts. Localized content based image retrieval. In Proceedings of the 7th ACM SIGMM international workshop on Multimedia information retrieval, pages 227-236 2005.
[21] Pau Rodriguez, Guillem Cucurull, Jordi Gonzålez, Josep M Gonfaus, Kamal Nasrollahi, Thomas B Moeslund, and Xavier Roca. Deep pain: Exploiting long short-term memory net- works for facial expression classification. IEEE transactions on cybernetics, 4345):331--3324 2017.
[22] Adria Ruiz, Ognjen Rudovic, Xavier Binefa, and Maja Pantic. Multi-instance dynamic ordinal random fields for weakly supervised facial behavior analysis. IEEE transactions on image processing, 27(8):3969 3982, 2018.
[23] Mohammad Tavakolian and Abdenour Hadid. Deep binary representation of facial expressions: A novel framework for automatic pain intensity recognition. In 2018 25th IEEE International Conference on Image Processing (ICIP), pages 1552-1956. IEEE, 2018
[24] Maximilian E Tschuchnig and Michael Gadermayr. Anomaly detection in medical imaging-a mini review. In Data Science-Analytie and Applications: Proceedings of the 4th International Data Science Conference-iDSC2222. pages 33-38. Springer, 2022.
[25] Jinwei Wang and Huazhi Sun. Pain intensity estimation using deep spatiotemporal and hand- crafted features. IEICE TRANSACTIONS on Information and Systems, 101(6)1572-1158, 2018.
[26] Xiaolong Wang, Ross Girshick, Abhinav Gupta, and Kaiming He. Non-local neural networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 7794-7803, 2018
[27] Xinggang Wang, Yongluan Yan, Peng Tang, Xiang Bai, and Wenyu Liu. Revisiting multiple instance neural networks. Pattern Recognition, 14:15-24, 2018.
[28] Xuwu Xin, Xiaowu Li, Shengfu Yang, Xiaoyan Lin, and Xin Zheng. Pain expression assessment based on a locality and identity aware network. IET Image Processing, 15(12):2948 -9958, 2021.
[29] Yongluan Yan, Xinggang Wang, Xiaojie Guo, Jiemin Fang, Wenyu Liu, and Junzhou Huang. Deep multi-instance learning with dynamic pooling. In Asian Conference on Machine Learning, pages 662-677. PMLR, 2018
[30] Hongrun Zhang, Yanda Meng, Yitian Zhao, Yihong Qiao, Xiaoyun Yang, Sarah Coupland, and Yalin Zheng. Difd-mil: Double-tier feature distillation multiple instance learning for histopathology whole slide image classification. In Proceedings of the IEFF/CVF Conference on Computer Vision and Pattern Recognition, pages 18802-18812 2022.
12

---

[31] Yong Zhang, Rui Zhao, Weiming Dong, Bao-Gang Hu, and Qiang Ji. Bilateral ordinal relevance multi-instance regression for facial action unit intensity estimation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 7034-7043, 2018.
[32] Lu Zhao, Liming Yuan, Kun Hao, and Xianbin Wen. Generalized attention-basel based deep multi- instance learning. Multimedia Systems, 29(1):275-28)) 2023.
[33] Zhi-Hua Zhou, Yin Sun, and Yu-Feng Li. Multi-instance learning by treating instances non-iid samples. In Proceedings of the 26th Annual International Conference Machine Learning, pages 1249-1256. ACM, 2009.
[34] Zhen Zhu, Mengde Xu, Song Bai, Tengteng Huang, and Xiang Bai. Asymmetric non-local neural networks for semantic segmentation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 593-602, 2019.
13


Reviews:
[{"review_id": "nLcdEstbe6", "reviewer": "Reviewer_4ybk", "summary": "This paper introduces _Regressor-guided MIL network (RGMIL)_ as a solution to address the challenges in Multiple instance learning, particularly in the multi-class classification scenario for pain-estimation. RGMIL introduces a novel aggregator, Regressor-Guided Pooling (RGP) in place of currently widely used attention-based aggregators. The design of RGP is based on simulating the correct inference process of humans when facing similar problems. The experiments show RGP out-performing evaluation benchmarks in MIL and works similar to supervised models for pain estimation.", "questions": "- My main suggestions for the authors is to address the writing in the paper and to highlight their contributions accordingly. The results looks very compelling, but the paper is quite difficult to follow to appreciate the design or importance of the method. \n- Other minor changes include: captions for standalone clarity, dataset details for reproducibility and clear understanding. ", "limitations": "The authors have not mentioned any possible limitations or potential negative societal impact of this work. I would suggest the authors to include some details in settings where this method may not be useful. ", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "- The aggregator introduced in this work is grounded in prior knowledge of Multiple Instance Learning task construction. As a result, rather than opting for a complex black box approach, the design remains simplistic.This work aligns with recent research trends that emphasize leveraging simple yet valuable prior knowledge and inductive biases to design more effective solutions.\n- Regressor-Guided Pooling (RGP), outperforms all MIL evaluation benchmarks including SOTA by a large margin (Table 1) and also works on-par with supervised models for pain estimation (Table 4). This compelling result clearly highlights the potential of the introduced method.\n- The code available has a well-written README and looks easy to reproduce.", "weaknesses": "- **Missing dataset details**: The details of dataset(s) construction for MIL in pain-estimation setting is missing. One of the contributions mentioned in this work is RGMIL being the first weakly supervised deep model for pain estimation, so I feel it is important for this work to also focus on pain estimation datasets and benchmarks.\n    - Firstly, it is not clearly mentioned whether the results presented in section 4.5 is for UNBC pain dataset. The way the dataset is constructed for the MIL setting is also not presented which makes it difficult to understand if the dataset is different or the same as supervised setting. \n    - Secondly, the result is presented on only a single dataset. Given that the contributions of the work also focuses on being the first weakly supervised model for pain estimation, it is important to benchmark on more datasets (for example BioVid dataset presented in [25]).\n\n\n- **Issues in paper structure and writing:** The paper lacks a cohesive flow, making it difficult to follow. The writing style is inconsistent both within the sections and throughout the entire document. This paper needs major restructuring and rewriting to be easy to follow and understand.\n    - The main paper contains many non-essential details and lacks conciseness. Few suggestions improve it: the assumptions section (3.1) should be condensed into smaller paragraphs, with more extensive details provided in the supplementary material. Similarly, sections 4.2, 4.3, and 4.4 are verbose and could benefit by retaining crucial details in the main paper and moving the remaining content to the supplementary.\n    - In its current stage, some important contributions are left behind in supplementary (A1, A2) which supports the method’s effectiveness.\n    - Figure and table captions (except Table 1) are not self-sufficient and needs referencing to the text for understanding. Authors should revise the captions for better stand-alone clarity.\n    - Figure 1(b) is not clearly visible or understandable.\n\n\n- **Missing important citations and explanations:** There are important statements without citations or explanations, based on which the paper makes assumptions for model design (lines 143-144) and presents results on a different dataset (lines 230-231).\n    - line 143-144: 'Experience has shown that when we add the aggregator ⇢ to the backbone + regressor model for MIL problems, the instance-level performance often decreases significantly.' I would suggest the authors to cite works that show that adding aggregator decreases instance-level performance. If it is by observation across paper, the authors should present this in a table or a figure to give strong basis for assumptions.\n    - line 230-231: 'Considering multiple factors, the UNBC pain dataset isn’t really a clear and flexible enough choice to present a demonstration.' What are the multiple factors here is not clear and I would suggest the authors to explain clearly.\n\n\n\n- **Little to no focus on second contribution:** The paper's focus on the first contribution is appreciated, but it leaves little room to highlight the second contribution, leading to confusion in understanding the relevance of pain estimation in this work. It was referenced multiple times but only given a limited space (10 lines: 326-336) in the overall paper. To improve clarity, the authors can choose to implement either of these two suggestions:\n    - Prioritize RGP as the main contribution and dedicate the paper to thoroughly highlighting its significance, while briefly mentioning pain estimation as a valuable benefit of the method only shortly. This also relieves the authors of having to benchmark any other datasets while fully highlighting the method and its effectiveness.\n    - Condense some verbose sections, to be more concise, and move extra details to supplementary. This ensures that the authors have more space to present the benchmark results on pain estimation.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for appreciating the effectiveness and novelty of our approach. \n\nComment-1: About writing and descriptions. \n\n[R] Thanks for your suggestions, and we will revise our writing so that the proposed method can be easily understood. \n\nComment-2：Missing dataset details\n\n[R] We construct the UNBC dataset for MIL as follows: we selected consecutive 64 frames in the video as a training bag. To increase the amount of data, we used a sliding window approach with a step size of 8 to produce training bags. Overall, each frame was sampled as an instance in different bags 8 times. In each fold of the experiment, the total number of training bags in the constructed MIL dataset averaged around 6000. Detail will be added into the article.   \n\nIn experiments of pain estimation, only UNBC dataset is used, because we only obtain this one public dataset. For BioVid, before, we have sent our application for downloading this data, but not receiving any replies. At present we have another pain dataset collected by Xi Jing Hospital, but this cannot be made public until now.\n\nAdditionally, we supplemented experiments on SIVAL[1] and new MMNIST for general multi-class classification, shown in PDF of global rebuttal.\n\nComment-3：about line 143-144 \n\n[R] Compared with the fully supervised mode, the available information obtained by MIL is limited. Therefore, the limit of the instance-level performance of MIL should be equivalent to the fully supervised mode. Two main architectures of MIL, introduced in section 2.1, differ from the fully supervised mode only on the presence of the aggregator. We think that the inability of MIL to achieve the performance of the fully supervised mode is due to the limitation of the learning stage of the new component (aggregator). This can be verified by the results of MMNIST presented in Table 2. As seen in the first column in Table 2, when we implement experiments with the current mainstream models under the mode 10/1 (train/test bag size), the performance is obviously worse than 1/1 (fully supervised) mode. This illustrates that the current aggregators are unable to obtain enough accurate information for the aggregation from the 10 instances, and it does not surpass the information obtained from a single instance in the fully supervised mode. This leads to inadequate performance.\n\n\nComment-4：about line 230-231  \n\n[R] \n\n- The UNBC dataset is of extremely class imbalance, and it’s not normal for MIL problem. For UNBC, the four metrics presented in Table 4 are usually used to validate the performance, since the accuracy in the UNBC dataset is easily inflated and is not reliable due to the imbalance distribution of data. Whereas, accuracy is the most intuitive and reliable metric, which we strongly prefer to use.\n\n- In experiments of UNBC dataset, 25 folds cross-validation is widely used to test the performance. It takes over a week to obtain one overall evaluation result, which reduces flexibility during the demonstration process and prevents conducting comparative experiments with multiple aggregators under different modes. \n\n- Both single-frame image and the feature extractor itself are large, and during training, increasing the bag length easily leads to insufficient GPU memory, limiting the experimental scenarios. \n\nComment-5: Little to no focus on second contribution \n\n[R]  Thanks for your suggestion. We agree with your concern that the contributions we presented is indeed not clear enough. Different from the mainstream MIL methods, RGMIL addresses the general multi-class classification problem in MIL scenarios by directly learning discriminative instance-level features. \n\nInitially, we try to design a pain estimation method from the view of MIL. In studying process, we find that it places strong emphasis on single-frame predictions during testing for pain estimation. However, most of existing MIL methods focus more on learning the aggregator black box itself, rather than learning single-frame features, meanwhile ignoring the instance-level performance. We also found that most MIL methods mainly focused on the binary-classification problem, little considering the multi-classification problem. \n\nMotivated by limitations of existing MIL methods, we designed RGMIL that explicitly enhances instance-level performance by transferring the learning task completely to the instance feature extraction stage through a special aggregation method. It is theoretically proved (in section 4.3 of paper and section A.1 of supplementary) that it is beneficial to improve instance-level performance. RGMIL emphasizes generality, and it is not used only on pain estimation but also other multi-classification problems. In our work, we validate the performance on benchmark datasets, MMNIST series, and a real application (the pain dataset, UNBC).\n\nComment-6: Limitation\n\n[R]Feasibility of training at instance-level with excessively large bag-lengths: RGMIL significantly improves instance-level performance than current methods. However, as described in section A.1 of supplementary, we think that MIL problems may have ambiguity inherently in extreme scenarios. It isn’t theoretically insured whether training instance feature extraction directly is feasible in extreme cases. As shown in Table 1, with a maximum of bag length 512, instance performance decays with increasing bag lengths. Due to the memory limitation, we did not achieve the experiment on longer bag-lengths (like several thousands) to confirm the feasibility. Thereafter, we will try our best to figure it out. \n\nMemory limitation: It is challenging to handle the excessively large bags when the feature extractor is involved in the training. For example, in the MMNIST experiment (with a single bag of length 1024), there is insufficient memory on an NVIDIA 4090 graphics card.\n\n[1]Rahmani, Rouhollah, et al. \"Localized content based image retrieval.\" Proceedings of the 7th ACM SIGMM international workshop on Multimedia information retrieval. 2005."}, {"review_id": "tPHR4AmIa7", "reviewer": "Reviewer_mChi", "summary": "This paper presents a new model, the Regressor-Guided Multiple Instance Learning network (RGMIL), which addresses the challenge of providing discriminative instance-level representation in multi-classification MIL scenarios. It introduces an aggregator, the Regressor-Guided Pooling (RGP), that enhances the instance-level performance of Multiple Instance Learning (MIL) tasks. RGMIL outperforms existing methods in various datasets and demonstrates the potential for instance-level predictions.", "questions": "See the weaknesses.", "limitations": "See the weaknesses.", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1.  The paper presents a novel model, the Regressor-Guided Multiple Instance Learning network (RGMIL), which brings a fresh perspective to the field of multi-classification MIL scenarios. \n2. The paper is well-structured and the problem and proposed solution are explained clearly. \n3. The experiments are well-designed and the promising results effectively support the authors' claims.\n4. The paper is inspiring for future research, including potential applications for multi-classification tasks such as the UCF-Crime dataset.", "weaknesses": "1. The paper lacks a clear definition and purpose for the metrics used in Table 4. Additionally, a thorough analysis of the results presented in the table is missing.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": " Thank you a lot for appreciating our efforts in addressing the use of multiple-instance learning (MIL) for instance-level predictions.\n\nComment-1: Weaknesses\n\n[R]The detailed information of the metrics can be found in section C in supplementary. We will give more details on the UNBC experiments and the additional information for Table 4. All the results regarding MMNIST and benchmark tests in the paper can be reproduced within 3 hours. Besides, the result on UNBC can also be reproduced in days.\n\nAdditionally, we provided supplementary results on two general multi-instance multi-class image datasets: SIVAL[1] and the new MMNIST series experiments, as shown in the PDF in global rebuttal. We next try to test on image classification datasets next, like video anomaly datasets and more general multi-class scenarios. \n\n[1]Rahmani, Rouhollah, et al. \"Localized content based image retrieval.\" Proceedings of the 7th ACM SIGMM international workshop on Multimedia information retrieval. 2005.\n"}, {"review_id": "fQmigfJev8", "reviewer": "Reviewer_oDjC", "summary": "This paper tries to overcome the drawback of low instance-level prediction performance  in  the existing Multiple Instance Learning (MIL) models. According to the authors, the low instance-level performance is because the existing techniques focus mostly on analyzing the relationship between instances and aggregating them while ignoring learning of effective instance-level representation. To this end, this paper proposes a novel aggregator function called Regressor-Guided Pooling (RGP) that directly learns discriminative instance-level representation by mimicking human inference process. An extensive evaluation is performed using the multiple MIL datasets along with the pain estimation datasets to validate the effectiveness of the proposed aggregator function. ", "questions": "1. How effective the proposed technique is in the video-anomaly detection tasks compared to competitive baselines?", "limitations": "See weaknesses", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. This paper has made a novel contribution to MIL field by identifying and addressing  key limitations of the proposed MIL techniques. The significance of the novel contribution is demonstrated through non-trivial performance gain  over other competitive baselines.\n2. The motivation for introducing the Regressor-Guided Multiple Instance Learning (RGMIL) framework is very novel, natural, and intuitive.\n3. This paper deals with the multi-classification scenario under the MIL setting which is unique and different from most of the MIL works that focus mostly on the binary-classification scenario. This unique scenario also helped to enhance the novelty of this paper.\n4. The extensive evaluation is conducted considering multiple MIL datasets. In addition to the quantitative result, the authors have done great job explaining why the proosed technique works and how it avoids the limitations of the existing MIL techniques. ", "weaknesses": "1. Most of the real-world video anomaly datasets are also used as a crucial testbed for the evaluation of MIL models [1, 2, 3].  Specifically, under video anomaly detection task, the MIL model is trained with video-level annotations (without having explicit access to the frame-level annotation). During the prediction process, the trained MIL model is used to perform frame-level prediction. The evaluation of the proposed MIL model on video-anomaly detection task is missing. I wonder how effective the proposed technique is in terms of making frame-level predictions. Also, in the related work section, the authors may need to explain how MIL techniques aimed to solve the video anomaly detection task are different from their work. \n2. Some of the figures deserve better treatment. For example, the caption in Figure 1 is not very descriptive. Further,  modules in the figures (with texts) are difficult to read and understand. \n3. The intuition behind Equation 6 is difficult to understand and is not very clear. \n\n\nReferences:\n[1]. Sultani et al. “Real-world Anomaly Detection in Surveillance Videos”. CVPR2018.\n[2]. Sapkota et al. “Distributionally Robust Optimization for Deep Kernel Multiple Instance Learning”. AISTATS2021.\n[3]. Tian et al. “Weakly-Supervised Video Anomaly Detection With Robust Temporal Feature Magnitude Learning”. ICCV2021.\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "\nThank you a lot for appreciating our efforts in addressing the use of multiple-instance learning (MIL) for instance-level predictions. \n\nComment-1: Weaknesses (about applications)\n\n[R] In our method, RGP enhances the instance-level performance explicitly by transferring the learning task to the instance feature extraction stage through a special and simple aggregation method. This is theoretically proven in Section 4.3 of the main paper and Section A.1 of the supplementary material. RGP does not conflict with the current MIL-based video anomaly detection models. With the involvement of the instance-level feature extractor in training, RGP can theoretically achieve better instance-level performance than the existing ABP series aggregators. This work emphasizes generality and is applicable not only to pain estimation but also to other multi-classification problems. In the field of video anomaly detection, we can directly apply RGMIL for anomaly detection or simply replace the existing aggregators in other models with RGP to ensure instance-level performance. \n\nBesides, there are some similarities and differences worth noting between our experiments and video anomaly detection problems. \n\nMain similarities: \n\n- Both are image classification problems, may related to multi class;\n\n- Both emphasize instance-level representation;\n\n Main Differences: \n\n- Our experimental scenario is related to the maximum label, but video anomaly detection typically does not involve the concept of a maximum label. Even in the case of multi-classification, there is no inherent ordering between class labels. \n\n- Some video anomaly detection datasets may provide videos that contain a large number of frames, potentially reaching thousands. Training single-frame features on such large videos may lead to insufficient GPU memory. \n\nComment-2: Weaknesses (about figures)\n\n[R] Thanks a lot for your suggestion. We will revise these figures and descriptions so that they are more readable and easily understood. \n\nComment-3: Weaknesses (about Equation 6)\n\nAs a non-parametric dynamic aggregate component, RGP still share the same view with attention mechanism that weights should rely on the feature itself. It is also reasonable to order the instances’ importance by their classification score that indicates the likelihood of being positive (critical instances) and assign weights accordingly. We could process the current instance classification score provided by regressor in different ways. We tested different combinations of regression methods and weight acquisition methods: \n\n- the output score is in the form of 1-dimensional logits output with a sigmoid function, and the weights are obtained based on the logits output; \n\n- the output score is in the form of 2-dimensional logits output with a softmax function, and the weights are obtained based on the high-bit output of logits; \n\n- the output score is in the form of 2-dimensional logits output with a softmax function, and the weights are obtained by dividing the high-bit output of logits by the low-bit output; \n\n- the output score is in the form of 2-dimensional logits output with a softmax function, and the weights are obtained by subtracting the low-bit output of logits from the high-bit output. \n\nThese designs have the same underlying principles but differ in real numerical computations. We found that the last method had significantly better performance, and then we used and formulated it as Equation 6 in this paper. In practice, the sigmoid output barely works. But as shown in section A.2 of the supplementary material, there are other methods that also works without Equation 6. \n\nThe method of normalization also affects the performance, and we tested parameterized normalization and unnormalized methods, both of which were not as good as simple unparameterized normalization. The two equations are only training tricks that make RGP more practical. \n\nComment 4: Questions \n\nThank you a lot for your suggestions. Due to the time limitations, we did not add this experiment on the dataset of video anomaly detection in this version, but we add experiments on two general multi-instance multi-class image datasets: SIVAL[1] and the new MMNIST series, as shown in the PDF in global rebuttal. Anomaly detection is not our major research interests, but we are also interested in the results for video anomaly detection problems. The experiments on anomaly detection will be very meaningful, and we will try to test it later. \n\n [1] Rouhollah Rahmani, et al. \"Localized content based image retrieval.\" Proceedings of the 7th ACM SIGMM international workshop on Multimedia information retrieval. 2005.\n"}, {"review_id": "IiXD5daik8", "reviewer": "Reviewer_A2Ve", "summary": "The paper presents a new approach for Multiple Instance Learning (MIL) called Regressor-Guided MIL (RGMIL). The contribution of RGMIL is a new aggregation approach Regressor-Guided Pooling (RGP). In this approach, the aggregation part of the MIL model is split into N branches (for N + 1 classes, assuming class 0 is a negative class), where each branch is responsible for classifying a single class. The model is applied to a range of binary MIL problems (MUSK1, MUSK2, FOX, TIGER, ELEPHANT, Webpages, 20NewsGroups, Messidor, and UCSB Breast), and two max-based multi-class problems: MMNIST (proposed in this work) and UNBC shoulder pain estimation. The model mostly outperforms existing methods, and analysis is conducted as to why RGP is an effective pooling strategy. ", "questions": "1. Why does each branch of the aggregator produce two outputs? I can see the two outputs are used to calculate a difference in Equation 6, but don't understand why a single output with a sigmoid activation and a threshold of 0.5 isn't used. This needs to be explained in more detail.\n2. If my understanding is correct, the regressor is used to calculate some form of attention weights (output of equation 8) and also classify the re-weighted instance representations. Is that correct? If so, what is the relationship to other MIL attention methods? It seems somewhat similar to the Additive MIL approach [1], which is not cited.\n3. Is each branch regressor assigned random weights that are then fixed during training of the feature extractor? If so, what is the motivation behind this?\n4. How would this approach work on multi-class problems that don't have some form of max-based ranking, e.g., SIVAL [2] or 4-MNIST-Bags [3]? My confusion lies in what happens when two or more branches all have positive indicator vectors $\\hat{Y}$. In the max-based datasets used in this work, the maximum branch is taken as the overall prediction (equation 13). However, I'm not sure how this would work for other types of multi-class MIL datasets that don't use max-based classes. This limits the significance and scalability of the work.\n\n[1] Javed, Syed Ashar, et al. \"Additive MIL: intrinsically interpretable multiple instance learning for pathology.\" Advances in Neural Information Processing Systems 35 (2022): 20689-20702.  \n[2] Rahmani, Rouhollah, et al. \"Localized content based image retrieval.\" Proceedings of the 7th ACM SIGMM international workshop on Multimedia information retrieval. 2005.  \n[3] Early, Joseph, Christine Evers, and Sarvapali Ramchurn. \"Model Agnostic Interpretability for Multiple Instance Learning.\" In International Conference on Learning Representations. 2022.", "limitations": "1. The work does not address the limitations of the novel method or suggest any areas for future work.\n2. The main limitation of the work is that it is very hard to follow and difficult to understand how the novel method works. This is frustrating as the results appear promising.\n3. One potential limitation, discussed in the questions above, is how this could be applied to multi-class datasets that do not follow some form of max-based assumption.\n\nDue to the difficult in understanding the approach and the lack of clarity in the method, I am leaning towards rejection as I don't think the method is reproducible from what is presented. However, there is certainly promise in the approach and the results appear strong.", "rating": 3, "confidence": 4, "soundness": 3, "presentation": 1, "contribution": 2, "strengths": "**Originality**\n1. The RGB pooling is an original approach, particularly with the double pass through the regressor.\n2. Section 4.3 is also a novel investigation into the bottlenecks of MIL model training and can help understand why certain models succeed/fail.\n\n**Quality**\n1. The evaluation covers a range of datasets and shows strong results, outperforming most other methods in most cases.\n2. An ablation study is used to show how performance changes when training on different sizes bags but only classifying single instance bags.\n\n**Significance**\n1. Given the strong performance of RGMIL, it would appear useful in MIL settings that use max-based problems, i.e., traditional binary MIL or problems like pain estimation.", "weaknesses": "**Originality**\n1. I believe some elements of the proposed method are already well known concepts, e.g., equation 4 appears to be one hot encoding, and equation 8 is a softmax. Simplifying these in the methodology and focusing on the novel parts of the method will help understanding. \n\n**Quality**\n1. Some elements of the approach are lacking justification (see questions below).\n2. The MMNIST dataset is proposed but then only evaluated in an Instance-level setting. A strong evaluation could involve looking at the performance when the training and test bag sizes are the same (i.e., 10/10, 16/16 etc.) as well as the instance-level setting.\n\n**Clarity**\n1. I found the paper very hard to follow and it took a considerable effort to unpick what the proposed method was actually doing.\n2. It is not clear quite how the method works, particularly with regards to training. In Section 3.2, I am puzzled by the statement *\"Parameters in both the RGP and regressor will NOT be involved in gradient descent\"*. Please see questions below.\n2. The work mentions how CNNs work and how humans would approach a MIL problem as justification for their approach. This makes the narrative harder to follow and detracts from the design of the novel pooling mechanism. I think this comparison would be best placed in the appendix, with more space allocated to discussing the model architecture and explaining how it works.\n3. Figure and Table captions often contain little information, e.g., Table 2 does not state which dataset the model is trained on nor what the different columns means, Figure 2 should have y-axis labels, etc.\n4. Some of the mathematical notation should be revisited to aid understanding. For example, is it necessary to have $M$ as the total number of instances and then $t$ as the number of instances per bag? Equations 5 through 9 jump through different notations, e.g., $fs$ for representations, then $H$, then $F_i$.\n5. I would suggest another thorough proof-read to remove grammatical errors that make the work hard to follow.\n\n**Significance**\n1. My understanding is that this approach is only applicable to multi-class problems that have some max-based ordering of classes, e.g., in pain estimation, if branch 1 and 2 are both positive, then the prediction is class 2. I don't know how this would work for general multi-class problems (see questions below), which limits its significance.\n2. I disagree with the final paragraph of Section 2.2 (comparing t < c and t > c). Instance-level performance is important for interpretability in both cases - just because there are a greater number of instances per bag does not mean instance-level performance becomes less important. \n\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks a lot for your many valuable comments. It’s great that you appreciate our originality and investigation of the MIL model.\n\nComment 1：Weaknesses-Originality \n\n[R] We will revise the writing thoroughly according to all comments. \n\nComment 2: Weaknesses-Quality \n\n[R] We agree with the importance of bag-level performance. We have given additional general multi-classification experiments both on bag and instance level in global rebuttal(see PDF).  \n\nComment 3: Weaknesses- Clarity\n\n[R]: 1. We will revise our descriptions to make paper  more readable. \n\n2.More responses can be found in question parts.\n\n3.As a starting point and important inspiration for designing the pooling method, we mentioned CNN here. Details have been put into the supplementary.\n\n4.We have modified relevant tables and figures.  \n\n5.We have fixed inconsistencies in the mathematical notations. \n\n6. We will  correct grammatical errors as suggested. \n\nComment 4: Weaknesses- Significance \n\n[R]: 1. We have achieved experiments on general multi-class datasets (SIVAL and the new MMNIST series), and results are shown in the global rebuttal. \n\n2. We understand and agree with your thoughts about the instance-level importance. As the results on MMNIST show, the performance decays with the training bag length increases. In supplementary (section A.1), we argue that there is only one additional deterministic inference process when a human is faced with the MIL scenario. Any inference result that complies with the inference rule is legal. We assume that given the fixed total data records number, if the bag size is too large, there may be too many correct possibilities in a bag, which could cause instance-level performance degradation. This is an unsubstantiated idea and will be removed from the article.  \n\nComment 5: Question-1 \n\n[R] The detailed responses are shown in the global rebuttal. \n\nComment 6: Question-2 \n\n[R] Yes. Between ABP and RGP, there are some important similarities and differences. They share the same view regarding the reliance on feature weights when obtaining bag representations. This view has been validated in many MIL models. This is a significant similarity. \n\nBetween RGP and the ABP, the main difference lies in how to obtain the weights and the perspective on the MIL problem itself. The ABP series often train matrices (related to $Q$, $K$, $V$) to obtain weights, which is not very direct. In our perspective, instance-level performance is crucial, and we essentially adjust our bag-level judgment based on the current judgments of individual instances, as in human reasoning for MIL problem. In RGMIL, it believes that the solution to MIL problems should regress back to the instance level, and the criteria for judgment are the same on instance and bag levels, which is why we use the double-regressor scheme to pass the consistency. We believe that there is no need for a parameterized black-box model to simulate the reasoning process of a MIL problem. Simply asking for the current judgment from the instance-level regressor would be a more natural and intuitive way of aggregating features. \n\nRGP is non-parametric, and the learning process can entirely occur in instance feature extraction. Theoretical analysis in section 4.3 and section A.1 of supplementary confirms the advantage on instance level. In contrast, ABP does not take extra consideration on instance-level performance and the actual human reasoning process. This is the key difference between RGP and ABP. \n\nWe have added the cite of Additive ABP [1]. For additive ABP, it only changes the order of summation and the output logits to improve interpretability. Essentially, it is considered as a member of the ABP series. Its motivation and research are quite different from our method. \n\nComment 7: Question-3 \n\n[R] Yes. In classification model, it is expected that obtained features are of linear separability. For example, in the architecture of supervised model (feature extractor + linear regressor), when the parameters of linear regressor are fixed and only the feature extractor need be trained, a discriminative extractor can still be obtained. In MIL models, the requirement for the feature is still linear separability. In the case of multiple branches, it becomes a joint optimization problem, and fixing the parameters of the regressor makes more easily training without causing performance degradation. Fixing the parameters of the linear regressor does not affect overall performance, but the different random initialization strategies have a slight impact on performance. In RGMIL, we adopt the kaiming_normal_() random initialization for the parameters of the linear regressor (with a mean of 0). \n\nComment 8: Question-4\n\n[R] We added experiments on general multi-class problems (SIVAL and new MMNIST series) that don’t have form of max-based ranking, and details are given in global rebuttal. \n\nIn the general multi-class scenario, during training, all bits in the indicator vector are considered reliable, and all corresponding branches participate in the calculation of loss function. If multiple branches output 1 during test, the branches can be sorted by querying the softmax high-bit values of each branch as confidence values, and the branch with the highest value can be selected as the final output. While in the new experiments, we did not perform extra processing on the outputs. We only modified the loss function during the training phase. Theoretically, RGMIL is flexible and can be fine-tuned based on specific circumstances.\n\nComment-9: Limitation\n\n[R] Thanks, we will revise our description and add the analyses about the limitation. The contribution and additional experiments are shown in global rebuttal. All results presented are guaranteed to be reproduced on github. \n\n[1]R. Rahmani, et al., Localized content based image retrieval, in ACM SIGMM workshop, 2005. \n\n[2] J. Early, et al., Model Agnostic Interpretability for Multiple Instance Learning, in ICLR, 2022.\n\n"}]

Author rebuttal:
About RGMIL:

RGMIL can address the general multi-class classification problem in MIL scenarios by learning discriminative instance-level features. It holds a different view that we can solve MIL problems by learning on instance level like human does. Inherently coming with interpretability, RGP significantly enhanced instance-level performance by transferring the learning task to the instance feature extraction stage through a special and simple aggregation method. The advantage is theoretically proven in Section 4.3 of the main paper and Section A.1 of supplementary material. When there is emphasis on single-instance predictions, RGP can theoretically achieve better instance-level performance than the existing aggregators. This work emphasizes generality and is applicable not only to pain estimation but also to other general multi-classification problems. 

About Equation 6:

As a non-parametric dynamic aggregate component, RGP still share the same view with attention mechanism that weights should rely on the feature itself. It is also reasonable to order the instances’ importance by their classification score that indicates the likelihood of being positive (critical instances) and assign weights accordingly. We could process the current instance classification score provided by regressor in different ways. We tested different combinations of regression methods and weight acquisition methods:

1.the output score is in the form of 1-dimensional logits output with a sigmoid function, and the weights are obtained based on the logits output; 

2.the output score is in the form of 2-dimensional logits output with a softmax function, and the weights are obtained based on the high-bit output of logits;

3.the output score is in the form of 2-dimensional logits output with a softmax function, and the weights are obtained by dividing the high-bit output of logits by the low-bit output;

4.the output score is in the form of 2-dimensional logits output with a softmax function, and the weights are obtained by subtracting the low-bit output of logits from the high-bit output.

These designs have the same underlying principles but differ in real numerical computations. We found that the last method had significantly better performance, and then we used and formulated it as Eq.6 in this paper. In practice, the sigmoid output barely works. But as shown in section A.2 of supplementary material, there are other methods that also works without Eq.6.

The method of normalization also affects the performance, and we tested parameterized normalization and unnormalized methods, both of which were not as good as simple unparameterized normalization. The two equations are only training tricks that make RGP more practical.

About supplementary experiments：

As requested, we additionally conducted experiments on two general image multi-classification datasets as supplements in the PDF: 

- Table 1 and Figure 1 in the PDF are the MMNIST series instance-level results in the main paper. According to suggestions, we improved the presentation of charts and titles. In this series of experiments, for the MMNIST dataset, the label is determined by the maximum value in the input bag, and the model's output during test is obtained by selecting the branch with the highest position among all branches with an output of 1.   

- Table 2 is the bag-level performance reports on SIVAL dataset. SIVAL consists of 25 classes of complex objects photographed in different environments, where each class contains 60 images. Each image is segmented into approximately 30 segments, and each segment is represented by a 30-dimensional feature vector that encodes information, such as the segment’s colour and texture. The segments are labelled as containing the object or containing background. We selected 1,310 classes as positive classes, and randomly sampled from the other classes as the negative class. We set up three Linear+ReLU blocks as feature extractors. The test scenario involves image classification problems ranging from binary to 11-class classification. Each image is treated as a bag, resulting in bag-level performance evaluation. Considering that the classes in the SIVAL dataset do not have any ranking relationship, all bits of the output indicator vector are reliable, so all branches are involved to the computation of loss function. Continuing with the current version's approach, the model's output during test is still obtained by selecting the branch with the highest position among all branches with an output of 1. Despite learning instance-level features only, RGP still exhibits impressive performance on bag level. 

- Table 3 is the instance-level performance reports on MMNIST. We adopted a different approach in constructing the dataset this time: while still using the 4-branch RGMIL, we selected images with labels ranging from 0 to 6 as the negative class (background class), and images with labels 7, 8, and 9 as three different positive classes. Similar to SIVAL, there is no sequential relationship among the different positive class labels. Each positive class and the negative class account for approximately 25% of the dataset. Within one positive bag, the number of images of positive class is approximately 10%. Like to the SIVAL tests, during training, all bits of the output indicator vector are reliable, so all branches are involved to the computation of loss function. Continuing with the current version's approach, the model's output during test is still obtained by selecting the branch with the highest position among all branches with an output of 1. Unlike the original MMNIST series, the convergence speed of different methods varies much. For example, it's very difficult for MAX and DSP to converge even training for hundreds of epochs. Thus, we present the average of 10-time test accuracy is taken for each model after convergence or training after 100 epochs. RGP still shows dominance in the scenario on instance level. 

Meta-review: This paper received three positive ratings (5,6,7) and one negative rating (3). Given the lack of consensus amongst the reviewers, the AC examined the (shared) concerns. Overall, the reviewers find the paper interesting and with strong performance. The main open concern as expressed by the negative reviewer is the general applicability of the problem in MIL. The authors have provided extra instance- and bag-level experiments. The instance-level results are promising, while competitive results are obtained for the bag-level experiments. Since the method is mostly designed for instance-level performance, the AC deems that the authors have sufficiently addressed the concerns of the reviewer. The AC therefore recommends acceptance.
Decision: Accept (poster)