# HLFSRNN-MIL: A Hybrid Multi-Instance Learning Model for 3D CT Image Classification. _Applied_
## 3 Proposed Methods
### SRNN Method for Processing Instance with Temporal Relationships  
In the classic MIL scenario, there is a fundamental assumption that each instance within a package is independent of one another. However, when a patient's 3D CT image is employed as a MIL package, there exists a contiguous relationship between each slice. In this context, the target lesion at the same position might consecutively appear across several adjacent slices. Given that these lesions actually pertain to the same morphologically similar structure, we propose the utilization of a recurrent neural network to process a certain length of the instance sequence. In doing so, it is crucial to maintain the spatial order of instance upon input. A sequence of sequentially continuous instances is termed a sequence.  
The significance of the SRNN method lies in its ability to identify an instance similarity vector that aligns with the target variations within a sequence. However, the uncertainty regarding the number of instance \(k\) in each package presents a challenge. One existing approach is to employ a fixed-length sequence template. In this method, the length of the template must exceed the length of each package used. When the size of the package falls short of this length, the vacant portion is filled with zeros. However, this approach gives rise to two issues. Firstly, it can lead to inefficient utilization of computational resources when dealing with packages containing a small number of instances (with the largest package approximately 10 times the size of the smallest package). Secondly, the padding of zeros may degrade the learning performance of the network, particularly in cases of highly unbalanced data.  
This paper proposes to employ a sliding window \(w\) to encode sequential packages. The selection of \(w\) entails additional configuration, yet within packages of varying sizes, \(n\) encoded sample fragments emerge:  
\[n=k-w-1 \tag{11}\]  
After such processing, the sequence length of each input to the recurrent neural network equals \(w\), resulting in:  
\[Fr=Se\Big{(}HIFF(M)^{T}\Big{)} \tag{12}\]  
Here, \(Se\) is a sliding encoder, and then the recurrent neural network uses a long short-term memory (LSTM) network. The input is a feature map obtained by the HLFFF method, as shown in Figure 4. Here, the \(n\)-th instance sequence obtained by \(Se\) in order is considered to be the state at time \(t\), and the gated unit at time \(t\) is calculated by referring to the mathematical calculation formula of LSTM:  
\[i_{n_{t}}=\sigma(W_{i}Fr_{n_{t}}+U_{i}h_{t-1}) \tag{13}\]  
\[f_{n_{t}}=\sigma\Big{(}W_{f}Fr_{n_{t}}+U_{f}h_{t-1}\Big{)} \tag{14}\]  
\[o_{n_{t}}=\sigma(W_{o}Fr_{n_{t}}+U_{o}h_{t-1}) \tag{15}\]  
\[\widetilde{c}_{t}=tanh(W_{c}Fr_{n_{t}}+U_{c}h_{t-1}) \tag{16}\]  
\[c_{t}=i_{n_{t}}\odot\widetilde{c}_{t}+f_{n_{t}}\odot\widetilde{c}_{t-1} \tag{17}\]  
\[h_{t}=o_{n_{t}}\odot tanh(c_{t}) \tag{18}\]  
Here, \(\sigma\) represents the sigmoid function, while \(W_{i}\), \(W_{f}\), \(W_{o}\), \(U_{i}\), \(U_{f}\), and \(U_{o}\) denote the weight matrices that can be learned within the LSTM. \(Fr_{n_{t}}\) signifies the sequence vector of the \(t\)-th time step of the \(n\)-th sequence, \(h_{t}\) denotes the hidden state at the \(t\)-th time step, and \(c_{t}\) represents the memory unit at the \(t\)-th time step. Particularly, \(W_{c}\) represents the weight matrix employed to calculate the memory unit required for the next sequence. It is utilized to map the hidden state \(t-1\) of the current time \(t\) input \(Fr_{n_{t}}\) and the previous time \(t-1\) to the memory unit \(c_{t}\) needed for the subsequent sequence.