# HLFSRNN-MIL: A Hybrid Multi-Instance Learning Model for 3D CT Image Classification. _Applied_
## 3 Proposed Methods
### HLFFF Method for Processing High-Frequency and Low-Frequency Features  
For the MIL problem parameterized by the neural network, \(f(\cdot)\) is used as a feature extractor. This paper proposes an HLFFF method that combines the basic feature extraction network with Hilo attention [40], as shown in Figure 3. In the face of the complex problems of high-resolution images, the semantic information of features can be re-added from the perspective of high frequency and low frequency. This section will explain the HLFFF method with the feature extraction network based on ResNet-50. The output vector \(M\in R^{N\times D}\) from the last residual block of ResNet-50 is preserved. Here, \(M\) represents the feature matrix acquired from each instance under identical conditions, where \(N\) denotes the number of instances in the package, and \(D\) signifies the size of the hidden dimension. Subsequently, dimensionality reduction is applied to the feature vector of each instance, followed by global average pooling over a specified window  
Figure 3: Overview of the proposed HLFFF method. size. The data \(M\) before pooling and the data \(MP\) after pooling are utilized as inputs for high-frequency attention and low-frequency attention. Initially, leveraging the multi-head self-attention mechanism (MSA) [41], \(h\) self-attention heads are designated, each assigned with Query matrices (\(Q\)), Key matrices (\(K\)), and Value matrices (\(V\)):  
\[Q_{hi}=M_{k}W_{q}^{h},K_{hi}=M_{k}W_{k}^{h},V_{hi}=M_{k}W_{v}^{h} \tag{4}\]  
\[Q_{lo}=M_{k}W_{q}^{h},K_{lo}=M_{k}W_{k}^{h},V_{lo}=M_{k}W_{v}^{h} \tag{5}\]  
where \(hi\) and \(lo\) denote the high-frequency and low-frequency attention branches, respectively. \(W_{q}^{h},W_{k}^{h},W_{v}^{h}\in R^{D\times D_{h}}\) are the attention weights that \(QKV\) can learn. The input for \(K_{lo}\) and \(V_{lo}\) is \(MP\), where \(D\) represents the number of hidden weight matrices, and \(D_{h}\) is the number of hidden weight matrices corresponding to each self-attention head. Subsequently, the weighted sum of the output vectors from each head is computed:  
\[SA_{hi}^{h}(M_{k})=Softmax\Bigg{(}\frac{Q_{lo}K_{li}^{T}}{\sqrt{D_{h}}}\Bigg{)} V_{hi} \tag{6}\]  
\[SA_{lo}^{h}(M_{k},MP_{k})=Softmax\Bigg{(}\frac{Q_{lo}K_{lo}^{T}}{\sqrt{D_{h}}} \Bigg{)}V_{lo} \tag{7}\]  
Then, combine the output of each head:  
\[Hi(M)=concat\Big{[}SA_{hi}^{h}(M_{k})\Big{]}W_{o} \tag{8}\]  
\[Lo(M)=concat\Big{[}SA_{lo}^{h}(M_{k},MP_{k})\Big{]}W_{o} \tag{9}\]  
In the formula, \(W_{o}\in R^{(N_{h}\times D_{h})\times D}\) denotes a linear projection layer for series output. Here, \(N_{h}\) represents the total number of self-attention heads in this layer, harmonizing the dimensions of output and input. In this process, a hyperparameter \(\alpha\in[0,1]\) is defined to regulate the number of self-attention heads participating in high frequency and low frequency, respectively. \(\alpha\times N_{h}\) denotes the number of self-attention heads assigned to high frequency, while \((1-\alpha)\times N_{h}\) denotes the number of self-attention heads allocated to low frequency. Depending on the nature of the problem at hand, the value of \(\alpha\) can be adjusted to prioritize either high frequency or low frequency, thereby potentially reducing computational complexity to some extent. High-frequency attention (\(Hi\)): The intuitive concept here involves utilizing high-frequency components to encode local details of the target. This is achieved by capturing high-frequency information through the establishment of a local self-attention window (for example, a 2 \(\times\) 2 window). Low-frequency attention (\(LF\)): Each window employs average pooling to acquire down-sampled feature information \(MP\), which is then mapped to \(K_{lo}\) and \(V_{lo}\), while \(Q_{lo}\) utilizes the original feature information \(M\). This approach not only captures abundant low-frequency information but also alleviates the complexity of Formulas (5) and (7).