# Combining the Transformer and Convolution for Effective Brain Tumor Classification Using MRI Images
## 3 Research Methodology
### Intelligent Merge Module  
In the IMM, the embeddings of the transformer patch are combined with the CNN features maps and given back to the CNN feature extractor. The IMM makes an effort to make the integration process intelligent so that can gather and choose the most useful data for prediction. The IMM has two inputs, namely \(I^{c}_{input}\) (B \(\times\) C \(\times\) H \(\times\) W) and \(I^{F}_{input}\) (B \(\times\) C \(\times\) H \(\times\) W), as illustrated in Figure 7. \(I^{c}_{input}\) originates from the CNN stage, whereas \(I^{c}_{output}\) comes from the FFM. There are many global features in \(I^{F}_{input}\), which is identical to \(F^{c}_{output}\) of the FFM. Here, we construct dependencies between the channels of \(I^{F}_{input}\) in order to identify the most useful global characteristics and use these features for aggregation. Figure 7 illustrates how the descriptor \(D_{3}\) of size, B \(\times\) C \(\times\) 1 \(\times\) 1 reflects these dependencies. C represents the number of channels of \(I^{F}_{input}\). Figure 6: The proposed FFM structure. Figure 7: Representation of the IMM structure. Each channel of \(D_{3}\) only has one pixel, and the value of each pixel indicates the significance of the corresponding channel of \(I_{input}^{F}\). To accentuate informative channels and reduce less informative channels, \(I_{input}^{F}\) is multiplied by \(D_{3}\). Prior to obtaining \(D_{3}\), global average pooling is used to combine the spatial information of \(I_{input}^{F}\) and produce descriptor \(D_{1}\):  
\[D_{1}^{k}=[\sum_{i=1}^{H}\sum_{j=1}^{W}\ (I_{input}^{F})^{k}\ (i,\ j)]/\mathrm{H}/ \mathrm{W} \tag{5}\]  
where \(D_{1}\) and \(D_{3}\) both have the same size. Here, \((I_{input}^{F})^{k}\ (i,j)\) specifies the pixels on the kth channel of \(I_{input}^{F}\), while \(D_{1}^{k}\) represents the k-th element of \(D_{1}\). A descriptor is obtained and the representation of global features is strengthened by the pooling of global averages. Next, we condense \(D_{1}\) using a completely connected layer \(L_{1}\),  
\[D_{2}=L_{1}\ (D_{1}) \tag{6}\]  
where \(D_{2}\in\mathrm{B}\times(\mathrm{C}/\mathrm{r})\times 1\times 1\), \(\mathrm{r}\) is the compact ratio (set in this study to 16), and another fully linked layer, \(L_{2}\), is used to restore the channel number and make use of the data embedded in \(D_{2}\) once \(D_{2}\) has passed through a ReLU layer:  
\[D_{3}=\sigma\ (L_{2}\ (D_{2})) \tag{7}\]  
where the sigmoid function is represented by \(\sigma\). In the compact-restore process, the values of the items in \(D_{3}\) change dynamically during training, and \(D_{3}\) can pick up on the dependencies between the channels of \(I_{input}^{F}\). Here, \(I^{F}\) is generated by scaling \(I_{input}^{F}\) with \(D_{3}\) in the manner described below:  
\[I^{F}=I_{input}^{F}.D_{3} \tag{8}\]  
Channels that provide more useful information are further highlighted, and the feature representation of \(I^{F}\) is enhanced, both of which can be attributed to descriptor \(D_{3}\). The CNN feature extractor uses the pre-trained model to acquire input C, which includes class-specific data. The input, on the other hand, contains more local features than global feature representations, according to CNN's fundamental structure. We add these global features to input C via aggregation because \(I^{F}\) is transformed from transformer PE and is rich in global features. As a result, the representational power is increased and the proposed TECCN performs better. The output of IMM is the weighted sum of \(I\) input C and \(I^{F}\). \[\left(I_{output}\right)^{i}=\alpha^{i}.(I_{input}^{C})^{i}+\ \beta^{i}.\ (I^{F})^{i} \tag{9}\]  
where \((I_{output})^{i}\) stands for the feature map on the \(i\)th channel of \(I_{output}\), \((I_{input}^{C})^{i}\) and \((I^{F})^{i}\) stand for the \(i\)th feature channels of input \(C\) and \(I^{F}\), respectively. In this case, the length of the real number sequences is equal to the number of channels in \(I_{input}^{C}\).