# BTDNet: A Multi-Modal Approach for Brain Tumor Radiogenomic Classification
## 5 Conclusions and Future Work  
In this paper, we proposed BTDNet, a new multimodal approach that harmonizes the analysis of 3D image volumes consisting of different numbers of slices and annotations per volume. In more detail, BTDNet accepts as input an mpMRI (i.e., a FLAIR 3D volume, T1w 3D volume, T1 wCE 3D volume, and T2 3D volume) and predicts the MGMT promoter methylation status. BTDNet consists of four components: 3D analysis, the routing, the modality fusion, and the data augmentation.  
When BTDNet is fed with a new 3D volume input, a CNN performs the initial local (per 2D image/slice) analysis, extracting features from each slice. Then, these features are fed to an RNN and placed on top of the CNN to capture their temporal dependencies. The CNN and RNN networks perform global (per 3D volume) analysis. The RNN output features are then concatenated and fed to a mask layer, which dynamically selects RNN outputs, taking into account their input length. The output of the mask layer is then fed to a dense layer. This procedure is followed for each of the four modalities; the output features of the dense layer for each modality are then concatenated and fed to a dense layer that maps them to the same feature space. Finally, the output layer follows providing the final classification. The objective function for training BTDNet is based on the focal loss for multi-class classification.  
Excellent performance was achieved on the dataset of the RSNA-ASNR-MICCAI BraTS 2021 Challenge, verifying our developments and surpassing all state-of-the-art methods by large margins. In more detail, BTDNet outperformed the winning method of the challenge by 3.3% and the other top-four methods by 18.8-23.3% in terms of the F1 score. Finally, an extensive ablation study was conducted and presented, validating the contribution and performance gain that each component of BTDNet brings.  
It should be noted that in order for our method to be used in real-life clinical situations, a much larger and more diverse dataset will need to be used for training BTDNet, and clinical trials should be conducted.  
The hypothesis and limitation of our method is the assumption that all modalities are available and present. An interesting future direction could be to extend BTDNet to work in the case of one or multiple missing modalities. Another future direction could be to develop a joint method that initially performs brain and tumor segmentation and then predicts the MGMT promoter methylation status. This new method could be built upon BTDNet. Finally, our work only considered the temporal dependencies between data of the same modality, and the relationship between data of different modalities was not investigated. This inter-modality relation could be another intriguing future direction.  
**Author Contributions:** Conceptualization and methodology, D.K.; software, validation, investigation, and formal analysis, D.K., K.V., P.G. and S.R.; writing (original draft preparation, review, editing), D.K., K.V., P.G. and S.R.; supervision and project administration, D.K. All authors have read and agreed to the published version of the manuscript.  
**Funding:** This research received no external funding.  
**Institutional Review Board Statement:** Not applicable.  
**Informed Consent Statement:** Not applicable.  
**Data Availability Statement:** The dataset supporting the conclusions of this article is available at [[https://www.kaggle.com/c/rsna-mincai-brain-tumor-radiogenomic-classification](https://www.kaggle.com/c/rsna-mincai-brain-tumor-radiogenomic-classification)]([https://www.kaggle.com/c/rsna-mincai-brain-tumor-radiogenomic-classification](https://www.kaggle.com/c/rsna-mincai-brain-tumor-radiogenomic-classification)) (accessed on 30 October 2023).  
**Conflicts of Interest:** The authors declare no conflict of interest.