# BTDNet: A Multi-Modal Approach for Brain Tumor Radiogenomic Classification
## 3 Materials and Method
### Proposed Method
#### 3.1.1 Data Augmentation Component  
During training, when batches of 3D volumes (\(X\)) are sampled (i.e., all slices within the 3D volumes are sampled), initially, we perform geometric transformations such as random horizontal flip and rotation and create transformed batches of 3D volumes (\(T_{X}\)). It should be noted that we do not perform the same geometric transformation on all slices within the same volume. On the contrary, we apply a different geometric transformation to each slice of the same volume (e.g., one slice is augmented with horizontal flip and \(10^{\circ}\) rotation, whereas its following slice is augmented only with \(-4^{\circ}\) rotation).  
Then, we apply MixAugment [22] to these transformed batches of 3D volumes. MixAugment is a simple and data-agnostic data augmentation routine that trains a method on convex combinations of pairs of examples and their labels. It extends the training distribution by incorporating the prior knowledge that linear interpolations of feature vectors should lead to linear interpolations of the associated targets. Therefore, it acts as a regularizer to favor linear behavior in-between training examples and it improves the generalization ability of the network, which is always a real-world challenge that can hinder the applicability and effectiveness of any method. We extend MixAugment to construct virtual training examples/3D volumes (\(\tilde{X},\tilde{y}\)) (one for each modality), and the method is trained concurrently on both real (r) and virtual (v) examples/3D volumes. The virtual examples, e.g., the FLAIR modality, are constructed as follows (similarly, virtual examples are constructed for the other modalities):  
\[\tilde{X} =\lambda T_{X_{i}}+(1-\lambda)T_{X_{j}}\] \[\tilde{y} =\lambda y_{i}+(1-\lambda)y_{j} \tag{1}\]  
where \(T_{X_{i}}\in\mathcal{R}^{H\times W\times t_{1}}\) and \(T_{X_{j}}\in\mathcal{R}^{H\times W\times t_{1}}\) are two random FLAIR modality inputs (i.e., 3D volumes). H and W are the height and width of each slice (i.e., 2D image); \(t_{1}\) is the total number of slices within the volume after padding; \(y_{i}\) and \(y_{j}\in\{0,1\}^{2}\) are their corresponding one-hot labels; and \(\lambda\thicksim\mathcal{B}(a,\alpha)\in[0,1]\) (i.e., Beta distribution) for \(\alpha\in(0,\infty)\).  
During inference, we apply test-time data augmentation [13]. For a given input 3D volume (\(X_{i}\)), we create three transformed versions: (i) one where only horizontal flip is performed (\(F_{X_{i}}\)); (ii) one where random rotation is performed (\(R_{X_{i}}\)); and (iii) one where both horizontal flip and random rotation are performed (\(FR_{X_{i}}\)). We feed these three versions, along with the non-augmented 3D volume, to the method, which generates a prediction for each version (\(p^{F_{X_{i}}},p^{R_{X_{i}}},p^{FR_{X_{i}}},p^{X_{i}}\), respectively). The final prediction (\(p_{\text{final}}\)) is the sum of these four outputs (\(p_{\text{final}}=p^{F_{X_{i}}}+p^{R_{X_{i}}}+p^{FR_{X_{i}}}+p^{X_{i}}\)). It should be noted that we use the logits rather than the probabilities.