# DiffRS-net: A Novel Framework for Classifying Breast Cancer Subtypes on Multi-Omics Data
## 2 Materials and Methods
### The Attention Learning Model  
In this section, we construct an attention learning network model to classify breast cancer subtypes. As illustrated in Figure 1, the attention learning model simultaneously trains three tasks on three omics datasets, shares the representations through the attention mechanism, and finally integrates the output probabilities of each task to classify breast cancer subtypes. It consists of three main steps: (i) constructing modules for each omics via the module encoder, (ii) utilizing the attention mechanism to identify important modules across different omics data, and (iii) training on each omics in a fully connected layer. The module encoder consists of a fully connected layer, which links omics features to corresponding modules, each depicted by a vector. The weights of the fully connected layer, \(\mathcal{W}_{module}^{j}\), represent the association between features and modules of the \(j\)-th omics data, and the relationship between each feature and each module is represented by their corresponding weight vectors. For a training sample (\(x^{j}\), \(y\)), \(x^{j}\) denotes the sample from the \(j\)-th omics of \(J\) omics datasets, and \(y\) is its corresponding label. The module encoder for the \(j\)-th omics data is represented by \(\mathcal{F}^{j}_{module}\), and the corresponding module vectors \(M^{j}\) are defined as follows [42]:  
\[M^{j}\Big{(}x^{j}\Big{)}=\mathcal{F}^{j}_{\text{module}}\Big{(}x^{j}; \mathcal{W}^{j}_{\text{module}}\Big{)}\in R^{N^{j}\times D}, \tag{4}\]  
where \(\mathcal{W}_{module}\) denotes the weights of \(\mathcal{F}_{module}\), \(N^{j}\) indicates the number of modules of \(j\)-th omics data, and \(D\) is the dimension of the module vector. Let \(M\) denote the module vectors, with \(M\in\big{\{}M^{1},M^{2},M^{3}\big{\}}\) corresponding to Corr-mRNA, Corr-miRNA, and Corr-DNAm, respectively. A dot-product attention mechanism is then employed to extract important features from the set \(M\). Specifically, we utilize batch matrix multiplication to ascertain the attention scores between each pair of these three modules. Then, we use the Softmax function to normalize the attention scores to derive the final attention weights. Using the final attention weights, we perform a weighted summation of the corresponding feature representations, thereby obtaining feature representations that were adjusted by the attention mechanism. The attention mechanism matrix \(\text{ATT}_{M^{j}}\), \(i\in\{1,2,3\}\) can be defined as follows [43]:  
\[\text{ATT}_{M^{j}}=\sum_{M_{j}\in M}\ \frac{\exp\Big{(}Q_{M^{j}}K^{T}_{M^{j}} \Big{)}}{\sum_{M^{k}\in M}\exp\Big{(}Q_{M^{i}}K^{T}_{M^{k}}\Big{)}}V_{M^{j}}, \forall M^{i}\in M, \tag{5}\]  
where \(Q_{M^{j}}\) is the query vector for the task \(M^{i}\), \(K_{M^{j}}\) is the key vector for the task \(M^{j}\), and \(V_{M^{j}}\) is the value vector for the task \(M^{j}\). In this way, the feature representation of each task not only contains its own unique information but also integrates information related to other tasks, thus helping to enhance the understanding and representation of model on each task's features. Subsequently, each \(M,M\in\big{\{}M^{1},M^{2},M^{3}\big{\}}\) is multiplied by the corresponding \(\text{ATT}_{M^{i}}\), \(i\in\{1,2,3\}\) separately to produce new modified modules, which are then employed in the subsequent training in the hidden layer. Each hidden layer takes the previous layer as input and multiplies it with the trained weight matrix to obtain the input of the next layer. At the end of the network, the final classification layer flattens the resulting multidimensional data vectors and outputs the predicted classification probabilities for each breast cancer subtype.