# A Construction Method for a Dynamic Weighted Protein Network Using Multi-Level Embedding
## 2 Dynamic Weighted Protein Network Construction
### Multi-Level Embedding Method Construction
#### 2.3.1 Variational Graph Auto-Encoders  
VGAE is an unsupervised learning framework based on variational auto-encoders (VAEs) for graph structure data. It utilizes latent variables to learn interpretable latent representations of undirected graphs. The first embedding is performed by inputting the adjacency matrix \(A\) and the feature matrix \(X\) into an encoder consisting of two layers of graph convolution network (GCN), which are defined as follows:  
\[\text{GCN}(X,A)=\tilde{A}\text{ReLU}(\tilde{A}XW_{0})W_{1}. \tag{6}\]  
The first layer generates the low-dimensional representation \(\tilde{X}=\text{GCN}(X,A)=\text{ReLU}(\tilde{A}XW_{0})\), and the second layer captures the low-dimensional representation and generates \(\mu=\text{GCN}_{\mu}(X,A)=\tilde{A}\tilde{X}W_{1}\) and \(\log\sigma^{2}=\text{GCN}_{\sigma}(X,A)=\tilde{A}\tilde{X}W_{1}\), which are the mean and variance of the protein node vector, respectively. \(\tilde{A}=D^{-\frac{1}{2}}AD^{-\frac{1}{2}}\) refers to the symmetrically normalized adjacency matrix, while \(W_{0}\) and \(W_{1}\) refer to the initialization parameter matrix and the update parameter matrix after the GCN layer. \(\text{GCN}_{\mu}(X,A)\) and \(\text{GCN}_{\sigma}(X,A)\) share the first layer weights.  
Once the mean and variance are obtained, these parameterized latent variables allow for the definition of a uniquely deterministic multidimensional Gaussian distribution for each protein node. The protein nodes are sampled from this distribution to obtain the embedding representation \(Z\) of the protein nodes. This representation captures and quantifies the degree of interactions between proteins and the dynamics of the proteins over time. The matrix the embedding \(Z\) is defined as follows:  
\[q(Z|X,A)=\prod_{i=1}^{N}q(z_{i}|X,A), \tag{7}\]  
\[q(z_{i}|X,A)=N\big{(}z_{i}|\mu_{i},diag(\sigma_{i}^{2})\big{)}, \tag{8}\]  
where \(z_{i}\) represents the embedding vector of node \(i\), and \(\mu_{i}\) and \(\sigma_{i}^{2}\) are the mean and variance of node \(i\). After the mean and variance of all the node vectors are obtained, the node vectors are then formed by resampling; that is, the \(\epsilon\) obeying a standard Gaussian distribution is sampled first, and then \(z\) is computed based on the parameters with the following equation:  
\[z=\mu+\sigma\epsilon,\epsilon\sim N(0,1). \tag{9}\]  
After obtaining the embedding \(Z\), the distance metric can be calculated using the inner product without altering the vector size. Thus, the reconstructed adjacency matrix is  
Figure 2: Construction process of feature matrix.  
obtained by learning the similarity of each row in \(Z\) through the vector inner product [26], as shown in the following equation:  
\[\hat{A}=\sigma(ZZ^{\mathrm{T}}). \tag{10}\]  
The loss function consists of two parts: the first part is the reconstruction loss, and the other part is the Kullback-Leibler Divergence. Their equations are defined below:  
\[L_{\mathrm{recon}}\,=-\sum_{i,j}\bigl{(}A_{ij}\log\hat{A}_{ij}+(1-A_{ij})\log(1 -\hat{A}_{ij})\bigr{)}, \tag{11}\]  
\[L_{\mathrm{KL}}=\frac{1}{2}\sum_{i,j}\Bigl{(}1+\log\sigma_{ij}^{2}-\mu_{ij}^{2 }-\sigma_{ij}^{2}\Bigr{)}, \tag{12}\]  
where \(A_{ij}\) represents an element from the original data, while \(\hat{A}_{ij}\) represents the corresponding element in the model-reconstructed data. Equation (11) uses the cross-entropy loss function to measure the difference between the model-reconstructed and original adjacency matrix, accurately capturing the dynamic changes and associations between nodes in PPIN. Equation (12) measures the difference between the learned potential spatial distribution and the predefined distribution, which helps better distinguish the dynamic relationships between different nodes.