# Performance Evaluation of Graph Neural Network-Based RouteNet Model with Attention Mechanism
## 3 Methodology
### Detailed Internal Architecture of RouteNet with Attention  
Figure 8 illustrates the detailed internal architecture of the RouteNet model with an attention mechanism integrated into it. The architecture processes network data through multiple iterations, each aimed at refining the state information of network elements for accurate performance metric prediction. Figure 6: RouteNet model architecture with attention and message passing. Figure 8: Detailed internal architecture diagram of RouteNet with attention. Figure 7: RouteNet architecture with iterative message passing with attention layer. The flow steps are presented in the sections below:  
i. GRU Layers for Link and Path Updates: The model initiates with GRU layers, which take initial link states \((h_{L1},h_{L2},\ldots,h_{Ln})\) and path states \((h_{P1},h_{P2},\ldots,h_{Pn})\) as inputs. These states are processed through GRU layers, aiming to capture temporal dependencies and refine their representations. ii. Attention Mechanism: After the initial updates, the attention layers receive the path states \((h_{P1},h_{P2},\ldots,h_{Pn})\) and dynamically calculate attention weights. This process allows the model to focus on the most informative features within the path states by assigning higher weights to more significant inputs. The output of the attention layers are attention-refined path states \((h_{A1},h_{A2},\ldots,h_{An})\). iii. Iterative Processing and Message Passing: During \"\(T\)\" iterations, the model continually updates the link and path states. The attention mechanism plays a crucial role here by recalibrating the path states during each iteration based on the evolving context of the network's state, ensuring that crucial information is emphasized for future updates. iv. Performance Metrics Readout: Finally, the outputs from the last iteration of attention layers \((h_{A1},h_{A2},\ldots,h_{An})\) are fed into the readout layers, marked as 'R' in Figure 8. The readout layers are designed as a series of densely connected neural networks configured to map the attention-enhanced states to specific performance metrics. Algorithm 1 begins by initializing the attention weights for each path and link at time step 0. These attention weights, denoted as \(a_{p}^{0}\) for paths and \(a_{l}^{0}\) for links, are set to small positive values and will be updated during the message-passing process. Next, the algorithm initializes the hidden states of paths and links at time step 0. Each path's initial hidden state, denoted as \(h_{p}^{0}\), is formed by concatenating the corresponding path feature \(x_{p}\) with zeros for the remaining dimensions. Similarly, each link's initial hidden state, denoted as \(h_{l}^{0}\), is formed by concatenating the corresponding link feature \(x_{l}\) with zeros for the remaining dimensions. The algorithm then enters a loop from time step 0 to \(T\)-\(l\) to perform message passing between paths and links. Here, \(T\) denotes the iterations for message-passing steps, and within each iteration \((t)\), hidden states \(h_{p}\) and \(h_{l}\) are updated. These iterations converge to a fixed point from initial states \(h_{0}^{p}\) and \(h_{0}^{l}\). The value of \(T\) should be chosen in such a way that it balances capturing crucial information during the message-passing iteration with computational efficiency. The hidden state of the path is updated using the GRU function \(GRU_{t}\), which takes the current hidden state \(h_{p}^{t}\) and the hidden state \(h_{l}^{t}\) of the link and produces a new hidden state for the path. Additionally, the updated hidden state \(h_{p}^{t}\) is stored as the message \(\tilde{m}p,l^{t+1}\) from path \(p\) to link \(l\). In addition to updating the hidden states, the algorithm computes the attention weights for each path-to-link message and link-to-path message. This is carried out by applying an attention mechanism to the concatenation of the hidden states \(h_{p}^{t}\) and \(h_{l}^{t}\).