## 3 The Proposed DRL + GNN-Based Handover Scheme  
### GNN Architecture  
A graph neural network (GNN), introduced in [36], is a deep learning model designed for processing graph data. The GNN model comprises multiple graph convolutional layers, with each layer updating the representation of a node by aggregating both the node and its neighborhood data. Through this process of information aggregation, the model can capture features of both local nodes and global graph structures. This capability allows GNN to excel in processing graph data. GNNs leverage their \"black-box\" nature to learn the relationships between nodes and edges, iterating over the states of nodes and edges in the process. A message passing neural network (MPNN) forms the foundational framework of graph neural networks [37], employing an iterative message passing algorithm to propagate information between nodes and edges of a graph. In the algorithm proposed in this paper, DQN agents do not directly interact with the LEO satellite network environment; instead, they interact with it through a graph neural network. Specifically, the graph neural network learns the representation of the LEO satellite handover graph state, which serves as the interaction environment for the deep reinforcement learning agents. Given that the edge features of the satellite handover graph define the optimization problem for satellite handover selection, we employ an enhanced MPNN to conduct the message passing process between all edges in the graph. The structure of MPNN message passing is illustrated in Figure 5.  
MPNN is divided into message passing phase and readout phase. The message passing phase can be defined by message function, aggregation function and update function. The message function \(M()\) is used to generate the message of the neighboring edge to the central edge, and its input is the state of the central edge and the neighboring edges; the aggregation function \(A()\) is used to aggregate all the messages generated by the message function, and obtain the post-aggregation message as \(M\). Its inputs are the messages generated by all the neighboring edges to the central edge; the update function \(U()\) is used to update the state representation of the edges, and its inputs are the state of the edge's last iteration and the the aggregated message. Figure 5: MPNN passing architecture. In the message passing phase, the states of all the edges in the graph are first generated by the message function \(M()\), then all the messages are aggregated by the aggregation function A, and then the new state of each node is calculated using the update function \(U()\). The functions \(m()\) and \(U()\) can be learned by neural network. The specific message passing process is as follows: (1) combine the features of each edge with the features of the neighboring edges using full connectivity to generate the messages from the neighboring edges to this edge; (2) aggregate the messages generated by each edge with all its neighboring edges using element-by-element summation to obtain the aggregated message \(m\); (3) use a recurrent neural network (RNN) to update the state \(h\) of each edge; the input of the RNN model is the state of the edge's previous iteration and the aggregated message \(m\). The message-passing process can be expressed as  
\[m^{K+1}(c) = A_{c\in N(c)}[M(h_{e}^{k},h_{c}^{k})] \tag{14}\] \[h_{c}^{k+1} = U[h_{e}^{k},M^{K+1}(c)] \tag{15}\]  
where \(h_{e}^{k}\) and \(h_{c}^{k}\) denote the state of edge \(e\) and \(c\) after the \(k\)th iteration, respectively, and \(N(c)\) denotes that the set of all neighboring edges of \(c\); \(m^{K+1}(c)\) is the aggregation of messages generated by all neighboring edges to the central edge \(c\) in the \(k+1\)st iteration. Readout phase: the \(q\)-estimate corresponding to the current state and action are output through the readout function \(R()\). Finally, the DRL agnet selects the action with the highest \(q\)-value by comparing the \(q\)-estimate of a set of actions in the current state.