## 5 SD-CFN Routing
### Routing Algorithm  
This routing algorithm implements a learning process for finding the optimal paths for all node pairs in the data plane. The input data to Algorithm 1 include the learning rate \(\alpha\), parameter \(\epsilon\) (see Equation (20)), the number of learning rounds \(n\), connection of all nodes _Node_list_, link status between all nodes _Link_status_, and the pair of the computing resource requester and the target computing node \((T_{src},T_{dst})\). The output is the set of optimal reward routes for all node pairs given the link state. This path is formed by state-action pairs with the lowest values in the Q-table. Finally, based on the distribution of node computing power, the optimal path from the computing request source to the target computing node is determined as the final result of computing routing. This path is then installed in the switches as flow tables, facilitating the scheduling of computing resources through the network.  
The RSCR routing algorithm can find the optimal path for all pairs of nodes in the given topology. For each pair of nodes \((src,dst)\) in the Nodelist, routing learning can be performed following the steps in Algorithm 1, lines 1-17. First, initialize the Q-table to 0 (line 2), and then, the algorithm goes through episodes (line 3) until the state \(S_{t}\) becomes the final state \(dst\) (lines 5-9). Starting with \(src\) as the initial state (line 4), assuming the next state is not the target state \(dst\) (line 5), the agent selects the action \(A_{t}\) based on Equation (20) (line 6). The agent calculates the reward value according to Equation (18) (line 7), then updates the state using Equation (19) (line 8), and moves to the new state (line 9) until the episode concludes.  
After the RL agent completes learning, the optimal path between \(src\) and \(dst\) can be calculated based on the Q-table, specifically the path with the minimum Q-value (line 12). If \((src,dst)\) is the same as \((T_{src},T_{dst})\), then this node pair corresponds to the computing resource requester and target computing node pair we are looking for, and the corresponding path is the optimal computing resource path (lines 14-15). Finally, the knowledge plane stores routing information for all node pairs in _Paths_, and the optimal computing resource path is specially marked. Subsequently, the flow installation module retrieves these paths and installs them in the routing table of switches.  
```
Data: Learning rate: \(\alpha\) Exploration and exploitation parameter: \(\epsilon\) Learning episodes: \(n\) Connection of all nodes \((src,dst)\): \(Node\_list\) Link status between all nodes: \(Link\_status\) Node pairs of Initiating and the optimal node of the computing resource request: \((T_{src},T_{dst})\) Result: The optimal path set of the given nodes: \(Paths\)
1 while\((src,dst)\in Node\_list\)do
2 Initialization Q: \(Q(S,A)=0,\forall s\in S,\forall a\in A\);
3 forepisode \(\gets\) 1 to \(n\)do
4 Start in state \(S_{t}\)=\(src\in S\);
5 while\(S_{t+1}\) is not dstdo
6 Select \(A_{t}\) for \(S_{t}\) with policy derived from Q using \(\epsilon\)-greedy exploration and exploitation method;
7\(R_{t+1}\gets R(S_{t},A_{t})\);
8\(Q_{t+1}(S_{t+1},A_{t+1})=Q_{t}(S_{t},A_{t})+\alpha[R_{t}+min_{A}Q_{t}(S_{t+1},A)-Q_{t}(S_{t},A_{t})]\);
9\(S_{t}\gets S_{t+1}\)
10 end for
11
12 end for
13 Find the path with the minimum Q-value in the Q-table for the pair \((src,dst)\).
14
15 end for
16 if\((src,dst)=(T_{src},T_{dst})\)then
17\((T_{src},T_{dst})\) is the optimal path from the computing resource requester to the target computing node
18
19 The knowledge Plane stores all routing information \(Paths\), and additionally marks and stores the path \((T_{src},T_{dst})\) for the computing resource requester and the target computing node. return Result;
```  
**Algorithm 1**SD-CFN Routing: RSCR