## 3 Routing Approach Used-Snnr
### System Implementation  
The CSG approach uses a spiking neural network and reinforcement learning to select a path that delivers each bundle with optimized delay and packet loss. An SNN agent runs on each satellite, monitors the network performance on the links, and selects a link autonomously to deliver each bundle.  
The cognizant agent observes the environment by computing the latency and the packet loss ratio after forwarding a bundle on an outgoing link. It estimates the cost of delivering the bundle using the metrics according to the optimization goal. The agent compares the current cost with the moving average value of the cost up to the previous instance and rewards the link if the current cost is less; else, it penalizes the link. The reward/penalty is used to update the weights of a neuron's synapse, and the neuron that emits a spike at the earliest is used to select an outbound link next time. Thus, the performance metrics, delay, and packet loss ratio associated with each link are encoded in the time of emitting spikes by the neurons. The cognizant agent learns to choose a neuron corresponding to the outbound link, which gives the minimum latency and packet loss through repeated interactions with the environment. The interaction of the cognizant agent with its environment is depicted in Figure 1. In our context, the environment is the space communication network, which includes spacecraft and relay satellites on different orbits, surface elements on planets such as Mars and the Moon, and ground stations on the Earth, as shown in the right side of the picture.  
Each node computes the transmission time of a bundle's delivery on an incoming (ingress) link when it receives a forward report by calculating the difference between the time stamp of the previous node, which sends the report, and the time at which it received the report. This transmission time is sent back in a backward report to the previous node from which it received the forward report on the same link. The SNN agent counts the number of forward reports sent and the number of backward reports received on each outbound link and computes the packet loss ratio on the link. We compared the performance of snnr routing with the routing approaches that use the following cost functions: 1. snndp: a linear sum (5); 2. snnd: delay-only (6); 3. snnp: packet-loss-only (7); 4. static: CGR;
1. snndp--optimizes both the response time and packet loss using a linear cost function as below: \[cost_{dp_{i}}^{t}=k\ bundleLossRatio_{i}^{t}+\left(1-bundleLossRatio_{i}^{t} \right)\left(transmissionTime_{i}^{t}+stallTime_{i}^{t}\right),\] (5) where \(k\) is a constant used to balance the two metrics, i.e., \(bundleLossRatio\) is a decimal value, whereas \(transmissionTime_{i}^{t}\) is measured in milliseconds;
2. snnd--optimizes only the response time in delivering data from a satellite node to a ground station. The optimization of \(cost_{d}\) to optimize the delay is computed for an outbound link \(i\) at a satellite node at \(t\), computed as \[cost_{d_{i}}^{t}=transmissionTime_{i}^{t}+stallTime_{i}^{t};\] (6)
3. snnp--optimizes only the packet loss in delivering data. The optimization \(cost_{p_{i}}^{t}\) to minimize the packet loss ratio is computed for an outbound link \(i\) at a satellite node at \(t\) as \[bundleLossRatio_{i}^{t}=\frac{\left(forwardBundlesent_{i}^{t}-backward ReportsReceived_{i}^{t}\right)}{forwardBundlesent_{i}^{t}}\] (7) \[cost_{p_{i}}^{t}=bundleLossRatio_{i}^{t};\]
4. static--this is the same as CGR, which selects the earliest link in the shortest path between two satellite nodes.