# Improving End-To-End Latency Fairness Using a Reinforcement-Learning-Based Network Scheduler
## 2 Materials and Methods
### Problem Statement  
We consider an output buffered switch in Figure 1, where each output port module has a buffer. Table 2 explains the symbols used in Figure 1 and the simulation environment. The buffer of the output port module contains the output queues based on input ports. Each queue is paired with an input port to store packets coming from the corresponding input port. To implement such a node, the packet must contain input port information. The total number of queues in a node is M \(\times\) N. The scheduler of the output port module uses an RL-based scheduler. The scheduler of nodes not using the RL model operates as a FIFO. Figure 1: Node architecture used in the simulation. Let us consider a network that consists of three nodes with the topology shown in Figure 2. The sources in the network generate packets according to the packet generation scenarios in Table 3. T refers to the unit time of the simulation. We assume that the transmission delay occurring on the link in the simulation always takes T for every packet. This is to make it easier to observe the results by making all transmissions and receptions occurring in the simulation performed every T. To fix the transmission delay to T, link capacity is fixed to C, and the length of all packets is fixed to C \(\times\) T. It is assumed that all traffic originating from the source is of the same type, of the same nature, and of the same priority, since competing packets must have the same characteristics for fair competition. It is also assumed that the propagation delays of all links have a negligibly small value. Therefore, the propagation delay does not affect the E2E latency, so the E2E latency is obtained in units of T. As the simulation operates in units of T, the packet generation time points are also determined in units of T in the packet generation scenario. Since the scheduler is only meaningful when packets in the node need to be sent to the same port simultaneously, the packet creation scenario was designed with this consideration in mind. Table 3 presents a scenario in which packets contend once at node 0 and once at node 1. Although there may be no contention at node 1, the scheduler at node 0 selects a packet that increases the maximum end-to-end delay time, distinguishing it from the optimal case. Therefore, additional packets are not generated for contention at node 1. Since such a network environment requires precise time synchronization, it is well suited to a time-sensitive network, especially a LAN environment [31]. In this environment, the lowest possible maximum E2E latency in the example topology is 3T. To minimize the maximum latency, the following service scenario must be executed: At time 0 in node 0, when two packets from the two sources want to go out on the same link, the packet generated at source 0 has to travel a larger number of hops and must be sent out first. At time T, when two packets in node 1 from the two input ports want to go out on the same link, the two packets have the same number of remaining hops, but the packet generated \begin{table}
\begin{tabular}{c c} \hline \hline
**Symbol** & **Meaning** \\ \hline M & Number of an output port \\ N & Number of an input port \\ H & Number of nodes \\ C & Link capacity \\ T & Unit time in simulation \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Symbols used in topology description. Figure 2: Example topology. from source 0 has already suffered a delay time T, so it must be sent out first. When the scheduler operates in this way, the E2E latencies of packets generated from sources 0, 1, and 2 are 3T, 2T, and 2T, respectively, and the maximum E2E latency is minimized. The maximum E2E latency that occurs when the packet generated from source 0 is not transmitted first is 4T. In this case, the average E2E latency may be smaller than the average E2E latency when the maximum E2E latency is 3T. However, since we are interested only in the minimization of the maximum E2E latency, the average E2E latency will not be used as a performance indicator.