## 4 RL-Based Decision Making Solution
### Problem Domain
Then, the available bandwidth \(abw_{l}\) of link \(l\) is decided by:  
\[abw_{l}=C_{l}-ubw_{l} \tag{2}\]  
where \(ubw_{l}\) is the total throughput of the passing flow \(f\in F\) in \(l\in E\). The delay impacts the video quality, particularly at the beginning of viewing, causing the start-up delay. However, this impact can be minimised in the rest of the video delivery due to the existence of a buffer which helps to play the video smoothly [35]. The measurement of \(D_{l}\) follows the method published in [36]. RYU controller \(Z\) sendsa Link Layer Discovery Protocol (LLDP) packet to a source switch, \(y_{1}\in Y\) and records the sending timestamp. \(y_{1}\) then sends the \(LLDP\) packet to a destination switch \(y_{2}\) and from \(y_{2}\) to \(Z\). \(Z\) then obtains the total delay (\(T_{d}\)) by calculating the difference between sending and receiving times of the \(LLDP\) packet. At this moment, the \(T_{d}\) of the path has been registered \(Z-y_{1}-y_{2}-Z\). The next step is to compute the delay between \(Z-y_{1}\) and \(Z-y_{2}\). \(Z\) sends an \(OFPEchoRequest\) contains the sending time \(ST\) as the data. \(y\) responses with \(OFPEchoReply\) message and records the receiving time \(RT\). In this case, the delay between controller and switch \(D_{Zy}\) is calculated based on the following formula:  
\[D_{Zy}=\frac{ST-RT}{2} \tag{3}\]  
Now, the calculation of the delay between \(y_{1}\) and \(y_{2}\) is presented as follows:  
\[D_{y_{1},y_{2}}=T_{d}-D_{Zy_{1}}-D_{Zy_{2}} \tag{4}\]  
Link loss affects the received video streaming as it reduces its resolution. In this case, a frame may be lost, resulting in freezing in the most recent frame and jumping to the subsequent consecutive frame that arrives, making the video streaming inefficient to watch. Network congestion usually leads to packet loss [35]. To compute \(L_{l}\), statistics from the \(y\in Y\), have queried using \(OFPPortStatsRequest\) messages. By taking advantage of these statistics, the delta of transmitted packets by \(y_{1}\) and received packets by \(y_{2}\) can be computed, which results in the link loss ratio:  
\[L_{l}=\frac{tx_{op}-rx_{ip}}{tx_{op}} \tag{5}\]  
where \(tx_{op}\) is defined as the number of transmitted bytes of the output ports that can be obtained when receiving a reply message. And \(rx_{ip}\) is the number of received bytes of ingress ports that can be collected when another reply message is received in a different period. Jitter, also known as delay variation, makes video streaming packets work inefficiently. It affects the presentation of the correct order of the frames by making them wait in the queue. This may cause freezing for the most recent frame until the arrival of the overdue frame, which starts playing in brief to provide time preservation of other arrived frames [37]. The jitter is defined as the difference in the link delay \(D_{l}\):  
\[J_{l}=D_{lt_{2}}-D_{lt_{1}} \tag{6}\]  
where \(t_{1}\) and \(t_{2}\) denote respectively the previous and the current link delay. An explanation of the exchanged messages protocol between the forwarding switch \(y\) with the RYU controller \(Z\) and the RL-based solution is depicted in Figure 3. It contains the preliminary negotiation and the important messages between \(y\) and \(Z\). When a multimedia provider begins streaming, \(Z\) receives a packet_In from \(y\). \(Z\) utilises its modules to monitor and extract the statistical network information. Then, it passes this request to the application plane where the routing computation takes place. In the end, \(Z\) uses the flow installation module which starts to install an optimal path for video traffic between the two pairs. With the terms explained earlier, the research focuses on QoS based network optimisation and improving users' experience. The enhancement criteria in this work can be seen as maximising all links utilisation of video streaming based networks.