## 4 RL-Based Decision Making Solution  
### Problem Domain  
According to graph theory, let the infrastructure plane be modeled into an undirected graph \(G(V,E)\), where \(V\) represents the set of nodes and \(E\) is the set of links between the nodes. \(V\) is associated with four subsets \(V=(W,X,Y,Z)\), where \(W\) refers to video streaming customers, \(X\) indicates the video streaming servers, \(Y\) denotes OpenFlow switches, and \(Z\) is the SDN controller. Each customer \(w\in W\) is connected to a forwarding switch \(y\in Y\) through a link \(l\in E\). Each \(l\) in the network is related to a limited capacity \(C_{l}\), allocated to the link flows, and it defines the maximum possible flow allowed to travel via the link. Every traffic flow \(f\) is associated with two types of flows \(F=(F_{v}\cup F_{b})\), where \(F_{v}\) stands for video streaming flows and \(F_{b}\) indicates UDP traffic flows. Generally, a flow in the network is known as a sequence of communications between the source and destination nodes. It is typically defined by its 5-tuple attributes (source IP, destination IP, source port, destination port, protocol field). The clients request to watch a video from a streaming provider over the SDN-enabled network. The idea is to set up a feasible routing path for the video flows. If \(P\) contains the set of potential routes, then the routing method is employed to obtain the feasible path \(p\in P\), in which a path is specified by a set of links \(p=l_{1},\ldots,\ l_{n}\) that connects the nodes between source-destination pairs. If more traffic flows exist in the same link and compete over the network link capacity, the link gets heavily loaded and congested since the traffic flows trying to pass through this link exceed its bandwidth. In this case, the involved traffic flows are more likely to exhibit higher data loss and delay, which seriously impact the network QoS and users' QoE. Accordingly, this study presents an optimisation method that addresses this case by considering a network monitor, where network status is periodically monitored, and path decisions will be deployed according to the current status of network links. Therefore, RL agent will react quickly to congestion in the link and suggest another path for video transmission. In this case, the proposed solution shall provide a good viewing experience to the clients. The proposed solution of video streaming QoS/QoE optimisation considers four link-state metrics in each path \(p\): link available bandwidth \(B_{l}\), link delay \(D_{l}\), link loss rate \(L_{l}\) and link jitter \(J_{l}\). These parameters may cause distortions in the video streaming that affect the received service quality [33]. The resolution of the video is affected by the available bandwidth of the network. Having enough bandwidth will ease the transmission of the video smoothly hence providing better video quality. Whereas, the lack of available bandwidth affects customers' QoE as it leads to video streaming quality degradation [34]. To obtain the \(B_{l}\) through an end-to-end path between two pairs, the network statistics in the control plane is used to track the traffic flow and collect the received and transmitted bytes at each port at regular intervals of time. Compared with the retrieved values at two successive responses time, it is feasible to calculate the available bandwidth. Let say that the controller receives \(OFPortStatsReply\) message from the infrastructure plane, at time \(t_{1}\), which contains the number of bytes received, \(br_{t_{1}}\). Then, after an interval period \(\Delta t\), another \(OFPortStatsReply\) message is obtained, at time \(t_{2}\), which contains the number of bytes received \(br_{t_{2}}\), the used bandwidth can be computed as follows:  
\[ubw_{l}=\frac{br_{t2}-br_{t1}}{\Delta t} \tag{1}\]  
where \(\Delta t\) represents the period of the sample interval.