# Artificial Intelligence Control Logic in Next-Generation Programmable Networks
## 2 Problem Description
If the latter value is greater than the former value, the switching is performed. In this way, in each step, only a part of registered intents is considered, and usually only a small fraction of this part of intents is issued with a command to change a current path. The artificial neural network used in this research is a Deep-Q-Network (DQN) [29] consisting of four inputs (observations), two outputs, and four hidden layers of ten, five, five, and three neurons. We used DQNs, because it gave the best results in our internal studies. The learning process was implemented using Ray framework [30] and its goal was to maximize the total throughput in the network. The neural network takes a state of a network from a viewpoint of a single intent as an input and as an output it produces the expected q-function value for two cases: (a) when nothing happens and (b) when the path is switched. Figure 1: Conceptual model of the system. The observations provided to the artificial neural network depend on a considered intent and are computed using data collected from a network and from ONOS. The observations are as follows:  
* Ratio of efficiency on active path;
* Ratio of potential efficiency on inactive path;
* Edge occupancy percentage on active path;
* Edge occupancy percentage on inactive path. The first observation is a ratio of an obtained momentum throughput for the intent to the maximum throughput declared for this intent. The maximum declared throughput is provided by the owner of the intent when creating the intent using the ONOS interface. The second observation is a ratio of the minimum available capacity on all edges of the inactive path to the declared intent's maximum throughput. The third observation is the minimum ratio of an obtained momentum throughput for the intent to a total traffic on an edge for all edges of the active path. Similarly, the fourth observation is the minimum ratio of the obtained momentum throughput for the intent to a total potential traffic (current traffic increases by the current momentum throughput of the intent) on an edge for all edges of the inactive path. All these observations are computed independently for each intent using data collected from ONOS and ifstat. The artificial neural network that is fed with the above observations was trained using a heavily modified Iroko framework [31] that is based on Open AI Gym [32] and was originally implemented to optimize traffic in data centers. Episodes were run with the mininet framework using a modified version of goben [33] to generate various traffic patterns. The neural network was trained to maximize the total throughput in a network. The modifications to Iroko framework are as follows:  
* Procedures to collect traffic from a network using ifstat;
* Procedures to compute the previously described observations from the traffic;
* Objective function fed to Ray that now expresses the total throughput in a network;
* Dynamically modified network topologies that can change during training between episodes;
* Dynamically selected currently considered intents that also change between episodes;
* Support for dynamic traffic that can vary between iterations. The network was trained in series of episodes. Each episode was defined by an SDN network topology and a set of active traffic demands. Each demand was described by its source, destination, and an active time period. Note that the proposed neural network was trained using various topologies and traffic patters; thus, it is topology independent. Because of the limitations of mininet and utilized machines, the considered topologies consisted of ten switches at most. In the training process, we used three different topologies with eight different traffic patterns for each topology. The episodes were repeated for 36 times, totalling 864 episodes in the training process. Each episode lasted for 100 s in real time, resulting in 100 training steps. In each step, an active demand was selected at random and only this demand was considered in this particular step.