# Multi-Agent Reinforcement Learning Framework in SDN-IoT for Transient Load Detection and Prevention
###### Abstract  
The fast emergence of IoT devices and its accompanying big and complex data has necessitated a shift from the traditional networking architecture to software-defined networks (SDNs) in recent times. Routing optimization and DDoS protection in the network has become a necessity for mobile network operators in maintaining a good QoS and QoE for customers. Inspired by the recent advancement in Machine Learning and Deep Reinforcement Learning (DRL), we propose a novel MADDPG integrated Multiagent framework in SDN for efficient multipath routing optimization and malicious DDoS traffic detection and prevention in the network. The two MARL agents cooperate within the same environment to accomplish network optimization task within a shorter time. The state, action, and reward of the proposed framework were further modeled mathematically using the Markov Decision Process (MDP) and later integrated into the MADDPG algorithm. We compared the proposed MADDPG-based framework to DDPG for network metrics: delay, jitter, packet loss rate, bandwidth usage, and intrusion detection. The results show a significant improvement in network metrics with the two agents. M +
Footnote †: journal:  
1
Footnote 1: footnotemark:  
[PERSON]; [PERSON]; [PERSON]; [PERSON] Multi-Agent Reinforcement Learning Framework in SDN-IoT for Transient Load Detection and Prevention. _Technologies_**2021**, \(9\), 44. [[https://doi.org/10.3390/technologies9030044](https://doi.org/10.3390/technologies9030044)]([https://doi.org/10.3390/technologies9030044](https://doi.org/10.3390/technologies9030044))  
Academic Editors:  
[PERSON] and [PERSON] Received: 19 May 2021  
Accepted: 23 June 2021  
Published: 29 June 2021  
**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. **Published:** Dole,  
be intelligently programmed to react positively to possible transient network loads and intrusions that could destabilize its functionality and operations. With the advent of Machine Learning [11; 12], systems automation through data analytics has seen immense applications by researchers. Machine learning algorithms have diverse features with pros and cons in producing accurate models for classification, regression, clustering, and behavioral learning. The most popular categorization of machine learning algorithms is supervised learning models [13; 14]. Supervised learning models train agents with a labelled dataset. After enough training, the agent is fed with new data for identification and classification. Supervised learning presents an obvious disadvantage of cost in labelling the dataset properly. With variations in data from diverse applications, IoT also presents a difficulty in accurately classifying data. Unsupervised learning [15; 16] is more about identifying relationships and patterns and less of automation. This type of learning involves a training with unlabeled data and no output. With unsupervised learning, there is no guidance on efforts to get the right groupings. Reinforcement Learning (RL) [17], as shown in Figure 1, is the real basis for automation in machines where agents take actions on the environment based on intelligent policies guided by reward systems. Agents accumulate knowledge on the environment with continuous trials and error of actions until they master the relevant actions in an exploitation-exploration tradeoff [18]. In maintaining efficiency in networks, RL agents have seen dynamic integrations in SDN-IoT domains due to the expected network flow complexities envisaged by network operators and the possible malicious intrusion from compromised IoT devices by attackers. The network must be protected from different attack loopholes [19] in the SDN architecture while engineering dynamic multi-path routing [20; 21] strategies for packets from benign users.