## 4 Reinforcement Learning for IoT Networking Management
### Related Research Studies
\begin{table}
\begin{tabular}{p{56.9 pt} p{113.8 pt} p{113.8 pt} p{113.8 pt}} \hline \hline
**Reference** & **Year** & **Contribution** & **Application Domain** & **Algorithm Model** \\ \hline [PERSON] et al. [60] & 2019 & The authors of this paper propose RL-based congestion-avoided routing for underwater acoustic sensor networks to reduce end-to-end delay and energy consumption. & Underwater acoustic sensor networks—RCAR & Q-learning \\ \hline [PERSON] et al. [61] & 2019 & In this paper, the authors propose an RL-based data forwarding scheme for a node based on the number of unsuccessful transmissions. The node adaptively switches between single-path and multi-path routing to optimize energy consumption and the packet delivery ratio. & Underwater WSN—CARMA & Q-learning \\ \hline [PERSON] et al. [62] & 2023 & This paper presents a routing approach based on RL for CRs. The idea of this study is to add the channel selection decision capability to provide improvements in the average data rate and throughput. & CRs ad hoc networks—CRAHN & Q-learning \\ \hline [PERSON] et al. [63] & 2019 & In this paper, the authors propose a CNN-based scheme that continuously adapts and optimizes routing decisions based on network conditions. This approach computes the routing path combinations with high accuracy. & SDNs & CNN \\ \hline [PERSON] et al. [64] & 2015 & The authors of this paper propose RL-based routing in CR ad hoc networks to reduce the protocol overhead and end-to-end delay and improve the packet delivery ratio. & CRs ad hoc networks—CRAHN & Q-learning \\ \hline [PERSON] et al. [65] & 2017 & This paper proposes a DRL approach for optimizing routing in SDNs. The agent in this approach optimizes the routing policy based on traffic conditions to minimize network delays. & SDN & DQL \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Related research studies on the use of RL for network management. \begin{table}
\begin{tabular}{p{56.9 pt} p{113.8 pt} p{113.8 pt} p{113.8 pt} p{113.8 pt}} \hline \hline
**Reference** & **Year** & **Contribution** & **Application Domain** & **Algorithm Model** \\ \hline [PERSON] et al. [66] & 2023 & The authors of this paper proposed an ML-assisted centralized link-state routing system for an SDN-based network. This scheme utilizes historical data of parameters such as the latency, bandwidth, signal-to-noise ratio, and distance to make routing decisions. & \multirow{2}{*}{ \begin{tabular}{} \end{tabular}  } & \multirow{2}{*}{ \begin{tabular}{} \end{tabular}  } \\ \cline{1-1} [PERSON] et al. [67] & 2012 & In this paper, the authors proposed load balancing in a multi-sink WSN. This approach divides the network into several zones based on the remaining energy of hotspots around the sink node. ML is applied to the mobile anchor, enabling it to adapt to traffic patterns and discover an optimal control policy for its movement. & \multirow{2}{*}{ \begin{tabular}{} \end{tabular}  } & \multirow{2}{*}{ \begin{tabular}{} \end{tabular}  } \\ \cline{1-1} [PERSON] et al. [68] & 2017 & In this approach, the authors present a task scheduling algorithm for dynamic WSNs that minimizes the exchange of cooperative information and balances resource utilization. & WSNs—QS & Q-learning with shared value function \\ \hline [PERSON] et al. [69] & 2019 & In this paper, the authors introduce a Q-learning algorithm for task scheduling in WSNs based on support vector machine. Their proposed approach optimizes the application performance and reduces energy consumption. & WSNs—ISVM-Q & Q-learning and support vector machine \\ \hline [PERSON] et al. [70] & 2017 & This paper proposes a link quality monitoring strategy for the RPL in IPv6-WSN using a multi-armed bandit algorithm. The proposed approach minimizes overhead and energy consumption by employing both synchronous and asynchronous monitoring. & WSNs—RL-Probe and & Multi-armed bandit \\ \hline [PERSON] et al. [71] & 2020 & The authors of this paper propose a DRL-based QoS-aware secure routing protocol for the SDN-IoT.