## 1 Introduction
Recently, the DQN algorithm for Flying Ad Hoc Networks (FANETs) [8] was analyzed and studied. In their study, the authors agree that DQN contributes to a stable routing solution with low delay and minimum energy consumption. Unlike previous survey studies [9; 10], traditional DRL-based routing and deep RL-based routing methods were investigated with state-of-the-art research. In addition to the routing protocol itself, network architectures for applying machine learning have been addressed in many works. Specifically, Software-Defined Networking (SDN) [11] has been proposed to decouple the control and data planes in traditional networks through a logically centralized controller that provides a global view of targeted networks. Because a huge amount of data are collected in the centralized controller in SDN, this data can be used as an input dataset to train machine learning models for tasks such as routing and congestion control. Thus, SDN is regarded as a suitable architecture for implementing machine learning algorithms. Several comprehensive survey papers, such as [12; 13], have been published to discuss the implementation issues of machine learning algorithms in traditional networks. In both papers, the authors state that the outstanding capabilities of SDN make it easier to apply machine learning techniques. Additionally, traffic classification, routing optimization, quality of service, resource management, and security in SDN are presented. Based on the above analysis, there is extensive research on employing machine learning algorithms in MANETs under SDN to leverage knowledge obtained from past observations and improve network performance. Specifically, when considering node capability, SDN architecture is more suitable for MANETs than traditional networks, as indicated in [14]. Initially, the authors in [15] presented a new proactive routing protocol under the OpenFlow architecture. In this architecture, the controller in centrally managed MANETs learns the network topology without any location information and establishes reliable routes to forward data packets. A practical implementation of an SDN-MANET was presented in [16] to allow researchers to design new testbeds in centralized network management. A new network architecture to integrate SDN and MANET was presented in [17]. Two routing protocols, namely OLSR for IP forwarding and OpenFlow for SDN forwarding, were employed to provide seamless interoperability. Finally, scalability, support for dynamic network management, and resource optimization are also provided in this new architecture. Furthermore, because network performance is significantly affected by the routing protocol, researchers have focused on data-driven intelligent routing algorithms in SDN-based MANETs. However, there are few research papers on routing protocols based on machine learning in SDN-MANETs. Initially, the authors in [18] proposed a new method to extend network lifetime through a DRL approach. The energy-balanced network is accomplished by avoiding the use of nodes with low energy. Another approach to extend the traditional OLSR protocol in the SDN-MANET architecture was presented in [19]. The main technical contribution is deciding the best configuration of key parameters of the OLSR protocol by employing a machine learning approach based on datasets in an emulator. In this work, a prediction model is employed at the controller that stores recent information about network topology and traffic flows. This model is applied to adjust and configure OLSR protocol parameters. However, despite the promising potential, routing protocols based on machine learning in SDN-MANETs are in the initial phase. To utilize data-driven machine learning for routing in SDN-MANETs, in this paper, we present a new resilient routing protocol by combining DRL and UL. The former optimizes route selection while the latter reduces the update cost of the former. More specifically, DRL can provide system modeling with agents interacting with their environment and find optimization through a reward function.