## 1 Introduction  
Mobile Ad Hoc Networks (MANETs) are self-organized networks composed of mobile nodes without any fixed infrastructure. In such networks, each node can function as a source, destination, or relay node. Specifically, MANETs are characterized by nodes that not only generate their own data but also forward packets for others, forming a self-organizing, infrastructure-less network. Because the nodes in a MANET can move arbitrarily, traditional routing protocols often struggle to maintain stable routes. For instance, the Ad hoc On-Demand Distance Vector (AODV) protocol discovers routes only when needed, which can be efficient in low-mobility scenarios but may incur frequent route discovery overhead under high mobility [1]. Consequently, establishing a robust routing strategy in MANETs requires accounting for dynamic link changes and mitigating excessive control traffic. However, due to node mobility, the network topology frequently changes. Consequently, routing protocols in MANETs must be designed to establish paths that are resilient to mobility. In this context, two types of protocols, known as proactive and reactive, have been extensively studied and investigated. Additionally, many variant protocols have been proposed to enhance routing performance by introducing new routing metrics and network architectures, such as clustering. For instance, clustering-based adaptations of the Destination-Sequenced Distance Vector (DSDV) protocol have been proposed, where nodes are grouped into clusters with designated cluster heads [2]. This approach reduces routing overhead and improves stability, making it particularly effective in dynamic topologies. Moreover, geographical routing protocols can facilitate packet forwarding using the physical locations of nodes rather than node IDs, eliminating the need for maintaining extensive state information. Recently, unlike traditional mathematical model-driven routing protocols mentioned earlier, machine learning-based routing protocols have attracted researchers' interest. These data-driven protocols utilize data to train machine learning algorithms without specific modeling of the network environment. Therefore, it is possible to make routing decisions in an acceptable time using input data. There are three main classifications of machine learning: Supervised Learning (SL), UL, and RL. Among them, DRL is known as a suitable algorithm for routing in networks in terms of computing overhead. Routing protocols based on DRL aim to select the optimal route. Specifically, the Deep Q-Network (DQN) algorithm has been studied to address challenges in MANETs. Initially, link stability based on link expiration time, bandwidth efficiency, and the power metric were used in DQN to choose better routes before a route failure [3]. Another DQN algorithm to select highly stable and low-delay paths was presented by [PERSON] et al. [4]. To improve MANET performance, a cross-layer approach was addressed in [5] using a multi-agent DQN algorithm. The multi-agent DQN jointly optimizes power control, rate adaptation, and routing strategies. A delay prediction model is the main function of DQN to compute the global cost function based on node density, mobility, and data traffic features such as rate and number of flows. The DQN algorithm can be applied to Vehicular Ad Hoc Networks (VANETs), which support high mobility and a large number of vehicles. The proposed scheme, called Adaptive Routing Protocol based on Reinforcement Learning (ARPRL) [6], initializes the Q-table through HELLO packets. Additionally, the convergence speed problem of DQN was addressed by a reactive routing probe strategy. A multi-agent DRL technique [7] for highly dynamic MANETs was introduced using parameterized thresholds and a newly designed rule without configuring parameters. The proposed scheme controls the next-hop adaptive flooding decisions and addresses the scalability issue regardless of the trained range of network size, traffic profiles, and mobility patterns.