# Channel-Hopping Using Reinforcement Learning for Rendezvous in Asymmetric Cognitive Radio Networks
## 2 System Model and Related Works
### Related Works  
In the context of cognitive radio networks (CRNs), achieving efficient and reliable rendezvous among secondary users (SUs) remains a critical challenge. Various channel-hopping sequences have been proposed to address this issue, each with its own strengths and weaknesses. The JS hopping sequence is a widely studied method designed to facilitate rendezvous in asynchronous CRNs [6; 7]. This method involves SUs periodically \"jumping\" to different channels based on a predefined sequence and \"staying\" on these channels for a certain duration to increase the likelihood of successful rendezvous. Figure 2 illustrates an example of the operation of the JS algorithm. Here, \(M\) represents the total number of channels, and \(P\) denotes the smallest prime number greater than \(M\). It can be observed that the two SUs successfully attempt rendezvous on channel 1 during the second trial. This channel exploration is conducted based on the JS hopping sequence, where hopping (jumping) occurs up to the 10 th step, followed by a stay behavior. The deterministic nature of the JS  
Figure 1: Structure of rendezvous. sequence ensures predictable channel coverage, making it straightforward to implement in scenarios where global time synchronization is absent. However, its fixed hopping pattern can lead to inefficiencies in dynamic environments where channel availability varies frequently. Additionally, the potential for increased rendezvous time and collisions in dense networks limits its scalability and adaptability. M-sequences, or maximum-length sequences, offer an alternative approach, leveraging pseudorandom properties to reduce the predictability of channel-hopping and minimize collisions [13]. Generated by linear feedback shift registers (LFSRs), M-sequences ensure balanced channel usage and guarantee that all channels are eventually covered within a fixed period. While M-sequences provide robust collision avoidance and pseudorandomness, they introduce additional complexity in generation and implementation compared to simpler methods like JS sequences. Moreover, the fixed length of M-sequences, determined by the LFSR configuration, may not always align well with the number of available channels, leading to potential inefficiencies in certain scenarios [8; 11]. While the JS hopping sequence offers simplicity and deterministic channel coverage, it is less adaptable to dynamic environments and may face scalability issues in larger networks. On the other hand, M-sequence-based hopping provides better pseudorandomness and balanced channel usage, making it more suitable for environments where collision avoidance is critical. However, its complexity and fixed sequence length may pose challenges. Prime number-based sequences strike a balance between simplicity and collision avoidance but require careful tuning to ensure even channel coverage and avoid periodic collisions [10]. The choice of hopping sequence method should be guided by the specific requirements of the network, including the need for flexibility, computational efficiency, and robustness against collisions. Recent advances in cognitive radio networks (CRNs) have seen the application of reinforcement learning (RL) techniques to solve the rendezvous problem [5]. RL offers the advantage of being applicable in asymmetric models and is particularly beneficial in scenarios where mathematically defining a rendezvous sequence is challenging. Through learning, agents can autonomously discover optimal strategies. Prior research has predominantly focused on adversarial bandit-based approaches, where channel selection probabilities are dynamically adjusted based on past experiences to enhance the likelihood of successful rendezvous. While these methods effectively adapt to changing environments and optimize channel selection strategies, they often fall short in their reward settings. Specifically, they may not sufficiently minimize unnecessary attempts or reduce superfluous channel exploration.