## 3 Proposed Methodology
### Proposed Reinforcement Learning with Snake Optimizer  
This section presents the RL framework integrated with SO to maximize spectrum utilization in CRN. The essential elements in CRNs are PUs and SUs, where PUs are given priority access to communication channels. The main challenge lies in allocating the idle spectrum to SUs while minimizing interference with PU activity, thereby improving overall network performance. The SO operates as an RL agent within this environment and interacts dynamically with the CRN to learn an optimal channel allocation policy. At every time step, the agent perceives the current state of the environment \(S_{t}\), which includes the set of active and available SUs over channels \(P\), PU presence information, and interference or collision status. Based on this state, the SO selects an action \(A_{t}\) aimed at minimizing interference and maximizing spectral efficiency. Upon executing the action, the environment transitions to a new state \(S_{t+1}\) and provides a reward \(R_{t}\) that evaluates the impact of the action. The reward depends on reduced collisions, increased spectral efficiency, and overall network throughput. This feedback loop allows the agent to refine its channel allocation strategy iteratively to achieve higher long-term cumulative rewards. The SO-based RL interaction in the CRN environment is illustrated in Figure 2. This closed-loop system demonstrates how the SO adapts its decisions over time in response to environmental changes, accounting for multiple PUs and SUs operating across available communication channels. Figure 2: SO interaction with CRN environment. The SO (top of Figure 2) receives the current state \(S_{t}\) from the environment, containing details about currently allocated channels and any interference, representing the underlying network conditions. SO uses this state to determine an action \(A_{t}\), representing the number of SUs allocated to each channel to optimize performance. The environment transitions to a new state \(S_{t+1}\) and computes the corresponding reward \(R_{t}\), which is then returned to SO. This feedback loop continues as SO iteratively refines its strategy, treating the received rewards as utility for improving future decisions. The primary objective is to achieve flexible and effective channel assignment that maximizes spectral efficiency while avoiding significant impact on co-channel CUs. The interaction between SO and CRN environments in the RL framework can be mathematically modeled using a set of equations that define state transitions, agent actions, and reward functions. These formulations are critical for understanding how the agent adapts and evolves its allocation strategy over time. The subsequent state \(S_{t+1}\) is determined by the current state \(S_{t}\) and the action \(A_{t}\) executed by the SO, as described below:  
\[S_{t+1}=f(S_{t},A_{t})+e_{t} \tag{1}\]  
Here, \(f(S_{t},A_{t})\) represents the deterministic transition function describing how the system changes in response to an action, and \(e_{t}\) accounts for environmental randomness or noise. The reward \(R_{t+1}\) received after taking action \(A_{t}\) in state \(S_{t}\) is defined as follows:  
\[R_{t+1}=r(S_{t},A_{t}) \tag{2}\]  
To define the reward function more precisely, we express it as a weighted combination of key performance metrics:  
\[R_{t+1}=\alpha\cdot\text{SUE}_{t}-\beta\cdot\text{CR}_{t}+\gamma\cdot\text{SC} _{t} \tag{3}\]  
where \(\text{SUE}_{t}\) is the spectrum utilization efficiency at time \(t\), \(\text{CR}_{t}\) is the collision rate, and \(\text{SC}_{t}\) is the total spectral capacity achieved. The weights \(\alpha\), \(\beta\), and \(\gamma\) are positive real-valued parameters that control the contribution of each term to the overall reward. For our experiments, we use \(\alpha=0.4\), \(\beta=0.3\), and \(\gamma=0.3\), reflecting a balanced trade-off among efficiency, interference avoidance, and throughput.