## 3 Proposed Methodology
### Proposed Reinforcement Learning with Snake Optimizer
These weights can be adjusted to prioritize different objectives depending on the network conditions or application requirements. In the context of SO, which merges RL with population-based meta-heuristics, the above reward formulation guides the evolutionary selection process. Since SO does not rely on value approximation but rather on direct fitness evaluation, we reformulate the reward as an explicit fitness function used during optimization:  
\[\text{Fitness}_{t}=\alpha\cdot\text{SUE}_{t}-\beta\cdot\text{CR}_{t}+\gamma \cdot\text{SC}_{t} \tag{4}\]  
This fitness function is effectively equivalent to the reward structure in Equation (3), but is used in the context of solution ranking and selection within the SO algorithm. Readers will encounter related formulas and definitions to this fitness formulation throughout the optimization process in the upcoming sections. The reward function \(r(S_{t},A_{t})\) evaluates the outcome of taking action \(A_{t}\) in state \(S_{t}\). It serves as the primary feedback mechanism guiding the learning process, as the agent aims to maximize the cumulative reward over time by choosing increasingly effective actions. Ultimately, the agent selects an action \(A_{t}\) at each time step based on the current state \(S_{t}\) using an optimization strategy defined by the policy:\[A_{t}=\pi(S_{t}) \tag{5}\]  
The policy \(\pi(S_{t})\) maps from states to actions. It dictates which action the agent should take in a given state to maximize long-term rewards. Together, these equations form the foundation of the RL process, enabling SO to learn and adapt its strategy for optimal channel allocation in CRN. The decision policy \(\pi(S_{t})\) in our framework is implemented in a deterministic manner. It is directly derived from the optimization logic embedded within SO, which selects actions \(A_{t}\) based on the current environment state \(S_{t}\) and the optimizer's internal population configuration and fitness evaluations. In contrast to conventional reinforcement learning methods that employ stochastic policies or neural network approximators, our policy is rule-based and deterministic, governed by the heuristic mechanisms of SO. These include the exploration and exploitation strategies influenced by parameters such as the food quantity \(Q\), environmental temperature, and individual fitness levels. This design enables the SO to make interpretable and computationally efficient decisions suitable for real-time spectrum management in dynamic cognitive radio network environments. To further clarify how SO is embedded within a formal RL framework, we emphasize that our method departs from conventional RL formulations that rely on explicit Q-value updates, value functions, or neural approximators. Instead, SO operates as a meta-heuristic-based RL agent, where the policy is implicitly optimized through population-based exploration and exploitation. The architecture of the agent can be understood in terms of the following components:  
* **State representation (\(S_{t}\)):** Each state encapsulates the current occupancy status of channels, presence of PUs, the collision history, and active SUs. * **Action (\(A_{t}\)):** The action corresponds to a specific allocation of SUs to available channels, selected based on SO's optimization process. * **Policy (\(\pi(S_{t})\)):** Defined implicitly through the heuristic rules of SO algorithm, which evolves solutions over iterations using operations like mating, exploration, and fighting. * **Reward (\(R_{t}\)):** A scalar value computed based on performance indicators such as spectrum utilization efficiency, number of collisions, and spectral capacity. * **Learning mechanism:** Instead of value iteration or gradient-based learning, SO adjusts its population of candidate solutions based on fitness, without requiring Q-value updates or neural approximators. It is important to note that the state and action spaces in the proposed RL framework are discrete.