## 3 Methodology
### Q-Learning Integration in GA Mutation
The use of lowercase a denotes that the maximization is performed over all possible actions available at state \(\mathrm{S_{t+1}}\). This iterative process ensures that drones develop an optimal movement strategy that balances exploration and exploitation, ultimately leading to superior coverage and connectivity in dynamic environments. In order to compare the efficacy of various optimization methods, we measured important connectivity parameters such as the minimum and maximum connection distance encountered, mean connection distance, and total connections established. Table 1 outlines the modular structure of the hybrid optimization algorithm combining Genetic Algorithms and Q-Learning for drone deployment. To clarify the interaction between the genetic algorithm and Q-learning in the hybrid framework, we provide a high-level pseudocode representation in Algorithm 1. This complements Table 1 and summarizes the overall optimization loop in a concise manner. Input: Mission area, drone types, communication limits, GA and RL parameters Output: Optimized drone positions ensuring coverage, connectivity, and energy efficiency  
1. Initialize population with valid drone positions  
2. Initialize Q-table with default values  
3. FOR generation = 1 to \(\max\_\)generations DO  
4. Evaluate fitness of each individual using multi-objective function  
5. Apply selection and crossover  
6. FOR each individual DO  
7. Discretize state (position, coverage, energy, connectivity)  
8. Select RL-based action via \(\varepsilon\)-greedy policy  
9. Apply Q-learning based mutation to refine individual  
10. Update Q-table based on reward signal  
11. Return best solution found \begin{table}
\begin{tabular}{|l l|} \hline \multicolumn{2}{|l|}{Initialization} \\ - & Define polygon area and station location \\ - & Set drone types (FIXED\_WING, MULTIROTOR) \\ - & Define communication limits, population size, generations \\ - & Initialize Q-learning parameters \\ - & \(\eta\)—learning rate \\ - & \(\delta\)—discount factor \\ - & \(\epsilon\)—exploration rate \\ - & Q-table default initialization \\ - & Action discretization granularity \\ \multicolumn{2}{|l|}{Drone Behavior} \\ - & FIXED\_WING: prefers outer, longer-range points \\ - & MULTIROTOR: prefers inner, shorter-range points \\ \multicolumn{2}{|l|}{Utility Functions} \\ - & IS\_VALID\_POINT(point, type) \(\rightarrow\) Returns if within bounds and range \\ - & CALCULATE\_CHAIN\_DISTANCE(positions, chain) \(\rightarrow\) Computes total link \\ \multicolumn{2}{|l|}{distance} \\ \multicolumn{2}{|l|}{Genetic Algorithm} \\ - & CREATE\_INDIVIDUAL() \(\rightarrow\) Generates valid drone positions \\ - & EVALU(individual) \(\rightarrow\) Calculates fitness with penalties/bonuses \\ - & MUTATE(individual) \(\rightarrow\) Performs bounded mutations \\ \multicolumn{2}{|l|}{Q-Learning Integration} \\ - & DISCRETIZE\_STATE(positions) \(\rightarrow\) Maps continuous to discrete state \\ - & GET\_RL\_ACTION(state, \(\epsilon\)) \(\rightarrow\) Chooses action via epsilon-greedy policy \\ - & APPLY\_RL\_IMPROVEMENT(individual) \(\rightarrow\) Enhances individual via Q-table \\ \multicolumn{2}{|l|}{Hybrid Optimization Process} \\ - & Initialize population and Q-table \\ - & For each generation: \\ - & \(\bullet\) & Evolve population using GA \\ - & \(\bullet\) & Apply Q-learning improvements \\ - & \(\bullet\) & Evaluate fitness and update best \\ - & Save Q-table and return best solution \\ \multicolumn{2}{|l|}{Visualization and Evaluation} \\ - & PLOT\_SOLUTION(solution) \(\rightarrow\) Displays layout, stats, and connections \\ - & COMPARE\_RESULTS(ga\_solution, hybrid\_solution) \(\rightarrow\) Evaluates fitness, \\ \multicolumn{2}{|l|}{coverage, and connectivity metrics} \\ \multicolumn{2}{|l|}{Execution} \\ - & Run GA-only and hybrid algorithms \\ - & Compare and visualize outcomes \\ \hline \end{tabular}
\end{table} 
Table 1: Modular Overview of the Hybrid Optimization Framework Combining GA and Q-Learning.