## 2 Related Work
### Contributions  
The movement and rotation of the HAPS due to wind pressure can cause the cell range to shift, which in turn causes the degradation of users' throughput and handovers between cells [13; 14]. With the development of Global Positioning System (GPS) technology, HAPS can control itself to recover its original position state according to GPS positioning technology and thus restore the user signal quality. That being said, the position and rotation state of HAPS will always fluctuate within a certain range due to the unpredictable wind direction and wind force. However, we can improve throughput with beam steering. Thus, to solve the degradation of received power at UEs, we propose a Deep Reinforcement Learning Evolution (DRLEA) method. In our previous work [35], we proposed a Fuzzy \(\mathcal{Q}\)-learning method. This method used Fuzzy logic in the model-free RL method named \(\mathcal{Q}\)-learning to control multiple searches in a single training step. Compared with the \(\mathcal{Q}\)-learning, the proposed Fuzzy \(\mathcal{Q}\)-learning method has a lower cost of action searching. However, we found that the throughput performance of the proposed Fuzzy \(\mathcal{Q}\)-learning method under non-uniform user distribution scenarios needs to be improved. Therefore, we proposed a DRLEA method for dynamic antenna control in the HAPS system to reduce the number of low-throughput users. The proposed DRLEA considers that each iteration in the conventional DRL is a new generation. Starting from the first generation (the first iteration), DRLEA searches for the optimal solution in the current generation and trains the DNN to learn this evolutionary process. The DRLEA records the best result of the current generation as the historical optimal solution for guiding the next generation. If the current generation cannot find better solutions to reduce low throughput users, i.e. users whose throughput is lower than the median throughput prior to antenna parameters adjustment, the mutation happens (randomly selecting an action). The process is repeated until the specified execution time is reached. Compared with conventional RL methods, such as \(\mathcal{Q}\)-learning-based and DQN-based methods, the proposed method can avoid the sub-optimal solution as much as possible. This is because, in every iteration a random initial set of parameters is used, and is then optimized, leading to less chances of falling into the local optimal. In addition, the performance of user throughput is closely related to the user's SINR and bandwidth. We can improve the user's SINR by gradually adjusting the antenna parameters based on the user's feedback. As for the user's dedicated bandwidth, it is only a function of the coverage of the antenna array in which the user is located as well as the number of users within that coverage area. Therefore, we do not necessarily need to know the channel state information (CSI) to design the beamforming matrix. Compared with obtaining an accurate CSI, obtaining the user location information, such as using the GPS technique, is less costly in terms of resources and time. It is nonetheless easier and computationally less expensive to adjust the antenna array parameters to account for the changes to the footprint of the users' locations than that of their CSI. Moreover, we implement three conventional methods, PSO, \(\mathcal{Q}\)-learning, and DQN as benchmarks to evaluate the proposed DRLEA method and show that the proposed DRLEA is still reliable and efficient. This paper's contributions can be outlined as follows:* We propose a novel DRLEA method that addresses the problem of dynamic control of the HAPS antenna parameters to decrease the number of users with low throughput. The proposed method combined the EA and DRL to avoid sub-optimal solutions. * We design a new loss function that includes not only \(\mathcal{Q}\)-value of the predicted optimal action, but also the historical optimal solutions obtained from previous training.