# A Fuzzy-Logic-Based Load Balancing Scheme for a Satellite-Terrestrial Integrated Network
## 3 Results
### Performance of the Fuzzy-Logic- and Reinforcement-Learning-Based Access Algorithm  
The FDDPG model utilized in FLRL-AC was trained by Algorithm 1 and the hyper-parameters in Table 2. In order to reflect the improvements of FDDPG, two other baseline algorithms were utilized to train the neural networks. PPO is based on an actor-critic network similar to that of DDPG, which can solve the problem of continuous control. It balances the difficulty of implementation, the complexity of sampling and the effort required for debugging. For these reasons, it is widely used as a default reinforcement learning algorithm for new problems. The other baseline algorithm was DDPG without the ANFN, in which the state function and reward function are defined as  
\[s^{t}=\big{(}\zeta^{t}(1,1),\zeta^{t}(1,2),\cdots,\zeta^{t}(M,N)\big{)}, \tag{23}\]  
\[r_{1}^{t}=\frac{\sum_{m=1}^{M}\zeta\big{(}m,AC^{t}(m)\big{)}}{M}, \tag{24}\]  
instead of Equations (8) and (11); and \(\zeta^{t}(m,n)\) means SNR between user \(m\) and cell \(n\) at time \(t\). \begin{table}
\begin{tabular}{c c} \hline
**Parameter** & **Value** \\ \hline \(L\) & 3500 m \\ \(N\) & 3 \\ \(M\) & 15\(\sim\)90 \\ \(D\) & 1732 m \\ \(B_{g}\) & 30 MHz \\ \(B_{s}\) & 400 MHz \\ \(\epsilon\) & 0.1 \\ \(p_{s}\) & 46 dBm \\ \(N_{0}\) & \(5\times 10^{-5}\) w \\ \(\mu_{0}\) & 0 \\ \(\theta_{h}\) & 1 \\ \(v\) & 10 m/s \\ \(R_{agg}\) & 10 Mbps \\ \(p\) & 5 \\ \(Q\) & 5 \\ \(R\) & 5 \\ \(\Theta_{g}\) & 85\% \\ \(\Theta_{s}\) & 30\% \\ \hline \end{tabular}
\end{table} 
Table 1: Parameter configuration. Figure 4: Input membership functions. The environment was reset and users were dropped randomly in each drop. Users accessed cells based on different algorithms at time slot 0 and moved in later time slots. The overload ratio (OLR) is defined as  
\[OLR=\frac{N_{ol}}{N_{total}}, \tag{25}\]  
where \(N_{ol}\) is the number of time slots in which overloading occurs and \(N_{total}\) is the total number of the simulation slots. Results with different access algorithms are shown in Figure 5. RAND means users access randomly, which is considered as the lower limit. BCA [30] means users access with the best channel state. As the number of users increases, the OLR increases for all algorithms. The purpose of designing FDDPG and FLRL-AC was to pre-process the factors affecting overloading by ANFN and to reduce the training difficulty of the network in a high-dimensional state and action space. Additionally, the FEM calculated by ANFN gives a more accurate evaluation of the impact of users on overloading. Therefore, taking the statistics of FEM as the reward function could enable the agent to make better decisions. The advantages brought by FDDPG are not obvious when the number of users is small and the training dimensions are not too high. Therefore, the three curves of PPO, DDPG and FDDPG are close. As the number of users increases, the results of BCA become unacceptable because the resources of NTN cell are not effectively utilized, resulting in overloading of TN cells. PPO and DDPG have similar performances to the two baseline algorithms, and the gain brought about by FDDPG becomes more and more obvious with the increase users: up to 29%. Users who achieve their data requirements are called satisfied users. The satisfied ratio (SR) is defined as  
\[SR=\frac{M_{s}}{M_{total}}, \tag{26}\]  
where \(M_{s}\) is the number of satisfied users and \(M_{total}\) is the total number of users. Results in Figure 6 show that with RAND and BCA, many users fail to achieve their target rates. With the three access algorithms based on reinforcement learning, there are always more than 97% of users who achieve their data rate requirements, which again proves that active load balancing can significantly improve QoS.