## 5 Performance Evaluation
### Simulation Setup
#### 5.1.5 Decision Model  
The decision model is based on the DRL framework, so we introduce it as two parts: environment and agent.  
Environment: The input of the decision model is the output of the estimation model, and the estimation model can achieve high accuracy, so we used accurate SNR data directly in training the decision model. The initial state in the environment is the state at the first moment.  
After the agent has selected the action for that step, the environment will step forward accordingly, simulating time change in the real world. We classify 80% of the data as the training set and the other 20% as the test set. We consider one interaction between the satellite and the UE as a trajectory and test the current model performance after collecting one trajectory.  
The environment also calculates the throughput based on Equation (6) and returns it to the agent to update its parameters.  
Agent: The agent part is mainly composed of the actor and the critic. The neural network structure of actor and critic is \((256,128,64,3)\) neurons in each layer. The activation function between layers is \(relu(\cdot)\), and the output layer goes through a \(softmax\) function.  
The gradient algorithm we adopted is the PPO algorithm mentioned in Algorithm 1, whose memory \(||\mathbf{M}||\) size is 8192, batch size \(b\) is 2048, repeat time \(N\) is 40 times, and maximum number of steps, \(S_{max}\) is 60k. The network parameters are updated every \(||\mathbf{M}||\) step, and the test set data are run once to ensure that the network is not overfitted. The learning rate of both actor and critic networks is 0.001.  
The forgetting factor \(\gamma\) is discussed in the following sections, and we discuss the case when \(\gamma\) is \((0.3,0.5,0.7,0.9)\) separately. A more prominent forgetting factor means that the system values historical data more, while a smaller forgetting factor means that the system is more straightforward and related to nearby values.  
The baseline in the decision model experiment comprises the following.  
* Select Data Rate by BER [16]: The BER is the ratio of the erroneous bits to the total number of bits in a frame, and the BER decreases as the SNR increases. The data rate is selected according to BER as the highest data rate among the coding schemes with BER less than \(10^{-5}\). To simplify writing and drawing, we will hereafter refer to this method as \"BER\".
* Select Data Rate by FER [15]: FER is the probability that there is an error code in a frame. FER can be calculated as \[FER=1-(1-BER)^{L}\] (21) where \(L\) is the length of a frame. FER decreases rapidly as SNR rises. The data rate is selected according to FER as the highest data rate among the coding schemes with FER less than 0.1. To simplify writing and drawing, we will hereafter refer to this method as \"FER\".  
To verify the convergence and stability of the algorithm, the experiments were repeated three times for each pair of parameters. To demonstrate the necessity of the estimation model, we also use the state \((lat_{t},lont_{t},x_{t},y_{t},z_{t},w_{t})\) in the environment directly in the agent in the experiment, instead of feeding it to its predicted SNR, and the results will be described in Section 5.3.