## 4 Simulation Results  
We present the simulation results of heterogeneous edge computing network resource scheduling in this section. The simulation process is based on Pytorch. Edge computing nodes are randomly distributed over a 1000 * 1000 m square with edge server's location of \((500,500)\) (i.e., the center of this area). Multiple tasks arrive independently in heterogeneous edge computing network. According to the various needs of the task, each task can be \begin{table}
\begin{tabular}{c c} \hline \hline
**Description** & **Value** \\ \hline heterogeneous sensor network range (m * m) & 1000 * 1000 \\ number of tasks & 20, 30, 40, 50, 60 \\ subtask data volume requirements (kbit) & [3; 5] \\ number of sensors & 100 \\ free space model magnification (J/(bit*m2)) & 10 \\ multipath transmission model magnification (J/(bit*m4)) & 0.0013 \\ learning rate & \(1\times 10^{-5}\) \\ batch size & 64 \\ discount factor & 0.99 \\ generalized advantage estimator factor & 0.9 \\ clipping parameter & 0.2 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Parameters of the sensor networks and PPO. divided into 10 subtasks with different data volume requirements. As for the reinforcement learning part, we implement the proposed PPO resource scheduling algorithm based on the reinforcement learning framework proposed in [24]. The detailed parameters of the edge computing network and PPO are given in Table 1. In order to verify the effectiveness of the algorithm, we consider two baseline strategies. * Random assignment scheme. All subtasks are randomly assigned to nodes that can perform the subtask. This scheme has low complexity, but does not achieve network optimization. * Energy greedy scheme. The subtask is assigned to the edge computing node with the lowest energy consumption to complete the subtask without exceeding the maximum load of the node. After the load reaches the upper limit, the subtask is assigned to the edge computing node with the sub-low power consumption. We first analyze the convergence of the proposed algorithm as shown in Figure 3. Based on the average cumulative reward of 200 steps in each epoch, we trained 1,000,000 steps for models with different task numbers. It can be seen that the convergence speed of PPO resource scheduling decreases with the increase of task number. This is because the larger task number corresponds to a larger state space, and also leads to an increase in the complexity of the network. Therefore, large number of tasks require more steps to train. Moreover, we can also observe that the cumulative reward is declining while the task number is increasing. This is because the reward we designed consists of network energy consumption and network load balancing. As the number of tasks increases, the energy consumption of the network increases, resulting in a decrease in cumulative rewards. We can conclude that the proposed PPO resource scheduling has a stable training process and good convergence characteristics under different task numbers. We give the total energy consumption of the network under different task numbers in Figure 4. As we expect, the energy consumption of the network always increases with the increase of the task number. In addition, we can see that the proposed PPO resource scheduling performs significantly better than the random assignment scheme in terms of energy consumption, but slightly worse than the energy greedy scheme. When the task number is 20, the network energy consumption of PPO resource scheduling is 12.3 J, which is 3.6 J higher than greedy strategy and 10.4 J lower than random assignment scheme. The reason that PPO resource scheduling network energy consumption is higher than the energy greedy scheme is that we consider both network energy consumption and network load balancing in PPO resource scheduling. In order to ensure the overall load balancing of the network, the agent chooses to deploy subtasks on more nodes, thus sacrificing the  
Figure 3: Convergence of PPO resource scheduling. performance of network energy consumption to a certain extent.