## 5 Simulation Results  
In this section, simulation results are provided to evaluate the performance of the proposed DRL-based algorithm. We conduct the simulations through python 3.8 and Tensorflow 2.5.0. Fully-connected hidden layer with 10 neurons in both the baseline and policy networks are employed. The learning rates \(\alpha_{b}\) and \(\alpha_{p}\) are set as \(8e^{-3}\) and \(2e^{-3}\), respectively. The channel bandwidth between each IoT device and each MEC server is 200 KHz. The maximum CPU frequency of the SMS and the PMS are set as 1 GHz and 5 GHz, respectively. The length of time-slot \(\Delta\) is set as 100 ms. \(\mathcal{H}_{k,m}\) is set as \(\left\{2\times 10^{-6},4\times 10^{-6},6\times 10^{-6},8\times 10^{-6} \right\},k\in\mathcal{K},m\in\mathcal{M}\). Without loss of generality, \(c_{k,t},k\in\mathcal{K},m\in\mathcal{M},t\in\mathcal{T}\) are all set as 1000. \(\rho_{m}\) is set as \(1\times 10^{-27}\). \(\mathcal{L}_{k}\) is set as \(\left\{1.5\times 10^{4},3\times 10^{4},4.5\times 10^{4},6\times 10^{4}\right\}\) bits, \(k\in\mathcal{K}\). \(T\) is set as 40. Figure 2 shows the impacts of the initial values of \(\mathbf{\theta}\) and \(\mathbf{w}\) on the accumulated reward. The iterative algorithms are susceptible to the initial value of variables. In our paper, the initial value of \(\mathbf{\theta}\) and \(\mathbf{w}\) are randomly given, which is a common way in DRL algorithms. It can be seen from Figure 2 that different initial values of \(\mathbf{\theta}\) and \(\mathbf{w}\) have deep influences on  
the convergence performance of the our DRL based algorithm. In order to guarantee the performance of the algorithm, we must run the algorithm multiple times and select the one has best performance as the final output. Figure 3 shows the impact of the weights on the normalized number of tasks sent to the PMS per episode. It can be seen from Figure 3 that the case \((\omega_{0},\omega_{1})=(1,0)\) has the smallest number of tasks sent to the PMS. This is because the VSO only concerns the computing pressure of the PMS when \((\omega_{0},\omega_{1})=(1,0)\). When \((\omega_{0},\omega_{1})=(0,1)\), the VSO mainly tries to find a policy to minimize the number of dropped tasks, namely, to make the problem P\({}_{3,km}\) solvable. As the PMS has strongest computing power, the VSO will allocate many tasks to the PMS, which can be proofed by Figure 3. Finally, when \((\omega_{0},\omega_{1})=(1,1)\), the VSO must make a trade-off between the computing pressure on the PMS and the task dropping cost. Hence, when \((\omega_{0},\omega_{1})=(1,1)\), the number of tasks sent to the PMS per episode is higher than the case \((\omega_{0},\omega_{1})=(1,0)\) but smaller than the case \((\omega_{0},\omega_{1})=(0,1)\). Figure 4 shows the impact of the weights on the normalized number of dropped tasks per episode. When \((\omega_{0},\omega_{1})=(1,0)\), most of the tasks are allocated to the SMSs to  
Figure 3: The impact of the weights on the normalized number of tasks sent to the PMS per episode. Figure 2: The impacts of the initial value of \(\mathbf{\theta}\) and \(\mathbf{w}\) on the accumulated reward. reduce the computing pressure of the PMS. Since the computing power of the SMSs are weak, many tasks may be dropped. The case \((\omega_{0},\omega_{1})=(0,1)\) has the smallest normalized number of dropped tasks per episode, which is explained in Figure 3. Similar to that in Figure 3, the case \((\omega_{0},\omega_{1})=(1,1)\) has middle number of dropped tasks per episode. Figures 5 and 6 show performance comparisons between our proposed algorithm (\((\omega_{0},\omega_{1})=(1,1)\)) and two benchmark polices: only send to the PMS policy and random allocation policy. Only send to the PMS policy allocate all the IoT devices' tasks to the PMS, which is a policy adopted in typical single MEC server deployment scenario. Random allocation policy allocates each IoT device's task randomly to the \(M+1\) MEC servers.