## 3 Preliminaries: Network Model and Problem Definition
### TOMAC-PPO Algorithm Process
Experimental results show that this algorithm has better convergence speed and can effectively reduce the service cost, energy consumption, and task drop rate under high load and high failure rates. For example, the proposed TOMAC-PPO can reduce the average cost by from 19.4% to 66.6% compared to other offloading schemes under the same network load. In addition, the drop rate of some baseline algorithms with 50 users can achieve 62.5% for critical tasks, while the proposed TOMAC-PPO only has 5.5%. In future works, due to the innovation of the TOMDP dynamic time slot model proposed in this paper, it conflicts with existing DRL code patterns, resulting in certain encoding difficulties. Therefore, how to be compatible with existing DRL code frameworks is an area that TOMDP needs to optimize. In addition, the design of states, actions, and rewards in TOMDP is also a decisive factor in the optimization effect of the algorithm and can be further improved. Conceptualization, G.J. ; methodology and validation, R.H.; investigation and writing--original draft preparation, G.W. and Z.B. ; writing--review and editing, R.H. and G.W. All authors have read and agreed to the published version of the manuscript. This work was supported in part by the National Natural Science Foundation of China. No. 62062007, 2022 Guangxi Vocational Education Teaching Reform Research Key Project, and Research on the Six in One Talent Training Model of \"Post, Course, Competition, Certificate, Training, and Creation\" in the College of Artificial Intelligence Industry, No. CXGZJG2022A011. Design and Application of Computer Experiment Site Management System Based on Information Technology, 2023 YKYZ001. ## References

* [1] IoT and Non-IoT Connections Worldwide 2010-2025. Available online: [[https://www.statista.com/statistics/1101442/iot-number-of-connected-devices-worldwide](https://www.statista.com/statistics/1101442/iot-number-of-connected-devices-worldwide)]([https://www.statista.com/statistics/1101442/iot-number-of-connected-devices-worldwide](https://www.statista.com/statistics/1101442/iot-number-of-connected-devices-worldwide)) (accessed on 28 August 2024). * [2] IoT Is Not a Buzzword but Necessity. Available online: [[https://www.3i-infotech.com/iot-is-not-just-a-buzzword-but-has-practical-applications-even-in-industries/](https://www.3i-infotech.com/iot-is-not-just-a-buzzword-but-has-practical-applications-even-in-industries/)]([https://www.3i-infotech.com/iot-is-not-just-a-buzzword-but-has-practical-applications-even-in-industries/](https://www.3i-infotech.com/iot-is-not-just-a-buzzword-but-has-practical-applications-even-in-industries/)) (accessed on 28 August 2024). * [3] [PERSON]; [PERSON]; [PERSON]; [PERSON]; [PERSON]; [PERSON] Survey on the Methods of Computation Offloading in Molile Edge Computing. _J. Comput. Sci. Technol._**2021**, _44_, 2406-2430. * [4] [PERSON]; [PERSON]; [PERSON]; [PERSON]; [PERSON]; [PERSON]; [PERSON] Distributed artificial intelligence empowered by end-edge-cloud computing: A survey. _IEEE Commun. Surv. Tutor._**2022**, _25_, 591-624. [CrossRef]
* [5] [PERSON]; [PERSON]; [PERSON]; [PERSON]; [PERSON]; [PERSON] Edge computing with artificial intelligence: A machine learning perspective. _ACM Comput. Surv._**2023**, _55_, 1-35. [CrossRef]
* [6] [PERSON]; [PERSON]; [PERSON]; [PERSON] Offloading using traditional optimization and machine learning in federated cloud-edge-fog systems: A survey. _IEEE Commun. Surv. Tutor._**2023**, _25_, 1199-1226. [CrossRef]
* [7] [PERSON]; [PERSON]; [PERSON]; [PERSON]; [PERSON] [PERSON]; [PERSON] [PERSON] RUDDER: Return decomposition for delayed rewards. In Proceedings of the 33 rd Conference on Neural Information Processing Systems (NeurIPS 2019), Vancouver, BC, Canada, 8-14 December 2019; pp. 13544-13555. * [8] [PERSON]; [PERSON]; [PERSON]; [PERSON] Joint Optimization Method of Energy Consumption and Time Delay for Mobile Edge Computing. _J. Univ. Electron. Sci. Technol. China_**2022**, _51_, 737-742.