approach,” IEEE Journal on Selected Areas in Communications, vol. 37, no. 10, pp. 2377–2392, 2019.

[153] S. Messaoud, A. Bradai, O. B. Ahmed, P. T. A. Quang, M. Atri, and M. S. Hossain, “Deep federated q-learning-based network slicing for industrial iot,” IEEE Transactions on Industrial Informatics, vol. 17, no. 8, pp. 5572–5582, 2021.

[154] A. Gharsallah, N. Omheni, K. Ghanmi, F. Zarai, and M. Neji, “A seamless mobility mechanism for v2v communications,” in 2017 IEEE/ACS 14th International Conference on Computer Systems and Applications (AICCSA), 2017, pp. 1063–1069.

[155] R. Molina-Masegosa and J. Gozalvez, “Lte-v for sidelink 5g v2x vehicular communications: A new 5g technology for short-range vehicle-to-everything communications,” IEEE Vehicular Technology Magazine, vol. 12, pp. 30–39, 12 2017.

[156] A. B. Kihero, M. S. J. Solaija, A. Yazar, and H. Arslan, “Inter-numerology interference analysis for 5g and beyond,” in 2018 IEEE Globecom Workshops (GC Wkshps), 2018, pp. 1–6.

[157] S. D. Oro, L. Bonati, S. Member, F. Restuccia, T. Melodia, and S. Member, “Coordinated 5G Network Slicing: How Constructive Interference Can Boost Network Throughput,” pp. 1–14, 2021.

[158] D. Horgan, J. Quan, D. Budden, G. Barth-Maron, M. Hessel, H. van Hasselt, and D. Silver, “Distributed prioritized experience replay,” 2018.

[159] N. Yu, Z. Song, H. Du, H. Huang, and X. Jia, “Multi-resource allocation in cloud radio access networks,” in 2017 IEEE International Conference on Communications (ICC), 2017, pp. 1–6.

[160] A. Checko, H. L. Christiansen, Y. Yan, L. Scolari, G. Kardaras, M. S. Berger, and L. Dittmann, “Cloud ran for mobile networks—a technology overview,” IEEE Communications Surveys Tutorials, vol. 17, no. 1, pp. 405–426, 2015.

[161] G. Sun, K. Xiong, G. O. Boateng, G. Liu, and W. Jiang, “Resource slicing and customization in RAN with dueling deep Q-Network,” Journal of Network and Computer Applications, vol. 157, no. July 2019, p. 102573, 2020. [Online]. Available: https://doi.org/10.1016/j.jnca.2020.102573

[162] K. Yu, H. Zhou, B. Qian, Z. Tang, and X. Shen, “A Reinforcement Learning Aided Decoupled RAN Slicing Framework for Cellular V2X,” 2020 IEEE Global Communications Conference, GLOBECOM 2020 - Proceedings, vol. 2020-January, pp. 2–7, 2020.

[163] Q. Liu, T. Han, N. Zhang, and Y. Wang, “DeepSlicing: Deep reinforcement learning assisted resource allocation for network slicing,” arXiv, 2020.

[164] G. Sun, G. O. Boateng, D. Ayepah-Mensah, G. Liu, and J. Wei, “Autonomous Resource Slicing for Virtualized Vehicular Networks with D2D Communications Based on Deep Reinforcement Learning,” IEEE Systems Journal, vol. 14, no. 4, pp. 4694–4705, 2020.

[165] Y. Cui, X. Huang, P. He, D. Wu, and R. Wang, “A two-timescale resource allocation scheme in vehicular network slicing,” in 2021 IEEE 93rd Vehicular Technology Conference (VTC2021-Spring), 2021, pp. 1–5.

[166] J. Mei, X. Wang, and K. Zheng, “An intelligent self-sustained RAN slicing framework for diverse service provisioning in 5G-beyond and 6G networks,” Intelligent and Converged Networks, vol. 1, no. 3, pp. 281–294, 2021.

[167] F. Wei, G. Feng, Y. Sun, Y. Wang, S. Qin, and Y. C. Liang, “Network Slice Reconfiguration by Exploiting Deep Reinforcement Learning with Large Action Space,” IEEE Transactions on Network and Service Management, vol. 17, no. 4, pp. 2197–2211, 2020.

[168] X. Zhang, W. Lu, B. Li, and Z. Zhu, “DRL-based network orchestration to realize cooperative, distributed and tenant-driven virtual network slicing,” Optics InfoBase Conference Papers, vol. Part F138-, pp. 2019–2021, 2019.

[169] W. Lu, H. Fang, and Z. Zhu, “AI-assisted resource advertising and pricing to realize distributed tenant-driven virtual network slicing in inter-DC optical networks,” 22nd Conference on Optical Network Design and Modelling, ONDM 2018 - Proceedings, pp. 130–135, 2018.

[170] T. Li, X. Zhu, and X. Liu, “An End-to-End Network Slicing Algorithm Based on Deep Q-Learning for 5G Network,” IEEE Access, vol. 8, pp. 122 229–122 240, 2020.

[171] G. Yang, Q. Liu, X. Zhou, Y. Qian, and W. Wu, “Two-tier resource allocation in dynamic network slicing paradigm with deep reinforcement learning,” in 2019 IEEE Global Communications Conference (GLOBECOM), 2019, pp. 1–6.

[172] F. Yang, W. Wu, X. Wang, Y. Zhang, and P. Si, “Deep reinforcement learning based handoff algorithm in end-to-end network slicing enabling hetnets,” in 2021 IEEE Wireless Communications and Networking Conference (WCNC), 2021, pp. 1–7.

[173] Y. Sun, W. Jiang, G. Feng, P. V. Klaine, L. Zhang, M. A. Imran, and Y.-C. Liang, “Efficient handover mechanism for radio access network slicing by exploiting distributed learning,” IEEE Transactions on Network and Service Management, vol. 17, no. 4, pp. 2620–2633, 2020.

[174] A. Laghrissi and T. Taleb, “A survey on the placement of virtual resources and virtual network functions,” IEEE Communications Surveys Tutorials, vol. 21, no. 2, pp. 1409–1434, 2019.

[175] G. Wang, G. Feng, T. Q. Quek, S. Qin, R. Wen, and W. Tan, “Reconfiguration in network slicing-optimizing the profit and performance,” IEEE Transactions on Network and Service Management, vol. 16, no. 2, pp. 591–605, 2019.

[176] M. Pozza, P. K. Nicholson, D. F. Lugones, A. Rao, H. Flinck, and S. Tarkoma, “On Reconfiguring 5G Network Slices,” IEEE Journal on Selected Areas in Communications, vol. 38, no. 7, pp. 1542–1554, 2020.

[177] A. Tavakoli, F. Pardo, and P. Kormushev, “Action branching architectures for deep reinforcement learning,” CoRR, vol. abs/1711.08946, 2017. [Online]. Available: http://arxiv.org/abs/1711.08946

[178] C. Kachris and I. Tomkos, “Optical interconnection networks for data centers,” in 2013 17th International Conference on Optical Networking Design and Modeling (ONDM), 2013, pp. 19–22.

[179] X. Xue, F. Wang, F. Agraz, A. Pagès, B. Pan, F. Yan, X. Guo, S. Spadaro, and N. Calabretta, “Sdn-controlled and orchestrated opsquare dcn enabling automatic network slicing with differentiated qos provisioning,” Journal of Lightwave Technology, vol. 38, no. 6, pp. 1103–1112, 2020.

[180] H. Jiang, Y. Wang, L. Gong, and Z. Zhu, “Availability-aware survivable virtual network embedding in optical datacenter networks,” IEEE/OSA Journal of Optical Communications and Networking, vol. 7, no. 12, pp. 1160–1171, 2015.

[181] Y.-J. Liu, G. Feng, J. Wang, Y. Sun, and S. Qin, “Access control for ran slicing based on federated deep reinforcement learning,” in ICC 2021 - IEEE International Conference on Communications, 2021, pp. 1–6.

[182] C. Mei, J. Liu, J. Li, L. Zhang, and M. Shao, “5G network slices embedding with sharable virtual network functions,” Journal of Communications and Networks, vol. 22, no. 5, pp. 415–427, 2020.

[183] D. Harutyunyan, R. Fedrizzi, N. Shahriar, R. Boutaba, and R. Riggio, “Orchestrating End-to-end Slices in 5G Networks,” 15th International Conference on Network and Service Management, CNSM 2019, no. Cnsm, 2019.

[184] H. Wang, Y. Wu, G. Min, J. Xu, and P. Tang, “Data-driven dynamic resource scheduling for network slicing: A Deep reinforcement learning approach,” Information Sciences, vol. 498, pp. 106–116, 2019. [Online]. Available: https://doi.org/10.1016/j.ins.2019.05.012

[185] F. Fossati, S. Moretti, P. Perny, and S. Secci, “Multi-resource allocation for network slicing,” IEEE/ACM Transactions on Networking, vol. 28, no. 3, pp. 1311–1324, 2020.

[186] “Applying machine learning to end-to-end slice SLA decomposition,” in Proceedings of the 2020 IEEE Conference on Network Softwarization: Bridging the Gap Between AI and Network Softwarization, NetSoft 2020, 2020, pp. 92–99.

[187] G. Kibalya, J. Serrat, J. L. Gorricho, R. Pasquini, H. Yao, and P. Zhang, “A Reinforcement Learning Based Approach for 5G Network Slicing across Multiple Domains,” 15th International Conference on Network and Service Management, CNSM 2019, no. Cnsm, 2019.

[188] W. Guan, H. Zhang, and V. C. Leung, “Slice reconfiguration based on demand prediction with dueling deep reinforcement learning,” in GLOBECOM 2020 - 2020 IEEE Global Communications Conference, 2020, pp. 1–6.

[189] A. Rkhami, Y. Hadjadj-Aoul, and A. Outtagarts, “Learn to improve: A novel deep reinforcement learning approach for beyond 5g network slicing,” in 2021 IEEE 18th Annual Consumer Communications Networking Conference (CCNC), 2021, pp. 1–6.

[190] J. J. Alves Esteves, A. Boubendir, F. Guillemin, and P. Sens, “On the Robustness of Controlled Deep Reinforcement Learning for Slice Placement,” Journal of Network and Systems Management, vol. 30, no. 3, pp. 1–10, 2022.

[191] R. A. Addad, D. L. C. Dutra, T. Taleb, and H. Flinck, “AI-Based Network-Aware Service Function Chain Migration in 5G and beyond Networks,” IEEE Transactions on Network and Service Management, vol. 19, no. 1, pp. 472–484, 2022.

[192] F. Zhou, P. Yu, L. Feng, X. Qiu, Z. Wang, L. Meng, M. Kadoch, L. Gong, and X. Yao, “Automatic Network Slicing for IoT in Smart City,” IEEE Wireless Communications, vol. 27, no. 6, pp. 108–115, 2020.
[193] A. Ksentini, M. Jebalia, and S. Tabbane, "Fog-enabled Industrial IoT Network Slicing model based on ML-enabled Multi-objective Optimization," pp. 177-180, 2021.

[194] C.-C. Wu, V. Friderikos, and C. Stefanovic, "Multi-Objective Provisioning of Network Slices using Deep Reinforcement Learning," pp. 1-15, 2022. [Online]. Available: http://arxiv.org/abs/2207.13821

[195] N. Van Huynh, D. T. Hoang, D. N. Nguyen, and E. Dutkiewicz, "Real-Time Network Slicing with Uncertain Demand: A Deep Learning Approach," IEEE International Conference on Communications, vol. 2019-May, 2019.

[196] J. Koo, V. B. Mendiratta, M. R. Rahman, and A. Walid, "Deep Reinforcement Learning for Network Slicing with Heterogeneous Resource Requirements and Time Varying Traffic Dynamics," 15th International Conference on Network and Service Management, CNSM 2019, 2019.

[197] F. Mason, G. Nencioni, and A. Zanella, "A multi-agent reinforcement learning architecture for network slicing orchestration," in 2021 19th Mediterranean Communication and Computer Networking Conference (MedComNet), 2021, pp. 1-8.

[198] Q. Liu, T. Han, and E. Moges, "Edgeslice: Slicing wireless edge computing network with decentralized deep reinforcement learning," CoRR, vol. abs/2003.12911, 2020. [Online]. Available: https://arxiv.org/abs/2003.12911

[199] S. Abdulrahman, H. Tout, H. Ould-Slimane, A. Mourad, C. Talhi, and M. Guizani, "A survey on federated learning: The journey from centralized to distributed on-site learning and beyond," IEEE Internet of Things Journal, vol. 8, no. 7, pp. 5476-5497, 2021.

[200] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin, "Attention is all you need," 2017.

[201] N. Wu, B. Green, X. Ben, and S. O'Banion, "Deep transformer models for time series forecasting: The influenza prevalence case," ArXiv, vol. abs/2001.08317, 2020.

[202] T. T. Nguyen, N. D. Nguyen, and S. Nahavandi, "Deep reinforcement learning for multiagent systems: A review of challenges, solutions, and applications," IEEE Transactions on Cybernetics, vol. 50, no. 9, pp. 3826-3839, 2020.

[203] C. Liu, X. Xu, and D. Hu, "Multiobjective reinforcement learning: A comprehensive overview," IEEE Transactions on Systems, Man, and Cybernetics: Systems, vol. 45, no. 3, pp. 385-398, 2015.

[204] S. Lange and M. Riedmiller, "Deep auto-encoder neural networks in reinforcement learning," in The 2010 International Joint Conference on Neural Networks (IJCNN), 2010, pp. 1-8.

[205] N. Kornfeld and Z. Feng, "A latent variable model state estimation system for image sequences," in 2019 22th International Conference on Information Fusion (FUSION), 2019, pp. 1-8.

[206] A. Hussein, M. M. Gaber, E. Elyan, and C. Jayne, "Imitation learning: A survey of learning methods," ACM Comput. Surv., vol. 50, no. 2, Apr. 2017. [Online]. Available: https://doi.org/10.1145/3054912

[207] T. M. Hospedales, A. Antoniou, P. Micaelli, and A. J. Storkey, "Meta-learning in neural networks: A survey," IEEE Transactions on Pattern Analysis & Machine Intelligence, no. 01, pp. 1-1, may 5555.

[208] S. Saha, Z. Gan, L. Cheng, J. Gao, O. L. Kafka, X. Xie, H. Li, M. Tajdari, H. A. Kim, and W. K. Liu, "Hierarchical Deep Learning Neural Network (HiDeNN): An artificial intelligence (AI) framework for computational science and engineering," Computer Methods in Applied Mechanics and Engineering, vol. 373, no. October, 2021.

[209] K. Kowsari, D. E. Brown, M. Heidarysafa, K. Jafari Meimandi, M. S. Gerber, and L. E. Barnes, "Hdltex: Hierarchical deep learning for text classification," 2017 16th IEEE International Conference on Machine Learning and Applications (ICMLA), Dec 2017. [Online]. Available: http://dx.doi.org/10.1109/ICMLA.2017.0-134

[210] L. Zhang, L. Cheng, H. Li, J. Gao, C. Yu, R. Domel, Y. Yang, S. Tang, and W. K. Liu, "Hierarchical deep-learning neural networks: finite elements and beyond," Computational Mechanics, vol. 67, no. 1, pp. 207-230, 2021. [Online]. Available: https://doi.org/10.1007/s00466-020-01928-9

[211] H. Chergui, L. Blanco, L. A. Garrido, K. Ramantas, S. Kuklinski, A. Ksentini, and C. Verikoukis, "Zero-Touch AI-Driven Distributed Management for Energy-Efficient 6G Massive Network Slicing," IEEE Network, vol. 35, no. 6, pp. 43-49, 2021.

[212] M. A. Hossain, A. R. Hossain, and N. Ansari, "AI in 6G: Energy-Efficient Distributed Machine Learning for Multilayer Heterogenous Networks," IEEE Network, 2022.

[213] H. Larsson, H. Riaz, and S. Ickin, "Automated Collaborator Selection for Federated Learning with Multi-armed Bandit Agents," FlexNets 2021 - Proceedings of the 4th FlexNets Workshop on Flexible Networks,

Artificial Intelligence Supported Network Flexibility and Agility, Part of SIGCOMM 2021, pp. 44-49, 2021.

<figure><img src="image_11.png" /></figure>

HNIN PANN PHYU received the M.Sc. degree in communication networks and services from the Institut Mines-Telecom, France, in 2016, the M.Eng. degree in telecommunications from the Asian Institute of Technology, Thailand, in 2016. From 2016 to 2020, she was a Network Strategist and Architect Engineer with Telenor, Myanmar. She is currently pursuing the Ph.D. degree in Software and IT Engineering with École de technologie supérieure University, Montreal, Canada. Her

current research interests include next generation mobile communication systems, machine learning, big data analytics, resource management and network slicing.

<figure><img src="image_12.png" /></figure>

DIALA NABOULSI is an Assistant Professor at the École de Technologie Supérieure (ÉTS), Canada. Before that, she held a Research Professional position in the Ultra-TCS research chair, at ÉTS, Canada, a Research Associate position and a Postdoctoral Researcher position at Concordia University, Canada. She was also a Visiting Researcher at Ericsson, Canada. She held Course Lecturer positions at McGill University, Canada and Concordia University, Canada. She obtained

the Ph.D. degree in Computer Science from INSA Lyon, France in 2015. As part of a double degree program, she received in 2012 the M.Sc. degree in Computer Science from INSA Lyon, France and the M.Eng. degree in Telecommunications from the Lebanese University, Lebanon. Her research interests are in mobile networks, virtualized networks, and wireless networks. She is the holder of an NSERC Discovery Grant (2020-2026), on network slicing in future mobile networks. She has collaborations with several providers in communications technology such as Ultra Intelligence & Communications and Octasic.

<figure><img src="image_13.png" /></figure>

RAZVAN STANICA is an associate professor at INSA Lyon, France, and a research scientist with the Inria Agora team of the CITI laboratory. He obtained a M.Eng. degree and a Ph.D. in computer science, both from INP Toulouse, France, in 2008 and 2011 respectively. His research interests are revolving around wireless mobile networks, with a special focus on communication networks in urban environments.