# The Framework of 6G Self-Evolving Networks and the Decision-Making Scheme for Massive IoT
## 1 Introduction
proposed the self-learning AI architecture for 6G edge intelligence [35]. Analyzing the above works, we found that the existing research on 6G intelligent networking is not completely free from manual intervention, or considering litter on the self-update of the AI models and the self-evolution of the networks. However, in some massive IoT scenarios, the number of the active devices and the service requirements are constantly changing, which requires the network to be scalable and evolvable. Thus, we propose the framework of self-evolving networks, which is the closed-loop framework with the abilities of autonomous sensing, autonomous decision-making, and autonomous control, and the AI models can realize the autonomous updates to support the evolution of the networks. Table 3 gives the differences among our work and the other existing works. It can be seen that our proposed framework is more suitable for the massive IoT scenarios. \begin{table}
\begin{tabular}{l l l} \hline \hline  & **5G** & **6G** \\ \hline Architecture & Service-based architecture & AI-embedded architecture \\ Network functions & Software-defined network (SDN) to realize the high efficiency and flexibility & AI-embedded to realize the totally network intelligence \\ AI abilities & Utilizing AI to enhance some network & Realize autonomous sensing, autonomous decision-making, and autonomous control \\ \hline \hline \end{tabular}
\end{table} 
Table 2: The differences of AI abilities in 5G and 6G. There are many researches on autonomous decision-making schemes. For example, reference [36] discussed the cell association problem in ultra-dense networks and proposed the Q-learning and deep Q networks (DQN)-based intelligent fast cell association algorithms. As for the task offloading and resource allocation problem, Reference [37] proposed a reinforcement learning (RL)-based joint task offloading and migration schemes to raise the total revenue of mobile users. Reference [38] proposed the DQN and deep deterministic policy gradient (DDPG)-based task offloading and resource allocation to reduce the sum cost of tasks. Reference [39] proposed an multi-agent deep reinforcement learning (MADRL)-based joint bit rate selection and radio resource allocation scheme in fog-computing based radio access networks. Reference [40] proposed the DDPG based computing offloading algorithm to reduce the total system delay cost. Additionally, many researchers focus on the autonomous decision-making of the vehicle platoon networks, and reference [41] proposed the DRL-based joint resource management algorithm to improve the communication and computing efficiency in maritime network. Reference [42] proposed the reinforcement learning-based task offloading scheme to optimize the price decision and computing resource allocation in the vehicle platoon networks. Reference [43,44] proposed DDPG algorithm for decision-making in vehicular networks. There are few works focusing on the supporting network and the distributed decision-making schemes for massive IoT scenarios. In this paper, we creatively propose the framework of 6G self-evolving networks which have the abilities of autonomous sensing, autonomous decision-making, and autonomous control and can realize the self-evolution without the human involvement. Then, we propose the distributed dueling double DQN (DQQN)-based decision-making algorithm to obtain the optimal task offloading and resource allocation policy. Compared with Q-learning, DQN and double DQN (DDQN) algorithms, and other centralized algorithms, our method can avoid the overestimation of the Q value, and the distributed decision-making mechanism is more scalable. In the rest of the paper, we first propose the framework of 6G self-evolving networks and introduce the common decision-making mechanism in Section 2. Then, we propose the distributed decision-making scheme used in massive IoT, including the system model, the algorithm, and the simulation results in Section 3. Section 4 shows the use cases of the