# Analysis of the Role of 5G Communication Technology Based on Intelligent Sensor Network in the Construction and Design of the Internet of Things in Free Trade Zones

Authors: LingYan Meng
Journal: Journal of Sensors
 (2022)

---

## Abstract

With the rapid development of the Internet today, the number of various mobile communication terminals has increased rapidly, and 5G has also appeared. With the popularization of 5G technology, new development trends have emerged in more business scenarios. For example, in the 5G era, there will be a lot of room for development in traditional business applications. On this basis, we design a resource allocation that utilizes DQN technology to achieve 5G high-band services. First of all, considering the characteristics of boundary operation, the connection between the base station and the user, the transmission capacity of the base station to the user, and other factors are used as judgment factors, the total energy efficiency is maximized as the goal, and the requirements of mobile users are the constraints. Based on customer service quality assurance, QoS assurance is the object. On the basis of DQN, the convex optimization method is used to solve the given maximum transmission energy between nodes, and DQN is used for iterative iteration to obtain the optimal node and optimal power distribution. Through simulation experiments, the results show that the algorithm has high learning efficiency and convergence and can effectively optimize the allocation of network resources under the premise of ensuring the mobile terminal’s requirements for service quality.


---

## Body

## 1. Introduction

With the rapid development of communication technology, various types of smart devices are becoming more and more popular; the mobile phone business is also developing rapidly, and the storage capacity of the wireless network is also increasing rapidly. It is reported that in 2022, the global mobile phone data transmission will increase by 7 times compared with 2017, reaching 77.5 EB [1]. At the same time, various new service requirements and features are increasingly enriched, and higher requirements are placed on reliability, speed, and delay. These bring new problems to the optimization and design of future mobile communication networks [2]. Under the influence of the continuous increase in the types of communication services, the continuous increase in network traffic, the continuous increase in the number of terminal devices, and the continuous increase in the data transmission speed of users, the previous mobile phone communication technology has been unable to meet the future communication needs.In today’s rapid development of 5G, due to the large number of end users and various requirements, it is particularly important to realize the effective allocation of resources under the premise of meeting the needs of customers [3]. In addition, since energy dissipation is the most concerned issue in current wireless communication systems, energy conservation is already a very critical technology in future mobile networks [4]. Energy conservation is an important factor when designing such mechanisms. Therefore, on the premise of meeting customer requirements, how to reasonably and efficiently allocate resources such as network bandwidth and power is an urgent problem to be solved at present.Edge computing is a transfer of computing, processing, storage, and other functions from the center of the cloud to the edge of the network, which can better provide it with more information and better services [5]. With the development of big data technology and mobile communication technology, 5G technology has also been applied more and more and has been applied to many fields. Although 5G provides users with better services, 5G also requires more network coverage, capacity, and connectivity [6]. In order to effectively reduce the load of backhaul and meet the needs of 5G, the academic community has begun to conduct in-depth exploration of boundary computing. Services and content reduce backhaul link services and reduce end-to-end delay. IIoT integrates the characteristics of data interoperability, system interoperability, networking, etc., so that the information resources of enterprises can be effectively used, forming a brand-new industrial service system [7]. 4G will change the relationship between “people and things” and “things and things”, so as to truly “interconnect everything” [8]. IIoT realizes the interconnection between multiple devices through various communication methods, making it an advanced system that can collect, monitor, analyze, exchange, and efficiently transmit data. The emergence of IOT has enabled enterprises to transform from a simple enterprise information management system to a networked industrial automation, enabling network technology to be better applied to traditional manufacturing industries, thereby effectively alleviating problems such as waste of resources and overcapacity and improving productivity. In 5G networks, reinforcement learning (RL) has been applied in many fields, such as load balancing, interference management, and HO management [9]. As a result, RL agents interact with the surrounding environment and produce optimal behaviors. The RL agent is optimized not only based on immediate rewards using a greedy approach but also based on long-term rewards—accumulated rewards. Later DRL is an improvement over regular RL, which can handle more problems better. This paper mainly discusses two typical application schemes in 5G using IIoT technology to achieve efficient resource allocation under boundary computing. Two different situations are modeled by DRL algorithm to reduce the cost of network operation and improve the service quality of the system.


## 2. DQN-Based 5G High-Bandwidth Service Resource Allocation Method

### 2.1. 5G Edge Computing Intelligent Resource Allocation Framework

#### 2.1.1. 5G Edge Computing Service Architecture

Cloud computing has also laid a solid foundation for the emergence of edge computing, which is produced under the influence of cloud computing [10]. Cloud computing can realize the storage and operation of two services for the device. However, with the development of communication technology today, the demand for various applications is also increasing. Shortening the distance between the server and the user can not only improve the user’s experience but also reduce the cost. The emergence of technologies such as mobile edge computing, fog computing, and cloudlet all originate from the thinking that is closer to the user [11]. The essence of fog computing is still cloud computing, which mainly focuses on technologies such as resource processing and virtualization. However, an edge server is installed in the access network, which can easily collect local users’ data and serve the customers’ data and data storage, and this method can be completed in limited hops. On this basis, based on the edge server and cloud computing technology, the enhancement of the performance of the mobile terminal is realized. The three-level business structure is shown in Figure 1.Figure 1
5G edge computing service architecture.


#### 2.1.2. 5G Edge Computing Intelligent Resource Allocation Framework

Edge computing has more local services, servers closer to the data source, lower latency, more convenient positioning, and more convenient access to data. The edge server can serve clients, so it can communicate with the terminal locally, separate from other networks, so it can localize the service [12]. Since the mobile terminal service is often carried out in the access network, the interval between the mobile terminal and the user is usually one jump. Therefore, in the process of transmitting a large amount of data, it is applied to the data preprocessing, so as to realize the large amount of data analyzed. In addition, since the distance between the mobile server and the user is relatively close, more services can be implemented. Location data are available in the access network, while mobile edge computing is a combination of access network and cloud computing with the continuous development of communications and the Internet and the continuous improvement of hardware facilities and customer requirements [13]. After solving a large amount of data access, network-based edge servers must seek a more intelligent way of resource allocation to meet the different requirements of different types of users, so as to meet the ever-increasing QoS requirements. After years of development, deep reinforcement learning has application significance in many aspects. Applying it to the resource allocation system of edge servers can enable servers to complete data processing in a shorter time.


### 2.2. 5G Service Resource Allocation Method

Aiming at the high-frequency band requirements in 5G boundary computing, this paper designs a 5G boundary processing scheme based on DQN technology. Firstly, the convex optimal method is used to solve the association between multiple nodes to obtain the association between each node and obtain the minimum node energy. In this case, the DQN algorithm is used to transmit to users, which ensures not only the requirements of each user but also the energy-saving effect of the system. The diagram shows the flow of this algorithm (see Figure2).Figure 2
DQN-based 5G service resource allocation process.Reinforcement learning method is a learning method for problems in learning, which includes model reinforcement learning and model-free reinforcement learning. At present, in real life, pattern-based reinforcement learning methods are rare, and there are existing time series reinforcement learning methods and Monte Carlo methods. Among them, the Monte Carlo method reduces the convergence rate due to its large amount of variation, and the Q-Learning method can achieve better results [14].Q-Learning does not depend on the mode of the environment; it performs an action under a special condition, its value is a function of the value, an agent looks for the best action in the Q table, and in the process, it can obtain the maximum cumulative reward. In conventional Q-Learning, Q functions are defined to reflect long-term interests between behavior and a state. In summary, there is a Q table to be saved in Q-Learning; the Q of all state-movement cumulative returns is saved in the Q table, where the current state is saved in a row, and the current state associated with the next action is stored in the Q table. The data is saved in this column. In some cases, due to the large state and operation space, the amount of data in the Q table will become large, which brings great difficulties to the actual storage. Therefore, a neural network with self-learning function must be used to approximate the Q value.


### 2.3. Analysis of Simulation Results

#### 2.3.1. Simulation Settings

A simulation model of a 5G high-band service resource allocation scheme consisting of several base stations and several users is presented. This part is mainly aimed at high-bandwidth users, such as AR/VR users, long video transmission users, and music listening users (see Figure3).Figure 3
Simulation scene simulation diagram.In the simulation scheme, 5 base stations and several users are set, and it is assumed that the maximum transmission capacity of each base station is 3 W, and other simulation parameters are set as shown in Table1 [15].Table 1
Simulation parameter table.
Parameter nameParameter valueNumber of base stations5Base station maximum transmit power3 WPath loss103.8Noise power spectral density−174Fixed power consumption4.8 WBackhaul transmission power0.2 watt/mbpsCalculate power consumption10-9Calculate frequency750Operation time1 sBandwidth max20 MHzBandwidth allocation ratio0.2Playback memory pool capacity500Minimum sample size32


#### 2.3.2. Comparison of Results of Different Numbers of Users

When using DQN technology to realize 5G high-frequency band, due to too many users, the state space in DQN is too large, which will have a certain negative effect on the learning efficiency of the system [16]. To this end, we simulated four situations, namely, 30, 25, 20, and 15 users, and tested the performance of DQN under different numbers of users. Figure 4 is a comparison of excitation in four examples; a comparison of losses is shown in Figure 5 in four examples, and a comparison of energy efficiency is shown in Figure 6 in four examples.Figure 4
Comparison of reward values.Figure 5
Comparison of loss values.Figure 6
Comparison of energy efficiency values.In the reward comparison shown in Figure4, as the number of users increases, the steps to reach convergence also increase. This is due to the increase in the number of users of DQN, which increases the state and activity range of DQN and reduces the learning time, thus increasing the steps of convergence. In Figure 5, the DQN algorithm is trained for different numbers of users, and its convergence rate is consistent with the steps described above. It can be seen from Figure 6 that in the learning process of DQN, as the learning of DQN gradually approaches, the energy efficiency gradually becomes stable, lower, and more in line with the best design purpose of this model. The research results show that the learning efficiency and learning speed of DQN in the whole system will decrease due to the training step size, which shows that the learning efficiency of DQN in the whole system greatly improved, which shows that the DQN strategy selected in this paper works. Moreover, the more the number of users, the greater the energy consumption and the higher the energy utilization rate.


## 2.1. 5G Edge Computing Intelligent Resource Allocation Framework

### 2.1.1. 5G Edge Computing Service Architecture

Cloud computing has also laid a solid foundation for the emergence of edge computing, which is produced under the influence of cloud computing [10]. Cloud computing can realize the storage and operation of two services for the device. However, with the development of communication technology today, the demand for various applications is also increasing. Shortening the distance between the server and the user can not only improve the user’s experience but also reduce the cost. The emergence of technologies such as mobile edge computing, fog computing, and cloudlet all originate from the thinking that is closer to the user [11]. The essence of fog computing is still cloud computing, which mainly focuses on technologies such as resource processing and virtualization. However, an edge server is installed in the access network, which can easily collect local users’ data and serve the customers’ data and data storage, and this method can be completed in limited hops. On this basis, based on the edge server and cloud computing technology, the enhancement of the performance of the mobile terminal is realized. The three-level business structure is shown in Figure 1.Figure 1
5G edge computing service architecture.


### 2.1.2. 5G Edge Computing Intelligent Resource Allocation Framework

Edge computing has more local services, servers closer to the data source, lower latency, more convenient positioning, and more convenient access to data. The edge server can serve clients, so it can communicate with the terminal locally, separate from other networks, so it can localize the service [12]. Since the mobile terminal service is often carried out in the access network, the interval between the mobile terminal and the user is usually one jump. Therefore, in the process of transmitting a large amount of data, it is applied to the data preprocessing, so as to realize the large amount of data analyzed. In addition, since the distance between the mobile server and the user is relatively close, more services can be implemented. Location data are available in the access network, while mobile edge computing is a combination of access network and cloud computing with the continuous development of communications and the Internet and the continuous improvement of hardware facilities and customer requirements [13]. After solving a large amount of data access, network-based edge servers must seek a more intelligent way of resource allocation to meet the different requirements of different types of users, so as to meet the ever-increasing QoS requirements. After years of development, deep reinforcement learning has application significance in many aspects. Applying it to the resource allocation system of edge servers can enable servers to complete data processing in a shorter time.


## 2.1.1. 5G Edge Computing Service Architecture

Cloud computing has also laid a solid foundation for the emergence of edge computing, which is produced under the influence of cloud computing [10]. Cloud computing can realize the storage and operation of two services for the device. However, with the development of communication technology today, the demand for various applications is also increasing. Shortening the distance between the server and the user can not only improve the user’s experience but also reduce the cost. The emergence of technologies such as mobile edge computing, fog computing, and cloudlet all originate from the thinking that is closer to the user [11]. The essence of fog computing is still cloud computing, which mainly focuses on technologies such as resource processing and virtualization. However, an edge server is installed in the access network, which can easily collect local users’ data and serve the customers’ data and data storage, and this method can be completed in limited hops. On this basis, based on the edge server and cloud computing technology, the enhancement of the performance of the mobile terminal is realized. The three-level business structure is shown in Figure 1.Figure 1
5G edge computing service architecture.


## 2.1.2. 5G Edge Computing Intelligent Resource Allocation Framework

Edge computing has more local services, servers closer to the data source, lower latency, more convenient positioning, and more convenient access to data. The edge server can serve clients, so it can communicate with the terminal locally, separate from other networks, so it can localize the service [12]. Since the mobile terminal service is often carried out in the access network, the interval between the mobile terminal and the user is usually one jump. Therefore, in the process of transmitting a large amount of data, it is applied to the data preprocessing, so as to realize the large amount of data analyzed. In addition, since the distance between the mobile server and the user is relatively close, more services can be implemented. Location data are available in the access network, while mobile edge computing is a combination of access network and cloud computing with the continuous development of communications and the Internet and the continuous improvement of hardware facilities and customer requirements [13]. After solving a large amount of data access, network-based edge servers must seek a more intelligent way of resource allocation to meet the different requirements of different types of users, so as to meet the ever-increasing QoS requirements. After years of development, deep reinforcement learning has application significance in many aspects. Applying it to the resource allocation system of edge servers can enable servers to complete data processing in a shorter time.


## 2.2. 5G Service Resource Allocation Method

Aiming at the high-frequency band requirements in 5G boundary computing, this paper designs a 5G boundary processing scheme based on DQN technology. Firstly, the convex optimal method is used to solve the association between multiple nodes to obtain the association between each node and obtain the minimum node energy. In this case, the DQN algorithm is used to transmit to users, which ensures not only the requirements of each user but also the energy-saving effect of the system. The diagram shows the flow of this algorithm (see Figure2).Figure 2
DQN-based 5G service resource allocation process.Reinforcement learning method is a learning method for problems in learning, which includes model reinforcement learning and model-free reinforcement learning. At present, in real life, pattern-based reinforcement learning methods are rare, and there are existing time series reinforcement learning methods and Monte Carlo methods. Among them, the Monte Carlo method reduces the convergence rate due to its large amount of variation, and the Q-Learning method can achieve better results [14].Q-Learning does not depend on the mode of the environment; it performs an action under a special condition, its value is a function of the value, an agent looks for the best action in the Q table, and in the process, it can obtain the maximum cumulative reward. In conventional Q-Learning, Q functions are defined to reflect long-term interests between behavior and a state. In summary, there is a Q table to be saved in Q-Learning; the Q of all state-movement cumulative returns is saved in the Q table, where the current state is saved in a row, and the current state associated with the next action is stored in the Q table. The data is saved in this column. In some cases, due to the large state and operation space, the amount of data in the Q table will become large, which brings great difficulties to the actual storage. Therefore, a neural network with self-learning function must be used to approximate the Q value.


## 2.3. Analysis of Simulation Results

### 2.3.1. Simulation Settings

A simulation model of a 5G high-band service resource allocation scheme consisting of several base stations and several users is presented. This part is mainly aimed at high-bandwidth users, such as AR/VR users, long video transmission users, and music listening users (see Figure3).Figure 3
Simulation scene simulation diagram.In the simulation scheme, 5 base stations and several users are set, and it is assumed that the maximum transmission capacity of each base station is 3 W, and other simulation parameters are set as shown in Table1 [15].Table 1
Simulation parameter table.
Parameter nameParameter valueNumber of base stations5Base station maximum transmit power3 WPath loss103.8Noise power spectral density−174Fixed power consumption4.8 WBackhaul transmission power0.2 watt/mbpsCalculate power consumption10-9Calculate frequency750Operation time1 sBandwidth max20 MHzBandwidth allocation ratio0.2Playback memory pool capacity500Minimum sample size32


### 2.3.2. Comparison of Results of Different Numbers of Users

When using DQN technology to realize 5G high-frequency band, due to too many users, the state space in DQN is too large, which will have a certain negative effect on the learning efficiency of the system [16]. To this end, we simulated four situations, namely, 30, 25, 20, and 15 users, and tested the performance of DQN under different numbers of users. Figure 4 is a comparison of excitation in four examples; a comparison of losses is shown in Figure 5 in four examples, and a comparison of energy efficiency is shown in Figure 6 in four examples.Figure 4
Comparison of reward values.Figure 5
Comparison of loss values.Figure 6
Comparison of energy efficiency values.In the reward comparison shown in Figure4, as the number of users increases, the steps to reach convergence also increase. This is due to the increase in the number of users of DQN, which increases the state and activity range of DQN and reduces the learning time, thus increasing the steps of convergence. In Figure 5, the DQN algorithm is trained for different numbers of users, and its convergence rate is consistent with the steps described above. It can be seen from Figure 6 that in the learning process of DQN, as the learning of DQN gradually approaches, the energy efficiency gradually becomes stable, lower, and more in line with the best design purpose of this model. The research results show that the learning efficiency and learning speed of DQN in the whole system will decrease due to the training step size, which shows that the learning efficiency of DQN in the whole system greatly improved, which shows that the DQN strategy selected in this paper works. Moreover, the more the number of users, the greater the energy consumption and the higher the energy utilization rate.


## 2.3.1. Simulation Settings

A simulation model of a 5G high-band service resource allocation scheme consisting of several base stations and several users is presented. This part is mainly aimed at high-bandwidth users, such as AR/VR users, long video transmission users, and music listening users (see Figure3).Figure 3
Simulation scene simulation diagram.In the simulation scheme, 5 base stations and several users are set, and it is assumed that the maximum transmission capacity of each base station is 3 W, and other simulation parameters are set as shown in Table1 [15].Table 1
Simulation parameter table.
Parameter nameParameter valueNumber of base stations5Base station maximum transmit power3 WPath loss103.8Noise power spectral density−174Fixed power consumption4.8 WBackhaul transmission power0.2 watt/mbpsCalculate power consumption10-9Calculate frequency750Operation time1 sBandwidth max20 MHzBandwidth allocation ratio0.2Playback memory pool capacity500Minimum sample size32


## 2.3.2. Comparison of Results of Different Numbers of Users

When using DQN technology to realize 5G high-frequency band, due to too many users, the state space in DQN is too large, which will have a certain negative effect on the learning efficiency of the system [16]. To this end, we simulated four situations, namely, 30, 25, 20, and 15 users, and tested the performance of DQN under different numbers of users. Figure 4 is a comparison of excitation in four examples; a comparison of losses is shown in Figure 5 in four examples, and a comparison of energy efficiency is shown in Figure 6 in four examples.Figure 4
Comparison of reward values.Figure 5
Comparison of loss values.Figure 6
Comparison of energy efficiency values.In the reward comparison shown in Figure4, as the number of users increases, the steps to reach convergence also increase. This is due to the increase in the number of users of DQN, which increases the state and activity range of DQN and reduces the learning time, thus increasing the steps of convergence. In Figure 5, the DQN algorithm is trained for different numbers of users, and its convergence rate is consistent with the steps described above. It can be seen from Figure 6 that in the learning process of DQN, as the learning of DQN gradually approaches, the energy efficiency gradually becomes stable, lower, and more in line with the best design purpose of this model. The research results show that the learning efficiency and learning speed of DQN in the whole system will decrease due to the training step size, which shows that the learning efficiency of DQN in the whole system greatly improved, which shows that the DQN strategy selected in this paper works. Moreover, the more the number of users, the greater the energy consumption and the higher the energy utilization rate.


## 3. A3C’s Energy-Efficient Resource Allocation Method for 5G Wide-Connection Low-Latency Services

### 3.1. 5G Heterogeneous Network Architecture for IIoT Services

The IOT-based 5G wireless communication system includes two-level and multilevel heterogeneous networks of two types of BS and two types of E types. Some MBSs have larger signal coverage and some overlap with MBS coverage [8]. Use wireless loop connection for MBS and FBS. IOT devices include MBS and FBS, which are MIE and FIE, which undertake a lot of control and semistatic monitoring, respectively.On the IOT, a lot of traffic is generated. In order to make it more intelligent, it needs to be processed more deeply. Utilizing the ubiquitous edge computing network, data from hotspots can be buffered near the IOT device, while also reducing the distance between the IOT device and its interior. Therefore, “end-to-end delay”, which is regarded as a key indicator of industrial regulation, can be guaranteed. In order to obtain a fast and stable resource allocation, an ad hoc network with MAPOE capability is required, which can provide intelligent support for services with improved latency sensitivity and computing capability. The self-organizing resource allocation function configured on the edge cloud node can be realized by a centralized control system of the network optimization engine with AI. The integration of MAPOE in the central control system can greatly improve the resource allocation of the network.


### 3.2. Intelligent and Energy-Efficient Resource Allocation Framework

Reconfiguration and command control of resources are achieved through an executor, and information is passed to the destination resource planner. This plan will be continuously monitored, and the whole process will be carried out within the framework. MAPOE functions can utilize a hybrid management architecture as SON Agent and SON Controller. In addition, most resource allocations are centered on the use of resources, which often ignores energy efficiency. The large number of base stations in 5G IOT will cause huge energy loss. Based on the above theory, we believe that DRL should be performed on resource allocation here.


### 3.3. Analysis of Simulation Results

#### 3.3.1. Parameter Setting

In the simulation process, we also incorporate the more conventional industrial reality. Especially in industrial production, MBS can provide a wide range of signals in production, and FBS can achieve signal coverage within a certain range. MIE is a device that requires high speed and requires little data transmission such as a control node. The FIE is a relatively stable device that requires a higher data transmission rate such as a node with monitoring functions. Consider a network consisting of 1 MBS and 10 FBS; these base stations support 5 mobile communication devices and 80 mobile communication devices to simulate a high-intensity environment and set all devices within the message range of FBS and FIE. The settings for these parameters are shown in Table2.Table 2
Simulation parameter table.
Parameter nameParameter valueMIE data rate1 MbitFIE data rate2 MbitMBS maximum transmit power40 WFBS maximum transmit power5 WMBS static power25 WFBS static power10 WAdditive white Gaussian noise−174Maximum transmission delay200 msMBS transmit power to MI positive33 dBm


#### 3.3.2. Effect Comparison of Different Resource Allocation Methods

A3C is compared with two other RL and one conventional RL algorithms. The two main types of RL are DQN and Q-learning. Among them, the genetic algorithm is selected as the conventional genetic algorithm. The R3 excitation function is set as the excitation function used in the RL method. As can be seen from Figure7, the convergence of all three RL algorithms is better than that of the genetic algorithm, but their rewards are higher. Among them, the A3C method is the optimal resource allocation method. Although the convergence steps of the DQN and R3 algorithms are similar to those of the Q-Learning and R3 algorithms, the latter yields higher returns. Although the genetic algorithm has the highest convergence rate, the reward of GA is the least. The results show that the A3C algorithm is feasible.Figure 7
Convergence diagram with different resource allocation methods.Then, we further examine the efficiency of resource allocation using different devices at different time nodes. Here, it is usually from 6:00 to 22:00. The number of MIEs is 3-5, and the number of FIEs is 50-100. Figure8 shows the situation of the EE in various situations. As the number of A3C devices is increasing, it is more and more different from other technologies, which shows that A3C is more accurate on a large number of devices.Figure 8
Energy efficiency values at different time periods.Simply put, RL algorithms can achieve more accurate calculations with more states and operation spaces. Among the three RL algorithms, the A3C algorithm has the best performance, the highest reward and EE, and the lowest convergence speed. A3C’s approach can fuse “roles” and “critics” with unsynchronized threads. The A3C algorithm is more efficient in high dimensional or continuous motion. Among the three RL methods, DQN is the worst, and Q-Learning algorithm is the worst. Compared with Q-Learning, the DQN algorithm adopts a new strategy of experience replay and deviation, which improves the operating efficiency of the system.


#### 3.3.3. Effect Comparison of Different Clustering Algorithms

Two different clustering methods were used to assign FIE to 10 neighboring FBS. Using two different clustering algorithms, K-means and HAC, resource allocation was carried out in the A3C mode stimulated by R3, respectively. K-means is a method that uses distance as an evaluation index. The effect of two different clustering methods is shown in Figure9. Compared with K-means, HAC method has better convergence and higher rate of return. By comparing two different types of clustering methods, it is found that the HAC method has the advantages of energy saving, less energy consumption, and bandwidth distribution. The HAC method has better computational efficiency than the K-means method, which shows that the method selected in this paper has certain practical value.Figure 9
Convergence graphs of different clustering algorithms.


## 3.1. 5G Heterogeneous Network Architecture for IIoT Services

The IOT-based 5G wireless communication system includes two-level and multilevel heterogeneous networks of two types of BS and two types of E types. Some MBSs have larger signal coverage and some overlap with MBS coverage [8]. Use wireless loop connection for MBS and FBS. IOT devices include MBS and FBS, which are MIE and FIE, which undertake a lot of control and semistatic monitoring, respectively.On the IOT, a lot of traffic is generated. In order to make it more intelligent, it needs to be processed more deeply. Utilizing the ubiquitous edge computing network, data from hotspots can be buffered near the IOT device, while also reducing the distance between the IOT device and its interior. Therefore, “end-to-end delay”, which is regarded as a key indicator of industrial regulation, can be guaranteed. In order to obtain a fast and stable resource allocation, an ad hoc network with MAPOE capability is required, which can provide intelligent support for services with improved latency sensitivity and computing capability. The self-organizing resource allocation function configured on the edge cloud node can be realized by a centralized control system of the network optimization engine with AI. The integration of MAPOE in the central control system can greatly improve the resource allocation of the network.


## 3.2. Intelligent and Energy-Efficient Resource Allocation Framework

Reconfiguration and command control of resources are achieved through an executor, and information is passed to the destination resource planner. This plan will be continuously monitored, and the whole process will be carried out within the framework. MAPOE functions can utilize a hybrid management architecture as SON Agent and SON Controller. In addition, most resource allocations are centered on the use of resources, which often ignores energy efficiency. The large number of base stations in 5G IOT will cause huge energy loss. Based on the above theory, we believe that DRL should be performed on resource allocation here.


## 3.3. Analysis of Simulation Results

### 3.3.1. Parameter Setting

In the simulation process, we also incorporate the more conventional industrial reality. Especially in industrial production, MBS can provide a wide range of signals in production, and FBS can achieve signal coverage within a certain range. MIE is a device that requires high speed and requires little data transmission such as a control node. The FIE is a relatively stable device that requires a higher data transmission rate such as a node with monitoring functions. Consider a network consisting of 1 MBS and 10 FBS; these base stations support 5 mobile communication devices and 80 mobile communication devices to simulate a high-intensity environment and set all devices within the message range of FBS and FIE. The settings for these parameters are shown in Table2.Table 2
Simulation parameter table.
Parameter nameParameter valueMIE data rate1 MbitFIE data rate2 MbitMBS maximum transmit power40 WFBS maximum transmit power5 WMBS static power25 WFBS static power10 WAdditive white Gaussian noise−174Maximum transmission delay200 msMBS transmit power to MI positive33 dBm


### 3.3.2. Effect Comparison of Different Resource Allocation Methods

A3C is compared with two other RL and one conventional RL algorithms. The two main types of RL are DQN and Q-learning. Among them, the genetic algorithm is selected as the conventional genetic algorithm. The R3 excitation function is set as the excitation function used in the RL method. As can be seen from Figure7, the convergence of all three RL algorithms is better than that of the genetic algorithm, but their rewards are higher. Among them, the A3C method is the optimal resource allocation method. Although the convergence steps of the DQN and R3 algorithms are similar to those of the Q-Learning and R3 algorithms, the latter yields higher returns. Although the genetic algorithm has the highest convergence rate, the reward of GA is the least. The results show that the A3C algorithm is feasible.Figure 7
Convergence diagram with different resource allocation methods.Then, we further examine the efficiency of resource allocation using different devices at different time nodes. Here, it is usually from 6:00 to 22:00. The number of MIEs is 3-5, and the number of FIEs is 50-100. Figure8 shows the situation of the EE in various situations. As the number of A3C devices is increasing, it is more and more different from other technologies, which shows that A3C is more accurate on a large number of devices.Figure 8
Energy efficiency values at different time periods.Simply put, RL algorithms can achieve more accurate calculations with more states and operation spaces. Among the three RL algorithms, the A3C algorithm has the best performance, the highest reward and EE, and the lowest convergence speed. A3C’s approach can fuse “roles” and “critics” with unsynchronized threads. The A3C algorithm is more efficient in high dimensional or continuous motion. Among the three RL methods, DQN is the worst, and Q-Learning algorithm is the worst. Compared with Q-Learning, the DQN algorithm adopts a new strategy of experience replay and deviation, which improves the operating efficiency of the system.


### 3.3.3. Effect Comparison of Different Clustering Algorithms

Two different clustering methods were used to assign FIE to 10 neighboring FBS. Using two different clustering algorithms, K-means and HAC, resource allocation was carried out in the A3C mode stimulated by R3, respectively. K-means is a method that uses distance as an evaluation index. The effect of two different clustering methods is shown in Figure9. Compared with K-means, HAC method has better convergence and higher rate of return. By comparing two different types of clustering methods, it is found that the HAC method has the advantages of energy saving, less energy consumption, and bandwidth distribution. The HAC method has better computational efficiency than the K-means method, which shows that the method selected in this paper has certain practical value.Figure 9
Convergence graphs of different clustering algorithms.


## 3.3.1. Parameter Setting

In the simulation process, we also incorporate the more conventional industrial reality. Especially in industrial production, MBS can provide a wide range of signals in production, and FBS can achieve signal coverage within a certain range. MIE is a device that requires high speed and requires little data transmission such as a control node. The FIE is a relatively stable device that requires a higher data transmission rate such as a node with monitoring functions. Consider a network consisting of 1 MBS and 10 FBS; these base stations support 5 mobile communication devices and 80 mobile communication devices to simulate a high-intensity environment and set all devices within the message range of FBS and FIE. The settings for these parameters are shown in Table2.Table 2
Simulation parameter table.
Parameter nameParameter valueMIE data rate1 MbitFIE data rate2 MbitMBS maximum transmit power40 WFBS maximum transmit power5 WMBS static power25 WFBS static power10 WAdditive white Gaussian noise−174Maximum transmission delay200 msMBS transmit power to MI positive33 dBm


## 3.3.2. Effect Comparison of Different Resource Allocation Methods

A3C is compared with two other RL and one conventional RL algorithms. The two main types of RL are DQN and Q-learning. Among them, the genetic algorithm is selected as the conventional genetic algorithm. The R3 excitation function is set as the excitation function used in the RL method. As can be seen from Figure7, the convergence of all three RL algorithms is better than that of the genetic algorithm, but their rewards are higher. Among them, the A3C method is the optimal resource allocation method. Although the convergence steps of the DQN and R3 algorithms are similar to those of the Q-Learning and R3 algorithms, the latter yields higher returns. Although the genetic algorithm has the highest convergence rate, the reward of GA is the least. The results show that the A3C algorithm is feasible.Figure 7
Convergence diagram with different resource allocation methods.Then, we further examine the efficiency of resource allocation using different devices at different time nodes. Here, it is usually from 6:00 to 22:00. The number of MIEs is 3-5, and the number of FIEs is 50-100. Figure8 shows the situation of the EE in various situations. As the number of A3C devices is increasing, it is more and more different from other technologies, which shows that A3C is more accurate on a large number of devices.Figure 8
Energy efficiency values at different time periods.Simply put, RL algorithms can achieve more accurate calculations with more states and operation spaces. Among the three RL algorithms, the A3C algorithm has the best performance, the highest reward and EE, and the lowest convergence speed. A3C’s approach can fuse “roles” and “critics” with unsynchronized threads. The A3C algorithm is more efficient in high dimensional or continuous motion. Among the three RL methods, DQN is the worst, and Q-Learning algorithm is the worst. Compared with Q-Learning, the DQN algorithm adopts a new strategy of experience replay and deviation, which improves the operating efficiency of the system.


## 3.3.3. Effect Comparison of Different Clustering Algorithms

Two different clustering methods were used to assign FIE to 10 neighboring FBS. Using two different clustering algorithms, K-means and HAC, resource allocation was carried out in the A3C mode stimulated by R3, respectively. K-means is a method that uses distance as an evaluation index. The effect of two different clustering methods is shown in Figure9. Compared with K-means, HAC method has better convergence and higher rate of return. By comparing two different types of clustering methods, it is found that the HAC method has the advantages of energy saving, less energy consumption, and bandwidth distribution. The HAC method has better computational efficiency than the K-means method, which shows that the method selected in this paper has certain practical value.Figure 9
Convergence graphs of different clustering algorithms.


## 4. Conclusion

Compared with 4G, 5G will increase the transmission rate, reduce the delay, and improve the quality of service for users. The wide application of 5G technology provides new ideas and development space for future business development, such as autonomous driving, massive machine communication, edge computing, and industrial Internet of Things. Due to the development of 5G technology, these schemes will have better development opportunities. In today’s rapid development of 5G, due to the large number of end users and various requirements, it is particularly important to realize the effective allocation of resources on the premise of meeting the needs of customers. In addition, since the energy dissipation problem has been the most concerned problem in the current wireless communication system, energy saving is already a very critical technology in the future mobile network. Therefore, in terms of ensuring network bandwidth and energy, it is necessary to ensure not only the stability of the network but also the operation of the network. DRL technology puts forward higher requirements for the solution of complex problems, and has made obvious progress in many fields, while the resource allocation of 5G network needs further research. On this basis, this paper adopts a new efficient resource allocation strategy by combining two typical applications in 5G networks, namely, edge computing and industrial IoT.



---

*Source: 4418231-2022-09-06.xml*