# A Modified Quad Q Network Algorithm for Predicting Resource Management
## 2 Literature Review: Related Work  
### The Latest Trends in Fuzzy Logic and Reinforcement Learning  
In recent years, reinforcement learning has shown the ability to achieve good results in games such as fuzzy fields and robotic manipulators. It has become increasingly popular in the field of motion planning due to its characteristics of not requiring much labeling data. Fuzzy systems have found applications such as control engineering [11], medicine [12], and autonomous agents [13]. In recent years, some researchers have tried to combine reinforcement learning and fuzzy control to address several complex issues in the field of robot control [14]. Fuzzy systems typically have several control parameters that are outputted from the system. These parameters are then transformed into exact numerical values by diffusing to deal with dependencies between outputting different commands, thus providing a solution to improve the accuracy of predictions in Q-learning [15]. Deep reinforcement learning, which employs neural networks to infer state values, has received a great deal of interest. It is capable of dealing with large complex unknown conditions [16; 17; 18; 19], including very large state and activity spaces. Deep reinforcement learning has achieved numerous breakthroughs in complex spectrum access in recent years owing to its inherent advantages. The works in [20] may be regarded as the pioneer work in the investigation of Deepreinforcement learning. It proposes a dynamic channel access scheme based on the Deep Q Network with experience replay [21]. The user selects one of the M channels to transmit its packet in each slot. Since the user only learns the channel state after selecting it, the related optimization decision problem can be expressed as a partially observable Markov decision method. Deep Q Network can promptly look for the best approach based on experience [22; 23; 24], thus it is commonly used in a variety of enterprises. Deep Q Network [25] is being investigated to solve both complex channel access and interference management. The Deep Q Network in [21] continues to pursue the learned policy over time slots and avoids learning a suitable policy. Actual Reinforcement Environments Learning in the channel research is complex, and requires retraining of the Deep Q Network. To fix this problem, an adaptive Deep Q Network scheme [26] is proposed, which evaluates the cumulative incentives of the current policies over time. When the incentive falls below a certain level, the Deep Q Network is retrained to find new good policies. The simulation results show that when the channel state change, the scheme can detect it and begin relearning to achieve high returns. A Deep Reinforcement Learning system based on actor-critics is suggested, which can improve performance in a large number of channels [27]. Most of the methods listed above are limited to instances of only one user rather than multi-user or multiple channel setting, where collisions are common. Taking this into consideration, a fully distributed model [28] is used for users to perform dynamic multi-user spectrum access. For Dynamic Spectrum Access, a deep multi-user reinforcement learning method [29] is developed. This method can effectively represent the overall complex environment while avoiding the high computational costs incurred by the broad state space and inefficient observability. The spectrum access scheme proposed in [30] is based on Q learning and can assign idle spectrum holes without interaction with users. proposes multi-agent reinforcement learning based on Q learning, which can effectively avoid interference and achieve good sensing performance. Each radio in this model considers the actions of other radios to be a part of the environment [31]. Each radio tries to avoid transmissions from other wideband autonomous cognitive radios while also avoiding a jammer signal that sweeps through the entire spectrum band of interest. However, it is more difficult to achieve a stable policy in a volatile and complicated climate.