# A Review of Research on Spectrum Sensing Based on Deep Learning
## 4 Deep-Learning-Based Cooperative Spectrum Sensing
### Applications of Deep Reinforcement Learning to Cooperative Spectrum Sensing  
Deep reinforcement learning (DRL) is an artificial intelligence method that combines the advantages of deep learning and reinforcement learning [103]. Its decision making and learning in a dynamic environment fit the needs of CSS. As a result, scholars are now beginning to introduce DRL into CSS to address distributed decision making, synchronization problems, and security and robustness issues in CSS. In Ref. [104], [PERSON] et al. considered cooperative spectrum sensing in multi-user scenarios and dynamic environments. Deep multi-user reinforcement learning (DMRL) with deep Q-network [105] as the underlying framework is proposed, which is capable of determining the optimal action for each state in a relatively large, dynamic, unknown environment. After training the DDQN by training all users individually, each SU performs conflict state determination via acknowledge (ACK) signaling. In simulation experiments, the proposed method demonstrated enhanced spectral efficiency. Compared to other dynamic spectrum access (DSA) techniques, this method exhibits a faster convergence rate and superior reward performance. Ref. [106], on the other hand, studies the dynamic spectrum sensing and aggregation problem in a multi-channel application scenario. The dynamic spectrum environment is modeled as a joint Markov chain by feeding the observed channel state values into a deep Q-network to approximate the action-value function. They addressed the dynamic spectrum sensing and aggregation problem by formulating it as a POMDP and proposing a DQN framework. Their simulation results show that DQN achieves near-optimal decision accuracy in most scenarios, even without a priori knowledge of the system dynamics. Moreover, DQN has the lowest computational complexity and has no effect on problem scaling. Unlike the two studies mentioned above, [PERSON] et al. proposed a conservative Q-learning implementation of local cooperative spectrum sensing [107]. This approach is able to learn complex data distributions more efficiently in offline DRL. The fusion center receives local sensing information from the SU to generate global decisions. They validated the proposed method through simulation experiments, demonstrating its comparable accuracy to other CSS methods and its performance improvement while reducing computation time. Scholars such as [PERSON] and others, on the other hand, focus on the problem of selecting appropriate neighbor nodes in CSS under correlated fading. They introduced the coordinated graph method (CG) in [108] to decompose the global reward into the sum of local terms, thus transforming the CSS problem into a max-plus problem that can be solved by message passing and finally combined with the QN algorithm to accelerate the convergence speed. Their experimental results show that both the flat false alarm probability and the missed detection probability of the proposed model decrease as the number of SUs increases for the same number of maximum neighbor nodes. In a real-world cooperative spectrum-sensing scenario, the data fusion center may suffer from security attacks, such as the presence of malicious nodes that can compromise the performance of the cooperative spectrum-sensing system or the spectrum allocation. The following types of security issues are commonly found in CSS: false data injection attacks, identity spoofing attacks, and data tampering attacks. In view of these security issues in CSS, scholars have started to study related countermeasures. Addressing the potential danger of malicious SUs sending forged data to data fusion centers to jam CSS systems, [PERSON] et al. [109] tried to use agent intelligence in DRL to be able to effectively avoid the fake data sent to the FC. The proposed model is also based on a deep Q-network, which is made robust to fake data attacks by the experience replay (ER) algorithm.