## 2 Related Work  
The effective application of cyber deception defense to combat attacks and enhance system security performance is a pivotal research challenge within the realm of cyber deception defense. This section provides an overview of the seminal works in this domain. Deception asset deployment has historically grappled with dynamic inadequacies. To address this, game theory [15] has been integrated into the research as a theoretical framework for decision-making. This assists defenders in the dynamic deployment of deception assets, a subject that has been the focus of numerous studies. [PERSON] et al. [16] proposed a two-player Bayesian game based on attack-defense deception in IoT networks, and they used honeypot as a deceptive defense mechanism. In their article, they studied Bayesian equilibria in one-shot and repeated games, respectively, and the results show that there is a certain frequency of attacks beyond which the two parties will mainly adopt cheating as their strategy. The authors used honeypots as a deception strategy in imperfect information games. [PERSON] et al. [17] proposed a defense mechanism based on deception to mitigate Denial of Service (DoS) attacks. The defender deploys honeypots to attract the attacker and obtain attack information. They used signaling games with perfect Bayesian equilibria to model the interaction between defender and attacker. [PERSON] et al. [18] developed a honeypot-based defense system based on the signaling game, where the attacker is able to detect the honeypot. Extending from the empty talk game (a signaling game between the sender and the receiver with no signaling cost), they studied the signaling game with deception detection, in which the sender (the defender) transmits a signal and the receiver (the attacker) can detect the deception with a certain probability. The results show that the ability of the attacker to detect deception does not necessarily reduce the defender's utility. [PERSON] et al. [19] used cyber deception tools to posthaste confirm attacker types so that better defense strategies could be adopted. The type of attacker is reflected in the actions and goals when planning the attack activity. The authors used a multi-stage Stackelberg game to model the interaction between the attacker and the defender. In this game, the defender is the leader who plays a strategy considering the strategy of the attacker. However, as a follower, the attacker chooses his own strategy after observing the strategy of the leader. [PERSON] et al. [20] proposed a scalable algorithm to allocate honeypots over an attack graph and formulated a two-person zero-sum strategic game between the network defender and an attacker. They investigated the trade-off between security cost and deception reward for the defender. Finally they analytically characterized the Nash Equilibrium defense strategies. However, the above studies do not consider the real-time decision of the deception asset deployment strategy. Here, we introduce the method of reinforcement learning [21]. The reinforcement learning defender can adjust the deception asset deployment strategy in real-time according to the system state, which solves the problem that the current deception asset deployment cannot make real-time decisions. [PERSON] et al. [22] introduced an online deception design and placement method for network decoys, taking into account the impact of defender behavior on attacker strategy and tactics in order to maintain a balance between usability and security tradeoff. The defender maintains beliefs regarding safe states, while modeling resulting actions as a partially observable Markov Decision Process (POMDP). An online deception algorithm is employed by the defender, utilizing a POMDP model to select actions based on observed attacker behavior.