###### Abstract  
In the face of an increasingly intricate network structure and a multitude of security threats, cyber deception defenders often employ deception assets to safeguard critical real assets. However, when it comes to the intranet lateral movement attackers in the cyber kill chain, the deployment of deception assets confronts the challenges of lack of dynamics, inability to make real-time decisions, and not considering the dynamic change of an attacker's strategy. To address these issues, this study introduces a novel maze pathfinding model tailored to the lateral movement context, in which we try to find out the attacker's location to deploy deception assets accurately for interception. The attack-defense process is modeled as a multi-agent stochastic game, by comparing it with random action policy and Minimax-Q algorithm, we choose Nash Q-learning to solve the deception asset's deployment strategy to achieve the optimal solution effect. Extensive simulation tests reveal that our proposed model exhibits good convergence properties. Moreover, the average defense success rate surpasses 70%, attesting to the model's efficacy. cyber deception defense; stochastic game; multi-agent reinforcement learning; Nash Q-learning +
Footnote †: journal:  
The purpose of cyber deception defense is to inject false information in the reconnaissance stage of an attack to mislead or confuse the attacker [9; 10; 11; 12] so that the attacker can take actions that are beneficial to the defender. For example, the honeypot system [13; 14] is to attract potential attackers, waste attack resources, and learn attack tools, attack strategies, and other information about attackers. Compared with MTD, cyber deception defense not only has strong concealment, but the deception assets are difficult to be found by attackers and can detect and locate attacks in time; they also can deploy deception assets at a lower cost and learn the attacker's information. As depicted in Figure 1, after an initial vulnerability scan, attackers devise a targeted strategy, breach an entry node, and initiate lateral movements within an intranet to locate high-value targets. This lateral movement involves maneuvers to access valuable nodes by leveraging internal phishing, compromising shared content, or exploiting backup authentication credentials. Defenders, in contrast, focus on deploying honeypots and other deceptive assets to inhibit attackers from pinpointing high-value targets. The effective deployment of these deception assets, particularly in dynamically changing environments, is a crucial research area in deception defense. However, deception asset deployment faces the challenges of lack of dynamics, inability to make real-time decisions, and not considering the dynamic change of an attacker's strategy. Aiming at the above problems, this paper uses game theory to study the cyber deception asset deployment problem and uses a multi-agent reinforcement learning method to find out the attacker's location for deception asset deployment to solve the optimal deployment strategy. Our contributions include:  
(1) Modeling the Intranet Scenario: Addressing the challenges of current deception asset deployment, we cast the attacker's lateral movement and deception asset deployment as a non-cooperative multi-agent stochastic game. We adopt the Nash Q-learning algorithm to solve the equilibrium strategy and ascertain the best deception asset deployment approach. (2) Introducing the Maze Model: For the attacker's lateral movement in the intranet and the process of the defender's deployment of deception assets, we present a maze pathfinding model to encapsulate the attack-defense process. The attacker aims to locate valuable maze nodes, whereas the defender seeks to pinpoint the attacker's location and deploy deception assets accordingly. (3) Validation via Simulation: To evaluate our model, we set up a Kubernetes cluster in a cloud environment for simulations. Our results validate the model's convergence and defensive prowess.