By modeling the jamming combating problem as a Colonel Blotto game, [PERSON] et al. [23] devised a power allocation strategy to protect IoT devices from jamming attacks while minimizing the power consumption of IoT sensors. To apply game theory in smart grids,Ferrag et al. [24] summarized various smart grid privacy protection schemes that are solved using game theory methods, such as smart grids with advanced metering infrastructure, data aggregation, smart grids marketing architecture, smart communities of homes gateways, and vehicle-to-grid architecture. Lv and Ai [25], an energy game matrix was proposed to achieve the dynamic management of energy system collaborative interaction. In addition, [PERSON] et al. [26] presented monitoring strategies that utility companies employ to detect security attacks, using a game-theoretic approach to allocate resources required for a defense based on the varying impacts of attackers. As an important data collection step in smart cities, mobile crowdsensing also relies on game theory models to solve practical problems, including perceiving cost, data quality, optimal price determination between requesters and providers, and incentive mechanisms as in the study by [PERSON] et al. [27]. [PERSON] et al. [28] constructed a non-cooperative differential game model to solve the trade-off between data utility and privacy protection in mobile crowdsensing, ensuring that the privacy of users can be protected and the platform can maximize data utility. [PERSON] et al. (2019) applied a two-stage Stackelberg game to analyze the interaction between crowd-aware service providers and mobile users, and reversely generalized the optimal incentive mechanism for crowd-aware service providers to efficiently recruit a sufficient number of users for more benefits. [PERSON] et al. (2019) proposed a mobile crowdfunding-based beacon control strategy that applies game theory to balance the benefit for each user between the transmission rate of sensory data and energy consumption so that the delivery ratio can be maximized with limited initial energy. As already mentioned, the computational method we use here is close to the methods used in the literature on \(Q\)-learning, where the value functions are repeatedly updated until they converge. An advantage of this method is that the computational method is relatively simple. One just needs to find the optimal value function at each step of the iteration for a player. Therefore, at no stage of the computational process does it require one to compute a fixed point. In fact, the computational program remains more or less unchanged at each step with only the future expected value functions being updated. A second advantage is that it uses a technique that has proved very useful in computing the solutions of Markov processes and can be scaled up relatively easily both in terms of the number of states as well as the number of players. The computational method is also quite widely applicable. In fact, we do not have any restrictions for the games we analyze except that the number of states be finite and the action sets of the players be finite. The rest of this paper is laid out as follows. In Section 2, we describe the model; in Section 3, we describe the payoffs and strategies as well as the central existence result. In Section 4, we discuss stationary equilibrium when the game has a K-period cycle. In Section 5, we provide a detailed description of the algorithm and its convergence properties. In Section 6, we compute the stationary Markov perfect equilibrium in three experiments to evaluate the proposed algorithm. Finally, we conclude the paper with Section 7.