# Research on Defensive Strategy of Real-Time Price Attack Based on Multiperson Zero-Determinant

Authors: Zhuoqun Xia; Zhenwei Fang; Fengfei Zou; Jin Wang; Arun Kumar Sangaiah
Journal: Security and Communication Networks
 (2019)

---

## Abstract

The smart grid solves the growing load demand of electrical customers through two-way real-time communication of electricity supply and demand sides and home energy management system (HEMS). However, these technical features also bring network security risks to the real-time price signal of the smart grid. The real-time price attack (RTPA) can maliciously raise the real-time price in smart meter, resulting in an increase in electrical customers load demand, causing the extensive damage to the power transmission lines due to overload. In this paper, we based on the behavioral relationship between load demand of electrical customers and real-time price of electricity suppliers (ES), defined the game relationship between RTPA, ES, and electrical customers, established a price elasticity of electricity demand (PEED) model, and proposed a defensive strategy of real-time price attack based on multiperson zero-determinant strategy (MPZDS). The experimental results show that the combination of MPZDS to some extent cut the expected load demand of electrical customers and protect the safety of power transmission lines.


---

## Body

## 1. Introduction

Smart grid (SG) is a cyberphysical system (CPS) that can be described as the next generation power grid, in which not only generation, transmission, and distribution of power but also utilization and management aspects of the grid are upgraded to improve the grid's reliability, efficiency, flexibility, scalability, safety, security, and environmental friendliness. Different from the traditional power grid, the SG relies on a two-way communication infrastructure and its overall performance is optimized using a number of existing and emerging technologies, including wireless sensor networks (WSNs) and other communication technologies, smart sensor devices, automation systems, monitoring, computers, and renewable energy solutions [1, 2]. However, these features have also brought many security risks to the smart grid; for example, on December 23, 2015, the Ukrainian power grid was attacked by malicious software called “BlackEnergy”, resulting in a massive blackout, this blackout was caused by the switching of seven substations, nearly two million people were influenced, and the time of power cut is 3-6h.With the rapid development of new technologies such as intelligent manufacturing [3, 4], in order to improve the reliability and effectiveness of smart grid, some scholars have proposed some concepts such as smart meters [5] and smart home management system [6], but these infrastructures also bring new security vulnerabilities to the smart grid. Power generation, distribution, and electricity consumption units are vulnerable to security attacks due to their strong openness, but attacker attacks on power generation and distribution units require a higher cost than electricity consumption units, therefore, to ensure that electricity consumption units from various types of cyberattacks are crucial.Information injection attacks on electricity consumption units are aimed at tampering with electricity price signals or information [7], and attackers can launch attacks more easily through automated and distributed software injection agents. Furthermore, because smart grids possess some features like load control and automatic energy consumption dispatch, this makes attacks more efficient. The automatic energy consumption dispatching unit schedules the energy consumption of indoor energy consumption equipment through the electricity price given by the electricity supplier and the power consumption information of the electrical customers, so as to minimize the energy consumption. The electricity price information is transmitted through the information network, which makes the false price injection attack can potentially lead to changes in the load demand of the electricity consumption unit, resulting in unbalanced load demand control. In recent years, the academia conducted a deep study on the cyberphysical security of smart grid [8–10]; however, there are few existing researches on the analysis and prevention of real-time price attack (RTPA). Real-time price information release requires low delay, and delay attack will bring too high real-time price information delay, which will affect the electrical customers to obtain real-time price information, thereby affecting the demand response of electrical customers and real-time scheduling of electricity supplier [11].Aiming at the scaling attack and delay attack of real-time price signal integrity, Tan et al. established a closed-loop based on the interdependence between real-time price signal and incentive demand of electricity price and deduced the basic conditions of real-time price system stability in the case of attack by using control theory; the experimental results help the system operators to effectively analyze the impact of attack on the stability parameters of the real-time price system [12, 13]. Based on scaling attack and time-delay attack, Giraldo et al. proposed an analysis method based on sensitivity function on the basis of considering power market model and real-time price integrity attack model, by adding low-pass filters to the price signal, selecting the price update cycle and controller parameters, designing robust control algorithm, and measures to detect abnormal behavior of the system to reduce the impact of the attack [14, 15]. However, it does not consider the potential impact of the attack on power demand side. Jia et al. have studied the false data attack against the node marginal price and proposed a geometrical analysis framework based on the upstream and downstream boundary conditions of the optimal data attack and validated the validity of the framework by PJM 5-node power system, IEEE 14-node power system, and IEEE 118-node power system [16]; however, the framework only guarantees the security of the node’s marginal price data and does not analyze the impact of the attack from the supply or demand side. The price tampering attack will cause the change of load configuration information of individual electrical customers, resulting in the load transfer or load redistribution, which leads to the change of load configuration information of the power network, which eventually leads to transmission lines damage in large areas due to overload, thus forming a cascading failure [17]. However, the above researches on the real-time price market focus on the protection of single-stage security and do not analyze in depth the impact of RTPA on the electrical customers load demand and how to prevent real-time price attack from the viewpoint of ensuring the normal load demand of electrical customers.In information security analysis, the classical prisoner dilemma model clarifies a rational game between the implementation and effectiveness of an attack between an attacker (such as FDIA) and a defender (such as IDS). In order to maximize the impact of an attack, the attacker will choose to strengthen the attack. At the same time, the defender will also choose to strengthen the defense to optimize the defense effect, which means that there is a noncooperative behavior between the attacker and the defender is the dominant strategy between the two sides of the game. However, in the Repeated Prisoner’s Dilemma Game, the choice of behavior strategies of both players is not the same as before. In order to avoid being punished for previous noncooperative behavior, each player may choose cooperation in the subsequent game process, which provides a theoretical basis for the generation of zero-determinant strategy. Zero-determinant strategy (ZDS) shows that the profits of both players satisfy a certain linear relationship, or one player controls the profits of the other player between the incompatible profits and the cooperative profits.In order to solve the above research shortcomings of the RTPA, this paper proposes a RTPA defense strategy based on multiplayer zero-determinants under repeated game to minimize the impact of attack and ensure the user’s expected load demand and transmission line safety.Our contributions are summarized as follows:(i)
We define the load supply and consumption behavior of ES, attackers, and electrical customers according to the relationship between load demand of electrical customers and real-time price of ES.(ii)
We set up a3×2 repeated game according to the behavioral relationship among the ES, the attackers, and the electrical customers, and each of the three players in the game has two kinds of behavioral states, namely, the active state and the idle state. In each stage of the game, all three players in the game may be active or idle and have the same structure of the game payoff matrix.(iii)
We propose Multiplayer Zero-Determinant Strategy (MPZDS) to ensure the safety of electrical customers' side and transmission lines in smart grid to prevent RTPA.The structure of this paper is as follows. Section2 defines the behavioral characteristics of ES, attackers, and electrical customers and establishes the power transmission line model. We analyze the game situation of ES, attackers, and electrical customers according to the price elasticity of electricity demand (PEED) model in Section 3. We next combine the MPZDS for safety analysis in Section 4. The effectiveness of the proposed method is validated through experiments in Section 5. The conclusion is shown in Section 6.


## 2. Related Definitions

This section models the power transmission lines and defines the behavior of the ES, attackers, and electrical customers to describe the dynamic characteristics of all the three.


### 2.1. Power Transmission Lines

From the point of view of power supply and demand system of the smart grid, each node inZ can represent a power generation unit, a power transmission unit, and a power consumption unit (such as a common customers, an industrial consumers, or a data center, and so on), L represents the transmission line between nodes, and the smart grid can be abstracted as GZ,L. In the node set Z, it includes the set of generating nodes PU⊂Z, the set of transmitting nodes OU⊂Z, and the set of electricity utilization nodes DU⊂Z. In the transmission line, LDi represents the power load distribution of the ith transmission line under stable state, LDtotal represents the total load demand of the electrical customers in the stable state, Lmax represents the maximum loadability of each transmission line, and RLP represents the protection rate of power transmission line. When the power load transmission exceeds the maximum loadability of the transmission line, the line i will be damaged due to overload, so the transmission line should meet the following conditions:(1)max⁡RLP∑i∈LLDi≤LDtotal0<LDi≤Lmax


### 2.2. Real-Time Price Attack

In smart grid, there are many ways to attack real-time electricity price signals. For example, attackers can destroy the intermediate nodes of smart grid communication network (such as routers) and obtain the encryption/decryption key in ISO issued/smart meter to intercept and forge data packets containing price data. In addition, attackers launch small-scale tampering attacks against real-time price signals, which may also iteratively amplify the attack effect through feedback, resulting in inefficiency of power system operation and large-scale fluctuation of load demand, and may cause larger and even whole network power failures, such as power outages.RTP is a dynamic price mechanism, the update cycle is usually 1h or 0.5h, or even shorter. At present, the minimum RTP update cycle is 5 minutes on the international. In order to better reflect the game characteristics among ES, attackers and electrical customers, the update cycle of RTP in this paper is 5 minutes. Supposet0 and t are the starting and ending moments of a single-stage game, ΔT is the period length of a single-stage game, and t0=t-ΔT, where DRTPAi is the relative increase of the customers load demand under the RTPA in the ith period, ΔP is the relative increase of the electricity price under the RTPA in the ith period, and then the behavior of RTPA can be defined as(2)DRTPAi=+Γi,ΔP≥0Thereto, + indicates the increases of load demand, and Γi≥0; if the RTPA is in an idle state, the relative increase of load demand is zero at this period; that is, DRTPAi=0.


### 2.3. Stability Control of Home Energy Management System

Some studies have shown that the robustness and convergence of the system model can be improved by optimizing the activation function of machine learning methods such as neural networks, but it cannot meet the requirements of both suppliers and demanders at the same time [18]. But the home energy management system (HEMS) is able to schedule the power consumption scheme of the intelligent electrical appliances optimally according to the information such as the real-time price and reduce the use of the intelligent electrical appliances during the high electricity price; in this way, the supply and demand balance between the electricity supply of ES and the electricity consumption of electrical customers can be balanced. In this paper, to reduce the impact of increased load demand caused by RTPA, HEMS moderately reduces the electrical customers load demand over an optional period of time. Suppose t0 and t are the starting and ending moments of a single-stage game, ΔT is the period length of a single-stage game, and t0=t-ΔT, where DHEMSi is the relative reduction of the customers load demand under the stability control of HEMS in the ith period, ΔP is the relative increase of the electricity price under the RTPA in the ith period, and then the stability control behavior of HEMS can be defined as(3)DHEMSi=-Λi,ΔP≥0Thereto, − indicates the decreases of load demand, and Λi≥0; if the HEMS is in an idle state, the relative decrease of load demand is zero at this period; that is, DHEMSi=0.


### 2.4. Electricity Suppliers Scheduling

By intercepting and forging the electricity price signal from the ES, the attacker maliciously increases the real-time price in the smart meter, causing a large amount of power load in the current period to be transferred to other periods to increase the total load demand on the demand side. From the point of view of ES, in order to meet the increased load demand of electrical customers, the ES can increase the planned power generation. Supposet0 and t are the starting and ending moments of a single-stage game, ΔT is the period length of a single-stage game, and t0=t-ΔT, where DESi is the relative increase of the planned power generation of ES in the ith period, ΔP is the relative increase of the electricity price under the RTPA in the ith period, and then the behavior of ES can be defined as(4)DESi=+Φi,ΔP≥0Thereto, + indicates the increases of planned power generation, and Φi≥0; if the ES does not increase the planned power generation, that is, the ES is idle, and then relative increase of planned power generation is zero at this period, that is, DESi=0.


## 2.1. Power Transmission Lines

From the point of view of power supply and demand system of the smart grid, each node inZ can represent a power generation unit, a power transmission unit, and a power consumption unit (such as a common customers, an industrial consumers, or a data center, and so on), L represents the transmission line between nodes, and the smart grid can be abstracted as GZ,L. In the node set Z, it includes the set of generating nodes PU⊂Z, the set of transmitting nodes OU⊂Z, and the set of electricity utilization nodes DU⊂Z. In the transmission line, LDi represents the power load distribution of the ith transmission line under stable state, LDtotal represents the total load demand of the electrical customers in the stable state, Lmax represents the maximum loadability of each transmission line, and RLP represents the protection rate of power transmission line. When the power load transmission exceeds the maximum loadability of the transmission line, the line i will be damaged due to overload, so the transmission line should meet the following conditions:(1)max⁡RLP∑i∈LLDi≤LDtotal0<LDi≤Lmax


## 2.2. Real-Time Price Attack

In smart grid, there are many ways to attack real-time electricity price signals. For example, attackers can destroy the intermediate nodes of smart grid communication network (such as routers) and obtain the encryption/decryption key in ISO issued/smart meter to intercept and forge data packets containing price data. In addition, attackers launch small-scale tampering attacks against real-time price signals, which may also iteratively amplify the attack effect through feedback, resulting in inefficiency of power system operation and large-scale fluctuation of load demand, and may cause larger and even whole network power failures, such as power outages.RTP is a dynamic price mechanism, the update cycle is usually 1h or 0.5h, or even shorter. At present, the minimum RTP update cycle is 5 minutes on the international. In order to better reflect the game characteristics among ES, attackers and electrical customers, the update cycle of RTP in this paper is 5 minutes. Supposet0 and t are the starting and ending moments of a single-stage game, ΔT is the period length of a single-stage game, and t0=t-ΔT, where DRTPAi is the relative increase of the customers load demand under the RTPA in the ith period, ΔP is the relative increase of the electricity price under the RTPA in the ith period, and then the behavior of RTPA can be defined as(2)DRTPAi=+Γi,ΔP≥0Thereto, + indicates the increases of load demand, and Γi≥0; if the RTPA is in an idle state, the relative increase of load demand is zero at this period; that is, DRTPAi=0.


## 2.3. Stability Control of Home Energy Management System

Some studies have shown that the robustness and convergence of the system model can be improved by optimizing the activation function of machine learning methods such as neural networks, but it cannot meet the requirements of both suppliers and demanders at the same time [18]. But the home energy management system (HEMS) is able to schedule the power consumption scheme of the intelligent electrical appliances optimally according to the information such as the real-time price and reduce the use of the intelligent electrical appliances during the high electricity price; in this way, the supply and demand balance between the electricity supply of ES and the electricity consumption of electrical customers can be balanced. In this paper, to reduce the impact of increased load demand caused by RTPA, HEMS moderately reduces the electrical customers load demand over an optional period of time. Suppose t0 and t are the starting and ending moments of a single-stage game, ΔT is the period length of a single-stage game, and t0=t-ΔT, where DHEMSi is the relative reduction of the customers load demand under the stability control of HEMS in the ith period, ΔP is the relative increase of the electricity price under the RTPA in the ith period, and then the stability control behavior of HEMS can be defined as(3)DHEMSi=-Λi,ΔP≥0Thereto, − indicates the decreases of load demand, and Λi≥0; if the HEMS is in an idle state, the relative decrease of load demand is zero at this period; that is, DHEMSi=0.


## 2.4. Electricity Suppliers Scheduling

By intercepting and forging the electricity price signal from the ES, the attacker maliciously increases the real-time price in the smart meter, causing a large amount of power load in the current period to be transferred to other periods to increase the total load demand on the demand side. From the point of view of ES, in order to meet the increased load demand of electrical customers, the ES can increase the planned power generation. Supposet0 and t are the starting and ending moments of a single-stage game, ΔT is the period length of a single-stage game, and t0=t-ΔT, where DESi is the relative increase of the planned power generation of ES in the ith period, ΔP is the relative increase of the electricity price under the RTPA in the ith period, and then the behavior of ES can be defined as(4)DESi=+Φi,ΔP≥0Thereto, + indicates the increases of planned power generation, and Φi≥0; if the ES does not increase the planned power generation, that is, the ES is idle, and then relative increase of planned power generation is zero at this period, that is, DESi=0.


## 3. Price Elasticity of Electricity Demand

In the process of sending price information from the electricity price database to the Energy Consumption Controller (ECC), RTPA can intercept and forge real-time price signals from ES to improve the electricity price in the ECC of smart meters. After receiving the wrong price information, the power users transfer the load demand of the current period to other periods (such as low price period), which will increase the total load demand of the power users and also increase the power supply pressure of the low price period, which will bring economic losses to the power users and destroy the balance of power supply and demand [19].From (2), (3), and (4), we know that RTPA, HEMS, and ES may be active or idle in each stage of the game. In this paper, we consider eight possible game scenarios in a single-stage period and divide the load demand of electrical customers into single-period (self-elasticity) load demand and multiperiod (cross-elasticity) load demand by the behavior analysis of price elasticity of electricity demand [20], and the impact of RTPA on the total load demand of electrical customers is analyzed by PEED model in eight game scenarios. Aalami et al. proposed single-period and multiperiod models of price elasticity of electricity demand [21]; we obtain a PEED model that contains both self-elasticity and cross-elasticity by reducing the time granularity; the PEED model is as follows:(5)Di=Dbdi+∑j=1288Ei,j×Dbdi×Pgdj-PbdjPbdj(6)Ei,j=Dgdi-Dbdi/DbdiPgdj-Pbdj/PbdjThereto, i=1,2,…,288 is the number of periods; when i=j, Ei,j is the self-elasticity coefficient of the price of electricity demand; when i≠j, Ei,j is the cross-elasticity coefficient of price of electricity demand. Dbdi is the load demand in the ith period of the base day, Pbdj is the real-time price in the jth period of the base day, Dgdi is the load demand in the ith period of the goal day, Pgdj is the real-time price in the jth period of the goal day, and Di is the total load demand of electrical customers in the ith period.Case 1 (idle ES, RTPA and HEMS).
The total load demand of electrical customers is in stable state. In (2), (3), and (4), ΔP=0, DESi=DRTPAi=DHEMSi=0, and the total load demand of electricity users in the ith period is(7)Di=Dbdi+∑j=1288Ei,j×Dbdi×Pgdj-PbdjPbdj≔Di,NCase 2 (idle ES, active RTPA, and idle HEMS).
RTPA is maliciously raising electricity price, resulting in an increase in total load demand. In (3) and (4), DESi=DHEMSi=0, and the total load demand of electricity users in the ith period is(8)Di=Dbdi+∑j=1288Ei,j×Dbdi×Pgdj-PbdjPbdj+DRTPAi=Di,N+DRTPAiCase 3 (idle ES, idle RTPA, and active HEMS).
HEMS aims to reduce the load demand of electrical customers by closing part of the interruptible or noninterruptible load and transferring part of the load demand to other periods in the optional period (such as peak period). In (2) and (4), ΔP=0, DESi=DRTPAi=0, and the total load demand of electricity users in the ith period is(9)Di=Dbdi+∑j=1288Ei,j×Dbdi×Pgdj-PbdjPbdj+DHEMSi=Di,N+DHEMSiCase 4 (idle ES, active RTPA, and active HEMS).
RTPA maliciously raise the electricity price and increase the load demand of electrical customers. Under the stability control of HEMS, electrical customers can manually or automatically transfer part of the load to other periods. At this time, the coefficient of price elasticity of electricity demand mainly shows cross-elasticity. The total load demand of electricity users in theith period is(10)Di=Dbdi+∑j=1288Ei,j×Dbdi×Pgdj-PbdjPbdj+DRTPAi+DHEMSi=Di,N+DRTPAi+DHEMSiunder the combined action of RTPA and HEMS, RTPA can increase the load demand of electrical customers by increasing the real-time price, and meanwhile, HEMS combined with MPZDS minimizes the impact of attacks, so that the total load demand of electrical customers is close to stable value. If DHEMSi<DRTPAi, then the total load demand of electrical customers will be greater than the stable value, that is Di>Di,Ni; if DHEMSi≥DRTPAi, then Di≤Di,Ni, but the stable control gain of HEMS in this case is slightly lower than that of Case 3. In this paper, we assume that both the game formed by HEMS and RTPA are rational, and then the MPZDS analysis in the following based on the condition DHEMSi≥DRTPAi.Case 5 (active ES, idle RTPA, and idle HEMS).
ES increase the planned power generation; then the supply of electricity is greater than the demand of electrical customers for electricity. In (2) and (3), ΔP=0, DRTPAi=DHEMSi=0, and the total load demand of electricity users in the ith period is(11)Di=Dbdi+∑j=1288Ei,j×Dbdi×Pgdj-PbdjPbdj+DESi=Di,N+DESiCase 6 (active ES, active RTPA, and idle HEMS).
RTPA is maliciously raising electricity price, resulting in increased load demand of electrical customers. In (3), DHEMSi=0, and the total load demand of electricity users in the ith period is(12)Di=Dbdi+∑j=1288Ei,j×Dbdi×Pgdj-PbdjPbdj+DESi+DRTPAi=Di,N+DESi+DRTPAiCase 7 (active ES, idle RTPA, and active HEMS).
On the one hand, HEMS aims to reduce the load demand of electrical customers by closing part of the interruptible or noninterruptible load and transferring part of the load demand to other periods in the optional period (such as peak period). On the other hand, ES increase the planned power generation to meet the demand of electrical customers for electricity. In (2), ΔP=DRTPAi=0, and the total load demand of electricity users in the ith period is(13)Di=Dbdi+∑j=1288Ei,j×Dbdi×Pgdj-PbdjPbdj+DESi+DHEMSi=Di,N+DESi+DHEMSiif DHEMSi<DESi, then the total load demand of electrical customers will be greater than the stable value, that is Di>Di,Ni; if DHEMSi≥DESi, then Di≤Di,Ni. In this paper, we assume that both the game formed by ES and HEMS are rational; then the MPZDS analysis in the following is based on the condition DHEMSi≥DESi.Case 8 (active ES, RTPA, and HEMS).
RTPA is maliciously raising electricity price, resulting in increased load demand of electrical customers. The electrical customers can manually or automatically transfer part of the load to other periods under stability control of HEMS. At the same time, ES increase the planned power generation to meet the demand of electrical customers for electricity; the total load demand of electricity users in theith period is(14)Di=Dbdi+∑j=1288Ei,j×Dbdi×Pgdj-PbdjPbdj+DESi+DRTPAi+DHEMSi=Di,N+DESi+DRTPAi+DHEMSiunder the combined action of RTPA, ES, and HEMS, RTPA can increase the load demand of electrical customers by increasing the real-time price, and meanwhile, HEMS combined with MPZDS minimizes the impact of attacks, so that the total load demand of electrical customers is close to stable value. In addition, ES increase the planned power generation to meet the demand of electrical customers for electricity in order to achieve the balance between supply and demand of electricity. If DHEMSi<DRTPAi+DESi, then the total load demand of electrical customers will be greater than the stable value, that is Di>Di,Ni; if DHEMSi≥DRTPAi+DESi, then Di≤Di,Ni. In this paper, we assume that all the three players formed by HEMS, RTPA, and ES are rational, and then the MPZDS analysis in the following is based on DHEMSi≥DRTPAi+DESi.


## 4. Price Elasticity of Electricity Demand

### 4.1. Multiperson Zero-Determinant Strategy

Consider a3×2 repeated game, note that ni(t) is the behavioral states taken by game player i in the tth stage of the game, and “1” means the game player is active in the current game stage and “2” means the game player is idle in the current game stage, that is, n1(t),n2(t),n3(t) are the behavioral states taken by the HEMS, RTPA, and ES, respectively, in the tth stage of the game. Let n(t) be the behavioral states of all game players in the tth stage of the game, and n(t)∈1,23≔S. Based on the PEED model, if HEMS takes action state n1(t)=u, RTPA takes action state n2(t)=v, and ES takes action state n3(t)=w in the current game stage, where u,v,w∈1,2, then Du,v,w represents the total load demand of electrical customers in the current game stage, and the single-stage game payoff matrix is a 2×4 matrix, as shown in Table 1.(15)M=PH1,1,1PA1,1,1PE1,1,1..PH1,1,11-PA1,1,11-PE1,1,1.1-PH1,1,1PA1,1,11-PE1,1,11-PH1,1,11-PA1,1,1PE1,1,11-PH1,1,11-PA1,1,11-PE1,1,1PH1,1,2PA1,1,2PE1,1,2..PH1,1,21-PA1,1,21-PE1,1,2.1-PH1,1,2PA1,1,21-PE1,1,21-PH1,1,21-PA1,1,2PE1,1,21-PH1,1,21-PA1,1,21-PE1,1,2...............PH2,2,1PA2,2,1PE2,2,1..PH2,2,11-PA2,2,11-PE2,2,1.1-PH2,2,1PA2,2,11-PE2,2,11-PH2,2,11-PA2,2,1PE2,2,11-PH2,2,11-PA2,2,11-PE2,2,1PH2,2,2PA2,2,2PE2,2,2..PH2,2,21-PA2,2,21-PE2,2,2.1-PH2,2,2PA2,2,21-PE2,2,21-PH2,2,21-PA2,2,2PE2,2,21-PH2,2,21-PA2,2,21-PE2,2,2Table 1
Single-stage game payoff matrix.
HEMS ES n 3 = 1 n 3 = 2 RTPA n 2 = 1 n 2 = 2 n 2 = 1 n 2 = 2 n 1 = 1 D 1,1 , 1 D 1,2 , 1 D 1,1 , 2 D 1,2 , 2 n 1 = 2 D 2,1 , 1 D 2,2 , 1 D 2,1 , 2 D 2,2 , 2Theorem 1 (see [22]).
InK×N repeated games, both players have the same structure of payoff matrix in each game stage. If a game player has adopted a short-memory strategy, then for this game player, the short-memory strategy can achieve the long-term expected benefits under the long-memory strategy.Based on Theorem1, this paper presents a single-stage game process as a Markov chain with a single-memory cycle. In this paper, n(t)=(n1(t),n2(t),n3(t)) are the behavioral states of the tth stage of the game, Prob• is the probability that each player in the single-stage game takes each behavior state, and k=(k1,k2,k3), then(16)pHk=Probn1t+1=1∣nt=k,∀k∈Sthis formula represents the probability of HEMS taking action 1 in stage t+1 game in the case that HEMS, RTPA, and ES, respectively, adopt behavior states k1,k2,k3 in the tth stage of the game. Similarly, the following equations represent the probability that RTPA and ES will take action 1 in stage t+1.(17)pAk=Probn2t+1=1∣nt=k,∀k∈S(18)pEk=Probn3t+1=1∣nt=k,∀k∈SAccording to (16), (17), and (18), the state transition matrix of the Markov chain can be obtained as an 8×8 matrix as shown formula (15).Letπu,v,w be the probability that the HEMS, RTPA, and ES take the behavioral states u, v, w, and DT=D1,1,1,D1,1,2,D1,2,1,D1,2,2,D2,1,1,D2,1,2,D2,2,1,D2,2,2, and then the Markov chain has a uniquely stationary distribution of πT=π1,1,1,π1,1,2,π1,2,1π1,2,2,π2,1,1,π2,1,2,π2,2,1,π2,2,2 and πTM=πT. After the multistage repeated game, we can get the long-term expected benefits of electrical customers(19)μD=πTDNoteM∧=M-E, then(20)πTM∧=0According to Cramer’s Rule, we can draw the following:(21)adjM∧M∧=detM∧E=0From (20) and (21), We can conclude that det(M∧)=0, then for any vector fT=(f1,f2,f3,f4,f5,f6, f7,f8) there exists the following formula:(22)πTf=-1+PH1,1,1PA1,1,1PE1,1,1..-1+PH1,1,1.-1+PA1,1,1-1+PE1,1,1f1PH1,1,2PA1,1,2PE1,1,2..-1+PH1,1,2.-1+PA1,1,2-1+PE1,1,2f2...............PH2,2,1PA2,2,1PE2,2,1..PH2,2,1.PA2,2,1PE2,2,1f7PH2,2,2PA2,2,2PE2,2,2..PH2,2,2.PA2,2,2PE2,2,2f8NotemHT=(-1+PH1,1,1,-1+PH1,1,2,-1+PH1,2,1, -1+PH1,2,2,PH1,1,1,PH1,1,2,PH1,2,1,PH1,2,2); obviously, mHT is independent of the behavioral state taken by RTPA and ES. Assuming mH=f, then (22) equals zero. Assuming that f=aD+bI, where I is a column vector with all elements of 1, then (22) can be simplified to(23)aμD+b=0thereto, a,b is a nonzero real number.Combining the equationsmH=f and (23), we can get the probability that the HEMS will take the active state in different situations as follows:(24)pH1,1,1=1+1-D1,1,1μDb,pH1,1,2=1+1-D1,1,2μDbpH1,2,1=1+1-D1,2,1μDb,pH1,2,2=1+1-D1,2,2μDbpH2,1,1=1-D2,1,1μDb,pH2,1,2=1-D2,1,2μDbpH2,2,1=1-D2,2,1μDb,pH2,2,2=1-D2,2,2μDb


### 4.2. Analysis of Multiperson Zero-Determinant Strategy

In a single-stage game, RTPA injects false information of electricity price into the smart meter during the periodΔT, so that the indoor smart appliances generate additional load demands. RTPA aims to make the total load demand Di of electrical customers larger than the total load demand Di,Ni under steady state. HEMS optimizes the electrical customer’s electricity plan by optimized scheduling module and turns off some dispatchable loads; if necessary, HEMS can sacrifice partial comfort of electrical customers and turn off some nondispatchable loads. HEMS fully incorporates a MPZDS to minimize the impact of attacks so that the total load demand Di of electrical customers is close to the total load demand Di,Ni under steady state. The value of D is shown in Table 2.Table 2
Single-stage game load demand matrix.
HEMS ES n 3 = 1 n 3 = 2 RTPA n 2 = 1 n 2 = 2 n 2 = 1 n 2 = 2 n 1 = 1 D i , N i + D E S i + D R T P A i + D H E M S i D i , N i + D E S i + D H E M S i D i , N i + D R T P A i + D H E M S i D i , N i + D H E M S i n 1 = 2 D i , N i + D E S i + D R T P A i D i , N i + D E S i D i , N i + D R T P A i D i , N iTheorem 2 (see [23, 24]).
In the repeated game, assuming that the game playersN≥2, Ur,min and Ur,max are, respectively, the maximum and minimum values of row r in the N×2 repeated game payoff matrix, where r=1,2, then(25)Ur,min=min⁡Ur,1,…,Ur,2N-1Ur,max=max⁡Ur,1,…,Ur,2N-1If there existskmin,kmax∈1,2 which makes Ukmax,max≤Ukmin,min satisfied, the game player i can ignore the behavior strategy of other game players and control the long-term expected benefit of game player i in the interval [Ukmax,max,Ukmin,min].LetDr,max and Dr,min be the maximum and minimum values of row u in the single-stage game load demand matrix. Combined with the conclusions of (7)–(14), the maximum and minimum values of row u of D are(26)D1,max=Di,N+DESi+DRTPAi+DHEMSiD1,min=Di,N+DHEMSiD2,max=Di,N+DESi+DRTPAiD2,min=Di,NIn formula (26), D1,max≤D2,min and D2,max≥D1,min; since D1,max≤D2,min, we can get the long-term expected benefit μD of the game based on Theorem 2, so its value range is(27)D1,max,D2,min=Di,N+DESi+DHEMSi+DRTPAi,0Considering the upstream and downstream boundary values of long-term expected benefitμD, μD can be expressed as(28)μD=Di,N+αDESi+DHEMSi+DRTPAiwhere α is the excitation factor and 0≤α≤1. The value of μD increases as α increases and approaches the value of Di,N+DESi+DHEMSi+DRTPAi, which indicates that HEMS is more active in the process of stable control, which can make the total load demand of electrical customers approach the stable state.The value ofb in formula (24) is(29)max⁡-μDμD-D1,min,μDμD-D2,max≤b<0Combining (26), (28), and (29), we can get μD-D1,min≥D2,max-μD when α≥1/2, then bmin≤b<0, where bmin=μD/μD-D2,max, then(30)bmin=Di,N+DESi+DRTPAi+DHEMSiαDHEMSi+α-1DESi+α-1DRTPAiTaking into account the upstream and downstream boundary values ofb, note b=βbmin, then(31)b=β×Di,N+DESi+DRTPAi+DHEMSiαDHEMSi+α-1DESi+α-1DRTPAiwhere β is the control factor and 0≤β≤1. The probability that HEMS will take a stable control action increases as β increases.Assuming that the behavioral states of HEMS, RTPA, and ES areu, v, w, then pHu,v,w is the probability that HEMS will take action 1 in stage t+1 game when HEMS takes action state u, RTPA takes actions state v, and ES takes actions state w under the current game occuring in the period i-1, the tth stage of the game. Combining (24), (28), and (31), the probability of HEMS adopting stable control behavior in the ith period is shown in formula (32) and (33).(32)pH1,1,2=β×α-1DHEMSi+α-1DRTPAi+αDESiαDHEMSi+α-1DRTPAi+α-1DESi×Di,N+DESi+DHEMSi+DRTPAiDi,N+αDESi+DHEMSi+DRTPAi+1pH1,1,1=pH1,1,2-β×DESiαDHEMSi+α-1DRTPAi+α-1DESi×Di,N+DESi+DHEMSi+DRTPAiDi,N+αDESi+DHEMSi+DRTPAipH1,2,2=β×α-1DHEMSi+αDRTPAi+αDESiαDHEMSi+α-1DRTPAi+α-1DESi×Di,N+DESi+DHEMSi+DRTPAiDi,N+αDESi+DHEMSi+DRTPAi+1pH1,2,1=pH1,2,2-β×DESiαDHEMSi+α-1DRTPAi+α-1DESi×Di,N+DESi+DHEMSi+DRTPAiDi,N+αDESi+DHEMSi+DRTPAi(33)pH2,1,2=β×αDHEMSi+α-1DRTPAi+αDESiαDHEMSi+α-1DRTPAi+α-1DESi×Di,N+DESi+DHEMSi+DRTPAiDi,N+αDESi+DHEMSi+DRTPAipH2,1,1=pH2,1,2-β×DESiαDHEMSi+α-1DRTPAi+α-1DESi×Di,N+DESi+DHEMSi+DRTPAiDi,N+αDESi+DHEMSi+DRTPAipH2,2,2=β×αDHEMSi+DRTPAi+DESiαDHEMSi+α-1DRTPAi+α-1DESi×Di,N+DESi+DHEMSi+DRTPAiDi,N+αDESi+DHEMSi+DRTPAipH2,2,1=pH2,2,2-β×DESiαDHEMSi+α-1DRTPAi+α-1DESi×Di,N+DESi+DHEMSi+DRTPAiDi,N+αDESi+DHEMSi+DRTPAiIfα=1, indicating that HEMS is the most active in the process of stable control, then the probability of HEMS behavior state is(34)pH1,1,1=1,pH1,1,2=pH1,1,1+β×DESiDHEMSipH1,2,1=1+β×DRTPAiDHEMSi,pH1,2,2=pH1,2,1+β×DESiDHEMSipH2,1,1=β,pH2,1,2=pH2,1,1+β×DESiDHEMSipH2,2,1=β×DHEMSi+DRTPAiDHEMSi,pH2,2,2=pH2,2,1+β×DESiDHEMSiEquation (34) shows that if RTPA is active in the current game stage, the probability of HEMS taking stable control action will increase in the next round of game stage; if ES is active, the probability that HEMS taking stable control action will also increase in the next round of game stage. This behavioral characteristic shows that if RTPA is active, HEMS can take corresponding stable control behavior, and ES can optimize the behavior selection of HEMS to a certain extent in order to reduce the impact of the increase of total load demand caused by RTPA.


## 4.1. Multiperson Zero-Determinant Strategy

Consider a3×2 repeated game, note that ni(t) is the behavioral states taken by game player i in the tth stage of the game, and “1” means the game player is active in the current game stage and “2” means the game player is idle in the current game stage, that is, n1(t),n2(t),n3(t) are the behavioral states taken by the HEMS, RTPA, and ES, respectively, in the tth stage of the game. Let n(t) be the behavioral states of all game players in the tth stage of the game, and n(t)∈1,23≔S. Based on the PEED model, if HEMS takes action state n1(t)=u, RTPA takes action state n2(t)=v, and ES takes action state n3(t)=w in the current game stage, where u,v,w∈1,2, then Du,v,w represents the total load demand of electrical customers in the current game stage, and the single-stage game payoff matrix is a 2×4 matrix, as shown in Table 1.(15)M=PH1,1,1PA1,1,1PE1,1,1..PH1,1,11-PA1,1,11-PE1,1,1.1-PH1,1,1PA1,1,11-PE1,1,11-PH1,1,11-PA1,1,1PE1,1,11-PH1,1,11-PA1,1,11-PE1,1,1PH1,1,2PA1,1,2PE1,1,2..PH1,1,21-PA1,1,21-PE1,1,2.1-PH1,1,2PA1,1,21-PE1,1,21-PH1,1,21-PA1,1,2PE1,1,21-PH1,1,21-PA1,1,21-PE1,1,2...............PH2,2,1PA2,2,1PE2,2,1..PH2,2,11-PA2,2,11-PE2,2,1.1-PH2,2,1PA2,2,11-PE2,2,11-PH2,2,11-PA2,2,1PE2,2,11-PH2,2,11-PA2,2,11-PE2,2,1PH2,2,2PA2,2,2PE2,2,2..PH2,2,21-PA2,2,21-PE2,2,2.1-PH2,2,2PA2,2,21-PE2,2,21-PH2,2,21-PA2,2,2PE2,2,21-PH2,2,21-PA2,2,21-PE2,2,2Table 1
Single-stage game payoff matrix.
HEMS ES n 3 = 1 n 3 = 2 RTPA n 2 = 1 n 2 = 2 n 2 = 1 n 2 = 2 n 1 = 1 D 1,1 , 1 D 1,2 , 1 D 1,1 , 2 D 1,2 , 2 n 1 = 2 D 2,1 , 1 D 2,2 , 1 D 2,1 , 2 D 2,2 , 2Theorem 1 (see [22]).
InK×N repeated games, both players have the same structure of payoff matrix in each game stage. If a game player has adopted a short-memory strategy, then for this game player, the short-memory strategy can achieve the long-term expected benefits under the long-memory strategy.Based on Theorem1, this paper presents a single-stage game process as a Markov chain with a single-memory cycle. In this paper, n(t)=(n1(t),n2(t),n3(t)) are the behavioral states of the tth stage of the game, Prob• is the probability that each player in the single-stage game takes each behavior state, and k=(k1,k2,k3), then(16)pHk=Probn1t+1=1∣nt=k,∀k∈Sthis formula represents the probability of HEMS taking action 1 in stage t+1 game in the case that HEMS, RTPA, and ES, respectively, adopt behavior states k1,k2,k3 in the tth stage of the game. Similarly, the following equations represent the probability that RTPA and ES will take action 1 in stage t+1.(17)pAk=Probn2t+1=1∣nt=k,∀k∈S(18)pEk=Probn3t+1=1∣nt=k,∀k∈SAccording to (16), (17), and (18), the state transition matrix of the Markov chain can be obtained as an 8×8 matrix as shown formula (15).Letπu,v,w be the probability that the HEMS, RTPA, and ES take the behavioral states u, v, w, and DT=D1,1,1,D1,1,2,D1,2,1,D1,2,2,D2,1,1,D2,1,2,D2,2,1,D2,2,2, and then the Markov chain has a uniquely stationary distribution of πT=π1,1,1,π1,1,2,π1,2,1π1,2,2,π2,1,1,π2,1,2,π2,2,1,π2,2,2 and πTM=πT. After the multistage repeated game, we can get the long-term expected benefits of electrical customers(19)μD=πTDNoteM∧=M-E, then(20)πTM∧=0According to Cramer’s Rule, we can draw the following:(21)adjM∧M∧=detM∧E=0From (20) and (21), We can conclude that det(M∧)=0, then for any vector fT=(f1,f2,f3,f4,f5,f6, f7,f8) there exists the following formula:(22)πTf=-1+PH1,1,1PA1,1,1PE1,1,1..-1+PH1,1,1.-1+PA1,1,1-1+PE1,1,1f1PH1,1,2PA1,1,2PE1,1,2..-1+PH1,1,2.-1+PA1,1,2-1+PE1,1,2f2...............PH2,2,1PA2,2,1PE2,2,1..PH2,2,1.PA2,2,1PE2,2,1f7PH2,2,2PA2,2,2PE2,2,2..PH2,2,2.PA2,2,2PE2,2,2f8NotemHT=(-1+PH1,1,1,-1+PH1,1,2,-1+PH1,2,1, -1+PH1,2,2,PH1,1,1,PH1,1,2,PH1,2,1,PH1,2,2); obviously, mHT is independent of the behavioral state taken by RTPA and ES. Assuming mH=f, then (22) equals zero. Assuming that f=aD+bI, where I is a column vector with all elements of 1, then (22) can be simplified to(23)aμD+b=0thereto, a,b is a nonzero real number.Combining the equationsmH=f and (23), we can get the probability that the HEMS will take the active state in different situations as follows:(24)pH1,1,1=1+1-D1,1,1μDb,pH1,1,2=1+1-D1,1,2μDbpH1,2,1=1+1-D1,2,1μDb,pH1,2,2=1+1-D1,2,2μDbpH2,1,1=1-D2,1,1μDb,pH2,1,2=1-D2,1,2μDbpH2,2,1=1-D2,2,1μDb,pH2,2,2=1-D2,2,2μDb


## 4.2. Analysis of Multiperson Zero-Determinant Strategy

In a single-stage game, RTPA injects false information of electricity price into the smart meter during the periodΔT, so that the indoor smart appliances generate additional load demands. RTPA aims to make the total load demand Di of electrical customers larger than the total load demand Di,Ni under steady state. HEMS optimizes the electrical customer’s electricity plan by optimized scheduling module and turns off some dispatchable loads; if necessary, HEMS can sacrifice partial comfort of electrical customers and turn off some nondispatchable loads. HEMS fully incorporates a MPZDS to minimize the impact of attacks so that the total load demand Di of electrical customers is close to the total load demand Di,Ni under steady state. The value of D is shown in Table 2.Table 2
Single-stage game load demand matrix.
HEMS ES n 3 = 1 n 3 = 2 RTPA n 2 = 1 n 2 = 2 n 2 = 1 n 2 = 2 n 1 = 1 D i , N i + D E S i + D R T P A i + D H E M S i D i , N i + D E S i + D H E M S i D i , N i + D R T P A i + D H E M S i D i , N i + D H E M S i n 1 = 2 D i , N i + D E S i + D R T P A i D i , N i + D E S i D i , N i + D R T P A i D i , N iTheorem 2 (see [23, 24]).
In the repeated game, assuming that the game playersN≥2, Ur,min and Ur,max are, respectively, the maximum and minimum values of row r in the N×2 repeated game payoff matrix, where r=1,2, then(25)Ur,min=min⁡Ur,1,…,Ur,2N-1Ur,max=max⁡Ur,1,…,Ur,2N-1If there existskmin,kmax∈1,2 which makes Ukmax,max≤Ukmin,min satisfied, the game player i can ignore the behavior strategy of other game players and control the long-term expected benefit of game player i in the interval [Ukmax,max,Ukmin,min].LetDr,max and Dr,min be the maximum and minimum values of row u in the single-stage game load demand matrix. Combined with the conclusions of (7)–(14), the maximum and minimum values of row u of D are(26)D1,max=Di,N+DESi+DRTPAi+DHEMSiD1,min=Di,N+DHEMSiD2,max=Di,N+DESi+DRTPAiD2,min=Di,NIn formula (26), D1,max≤D2,min and D2,max≥D1,min; since D1,max≤D2,min, we can get the long-term expected benefit μD of the game based on Theorem 2, so its value range is(27)D1,max,D2,min=Di,N+DESi+DHEMSi+DRTPAi,0Considering the upstream and downstream boundary values of long-term expected benefitμD, μD can be expressed as(28)μD=Di,N+αDESi+DHEMSi+DRTPAiwhere α is the excitation factor and 0≤α≤1. The value of μD increases as α increases and approaches the value of Di,N+DESi+DHEMSi+DRTPAi, which indicates that HEMS is more active in the process of stable control, which can make the total load demand of electrical customers approach the stable state.The value ofb in formula (24) is(29)max⁡-μDμD-D1,min,μDμD-D2,max≤b<0Combining (26), (28), and (29), we can get μD-D1,min≥D2,max-μD when α≥1/2, then bmin≤b<0, where bmin=μD/μD-D2,max, then(30)bmin=Di,N+DESi+DRTPAi+DHEMSiαDHEMSi+α-1DESi+α-1DRTPAiTaking into account the upstream and downstream boundary values ofb, note b=βbmin, then(31)b=β×Di,N+DESi+DRTPAi+DHEMSiαDHEMSi+α-1DESi+α-1DRTPAiwhere β is the control factor and 0≤β≤1. The probability that HEMS will take a stable control action increases as β increases.Assuming that the behavioral states of HEMS, RTPA, and ES areu, v, w, then pHu,v,w is the probability that HEMS will take action 1 in stage t+1 game when HEMS takes action state u, RTPA takes actions state v, and ES takes actions state w under the current game occuring in the period i-1, the tth stage of the game. Combining (24), (28), and (31), the probability of HEMS adopting stable control behavior in the ith period is shown in formula (32) and (33).(32)pH1,1,2=β×α-1DHEMSi+α-1DRTPAi+αDESiαDHEMSi+α-1DRTPAi+α-1DESi×Di,N+DESi+DHEMSi+DRTPAiDi,N+αDESi+DHEMSi+DRTPAi+1pH1,1,1=pH1,1,2-β×DESiαDHEMSi+α-1DRTPAi+α-1DESi×Di,N+DESi+DHEMSi+DRTPAiDi,N+αDESi+DHEMSi+DRTPAipH1,2,2=β×α-1DHEMSi+αDRTPAi+αDESiαDHEMSi+α-1DRTPAi+α-1DESi×Di,N+DESi+DHEMSi+DRTPAiDi,N+αDESi+DHEMSi+DRTPAi+1pH1,2,1=pH1,2,2-β×DESiαDHEMSi+α-1DRTPAi+α-1DESi×Di,N+DESi+DHEMSi+DRTPAiDi,N+αDESi+DHEMSi+DRTPAi(33)pH2,1,2=β×αDHEMSi+α-1DRTPAi+αDESiαDHEMSi+α-1DRTPAi+α-1DESi×Di,N+DESi+DHEMSi+DRTPAiDi,N+αDESi+DHEMSi+DRTPAipH2,1,1=pH2,1,2-β×DESiαDHEMSi+α-1DRTPAi+α-1DESi×Di,N+DESi+DHEMSi+DRTPAiDi,N+αDESi+DHEMSi+DRTPAipH2,2,2=β×αDHEMSi+DRTPAi+DESiαDHEMSi+α-1DRTPAi+α-1DESi×Di,N+DESi+DHEMSi+DRTPAiDi,N+αDESi+DHEMSi+DRTPAipH2,2,1=pH2,2,2-β×DESiαDHEMSi+α-1DRTPAi+α-1DESi×Di,N+DESi+DHEMSi+DRTPAiDi,N+αDESi+DHEMSi+DRTPAiIfα=1, indicating that HEMS is the most active in the process of stable control, then the probability of HEMS behavior state is(34)pH1,1,1=1,pH1,1,2=pH1,1,1+β×DESiDHEMSipH1,2,1=1+β×DRTPAiDHEMSi,pH1,2,2=pH1,2,1+β×DESiDHEMSipH2,1,1=β,pH2,1,2=pH2,1,1+β×DESiDHEMSipH2,2,1=β×DHEMSi+DRTPAiDHEMSi,pH2,2,2=pH2,2,1+β×DESiDHEMSiEquation (34) shows that if RTPA is active in the current game stage, the probability of HEMS taking stable control action will increase in the next round of game stage; if ES is active, the probability that HEMS taking stable control action will also increase in the next round of game stage. This behavioral characteristic shows that if RTPA is active, HEMS can take corresponding stable control behavior, and ES can optimize the behavior selection of HEMS to a certain extent in order to reduce the impact of the increase of total load demand caused by RTPA.


## 5. Experiment Analysis

### 5.1. The Probability of HEMS Being Active

In this paper, the experimental hardware platform is AMD A8-5550M quad-core processor, 2.1GHz frequency, 6GB memory, software platform: Windows 10 operating system, Eclipse with PyDev integrated simulation tools, and MATLAB R2014a experimental environment; among them, Eclipse is 4.6.2; Python is 3.5.2. The experimental data uses the real-time price and electricity load demand data of the New South Wales of Australian Energy Market on April 6, 2017, and April 7, 2017, thereto, the data of April 6 for the base day data; data of April 7 for the goal day data. The significance of the data selection is that the weather and other environmental conditions and the electricity consumption habits of electrical customers are almost at the same in two adjacent days; the load demand of electrical customers is protected from weather conditions and electricity consumption habits of electrical customers. The coefficient of price elasticity of electricity demand is calculated from (6).In this paper, we assume that the update cycle of real-time price is 5min, and one day is divided into 288 periods. As a result, the daily load demand curve can be divided into three periods: low period, flat period, and peak period, as shown in Table3.Table 3
Segment of real-time price time.
Periods The start and end time of period Duration/h Low 00:00-06:00, 22:00-24:00 8 Flat 06:00-07:00, 11:00-18:00, 20:00-22:00 10 Peak 07:00-11:00, 18:00-20:00 6It is assumed that the real-time price market is in a normal state during the flat period, that is, when198≤i≤209; assume that an attacker intercepts the real-time price signal from the electricity suppliers and tamper with the real-time price information when i=210; assume that HEMS, RTPA, and ES are rational. Then under the ES scheduling, and eight kinds of game situations, HEMS take the probability (formula (32) and (33)) of action 1 (active) in the next game stage and the relationship of α, β shown in Figures 1 and 2, where (P111,P112,P121,P122,P211,P212,P221,P222)≔pH1,1,1,pH1,1,2,pH1,2,1,pH1,2,2,pH2,1,1,pH2,1,2,pH2,2,1,pH2,2,2.Figure 1
HEMS take the probability of action 1 in the next game stage.
(a)
HEMS take the probability of action 1 in the next game stage and the relationship ofα (b)
HEMS take the probability of action 1 in the next game stage and the relationship ofβFigure 2
The expected load demands of the four strategies vary with the number of different game stages.As can be seen from Figure1(a), the probability that HEMS will take action 1 in the next game stage will increase as α increases, no matter what kind of behavior of the attackers and electricity suppliers are in the previous game stage. If the HEMS is active during the previous game stage, the probability that the HEMS will take action 1 in the next game stage is significantly greater than the probability that the HEMS was idle during the previous game stage. If the RTPA is active during the previous game stage, the probability that HEMS will take action 1 in the next game stage is also greater than the probability that RTPA was idle during the previous game stage, the reason is that in the game process, HEMS has a memory depth of 1 under combining with a MPZDS, which can memorize the behavioral state of the three game players in the previous game stage to optimize the behavior selection of HEMS in the next game stage. In the previous game stage, if the behavior states of HEMS and RTPA remain unchanged, when the ES is active, the probability that HEMS takes action 1 in the next game stage is greater than the probability that ES was idle state in the previous game stage, the reason is that the dispatching effect of ES on load demand of electrical customers aims to make the power supply and demand balance, and this shows that ES can optimize the behavior selection of HEMS in the next game stage by participating in game. Similarly, as can also be seen from Figure 1(b), if the RTPA is active in the previous game stage, the probability that the HEMS will take action 1 in the next game stage is significantly greater than the probability that RTPA was idle in the previous game stage. Although the probability that HEMS takes action 1 in the next game stage decreases slightly when the HEMS was active in the previous game stage, but the mean value of the probability which is still greater or equal to the average of the HEMS was idle state in the previous game stage, and pH1,1,1+pH2,2,2=1, pH1,1,2+pH2,2,1=1, pH1,2,1+pH2,1,2=1, and pH1,2,2+pH2,1,1=1, and the relations of pH1,1,1 and pH2,2,2, pH1,1,2 and pH2,2,1, pH1,2,1 and pH2,1,2, and pH1,2,2 and pH2,1,1 are shown to be complementary.


### 5.2. Expected Load Demand Reduction Rate of Electrical Customers

Assuming that the number of repeated game phases is 300 round and the game period isΔT=5min, we, respectively, simulate the four kinds of single-memory strategies under different game stage numbers to get the game sequence of the corresponding strategies and calculate the expected load demand. The four kinds of single-memory strategies are Multiperson Zero-Determinant Strategies (MPZDS), Random Strategies (RS), Win-Stay-Lose-Shift Strategies [25] (WSLS), and Stochastic Cooperator Strategies [26] (SCS). Figure 2 shows the comparison between the four strategies and the expected load demand in stable state under different game stage numbers.As can be seen from Figure2, expected load demand reduction rate of MPZDS was 5.4%, expected load demand reduction rate of RS was 38.5%, expected load demand reduction rate of WSLS was 36.9%, and expected load demand reduction rate of SCS was 32.2% compared with the expected load demand of stable state. The expected load demand of MPZDS is closer to the expected load demand under stable state, because HEMS takes the behavioral state “1” with a high probability in combination with MPZDS under the condition of RTPA, to achieve stable control effect of the total load demand of electrical customers, and ES also allocates a certain amount of power to the electrical customers in order to balance the power supply and demand. Electrical customers do not need to turn off schedulable or nonschedulable loads too much while ensuring electricity consumption satisfaction under the MPZDS, but RS, WSLS, and SCS greatly reduces the total load demand of electrical customers by shutting off schedulable or nonschedulable loads, although it achieves the goal of preventing RTPA to a certain extent; it also greatly reduces the satisfaction with electricity consumption of electrical customers.Figure3 compares the expected load demand of four kinds of single-memory strategy and stable state under RTPA which occurs at different periods of time. Among them, the trough hours take i=272 (22: 00-24: 00), flat hours take i=210 (11: 00-18: 00), and peak hours take i=123 (07: 00-11: 00). As can be seen from Figure 3, if the RTPA occurs in the trough period, the expected load demand reduction rates of MPZDS, RS, WSLS, and SCS are 14.8%, 44.6%, 43.1%, and 39%, respectively; if RTPA occurs in the flat period, the expected load demand reduction rates of MPZDS, RS, WSLS, and SCS are 5.4%, 38.5%, 36.9%, and 32.2%, respectively; if RTPA occurs in the peak period, the expected load demand reduction rates of MPZDS, RS, WSLS, and SCS are 6.2%, 39%, 37.4%, and 32.8%, respectively. From the above analysis, we can see that MPZDS has better attack prevention effect than RS, WSLS, and SCS in view of RTPA maliciously increasing the total load demand of electrical customers; the reason is that, due to the influence of incentive factor α and control factor β and the electricity supplier’s scheduling, the probability that HEMS is active in the next game stage is greater than that of RS, WSLS, and SCS by taking MPZDS. In addition, Figure 3 shows that the expected load demand reduction rate during flat period is lower than the expected load demand reduction rates during peak period and trough period under the premise of HEMS combined with MPZDS; the reason is that the total load demand of electrical customers during flat period is stable and the actual load demand is lower than that during trough period and peak period; HEMS does not need to turn off a large number of schedulable or nonschedulable loads, so as to achieve a better load demand scheduling solution.Figure 3
Comparing expected load demands of different periods.


### 5.3. Power Transmission Line Protection Rate

It is assumed that the RTPA occurs during the peak period and the number of power transmission lines is 84. The expected load demand under a stable state is averaged out, and the maximum loadability of each transmission line is89.34MW, Figures 4(a), 4(b), and 4(c), respectively, when Lmax<89.34MW, Lmax=89.34MW, and Lmax>89.34MW, the protection rate of the power transmission line in four strategies.Figure 4
Power transmission line protection rates.
(a)
L m a x < 89.34 M W (b)
L m a x = 89.34 M W (c)
L m a x > 89.34 M WFrom Figures4(a), 4(b), and 4(c), if the maximum loadability of each transmission line is less than 89.34MW, the average protection ratios of power transmission lines under MPZDS, RS, WSLS, and SCS are, respectively, 44.8%, 29.6%, 31.3 %, and 31%, and the maximum power transmission line protection rates of MPZDS, RS, WSLS, and SCS are, respectively, 49.6%, 32%, 46.6%, and 39.7%, and the minimum power transmission line protection rates of MPZDS, RS, WSLS, and SCS are, respectively, 32.4%, 24.7%, 22.3%, and 22.1%; if the maximum loadability of each transmission line is equal to 89.34MW, the average protection ratios of power transmission lines under MPZDS, RS, WSLS, and SCS are, respectively, 94.8%, 79.1%, 80.7%, and 80.3%, and the maximum power transmission line protection rates of MPZDS, RS, WSLS, and SCS are, respectively, 99.6%, 82%, 96.6%, and 89.7%, and the minimum power transmission line protection rates of MPZDS, RS, WSLS, and SCS are, respectively, 82.4%, 72.4%, 68.1%, and 67.8%; if the maximum loadability of each transmission line is greater than 89.34MW, the average protection ratios of power transmission lines under MPZDS, RS, WSLS, and SCS are, respectively, 99.9%, 99.4%, 99.4%, and 99.4%, and the maximum power transmission line protection rates of MPZDS, RS, WSLS, and SCS are, respectively, 100%, 99.9%, 100%, and 100%, and the minimum power transmission line protection rates of MPZDS, RS, WSLS, and SCS are, respectively, 99.9%, 97.8%, 95.9%, and 95.7%. Therefore, the protection effect of MPZDS on power transmission lines is obviously better than that of RS, WSLS, and SCS, and the protection rate of power transmission lines increases with the increase of the maximum loadability of single transmission line. The reason is that when HEMS is combined with MPZDS, the active state is taken with a high probability, so that the expected load demand of electrical customers is closer to the expected load demand in stable state; however, RS, WSLS, and SCS significantly reduce the electricity load demand of electrical customers at the expense of sacrificing the electricity satisfaction of electrical customers, so that the expected load demand of electrical customers deviates excessively from the expected load demands under stable state.


## 5.1. The Probability of HEMS Being Active

In this paper, the experimental hardware platform is AMD A8-5550M quad-core processor, 2.1GHz frequency, 6GB memory, software platform: Windows 10 operating system, Eclipse with PyDev integrated simulation tools, and MATLAB R2014a experimental environment; among them, Eclipse is 4.6.2; Python is 3.5.2. The experimental data uses the real-time price and electricity load demand data of the New South Wales of Australian Energy Market on April 6, 2017, and April 7, 2017, thereto, the data of April 6 for the base day data; data of April 7 for the goal day data. The significance of the data selection is that the weather and other environmental conditions and the electricity consumption habits of electrical customers are almost at the same in two adjacent days; the load demand of electrical customers is protected from weather conditions and electricity consumption habits of electrical customers. The coefficient of price elasticity of electricity demand is calculated from (6).In this paper, we assume that the update cycle of real-time price is 5min, and one day is divided into 288 periods. As a result, the daily load demand curve can be divided into three periods: low period, flat period, and peak period, as shown in Table3.Table 3
Segment of real-time price time.
Periods The start and end time of period Duration/h Low 00:00-06:00, 22:00-24:00 8 Flat 06:00-07:00, 11:00-18:00, 20:00-22:00 10 Peak 07:00-11:00, 18:00-20:00 6It is assumed that the real-time price market is in a normal state during the flat period, that is, when198≤i≤209; assume that an attacker intercepts the real-time price signal from the electricity suppliers and tamper with the real-time price information when i=210; assume that HEMS, RTPA, and ES are rational. Then under the ES scheduling, and eight kinds of game situations, HEMS take the probability (formula (32) and (33)) of action 1 (active) in the next game stage and the relationship of α, β shown in Figures 1 and 2, where (P111,P112,P121,P122,P211,P212,P221,P222)≔pH1,1,1,pH1,1,2,pH1,2,1,pH1,2,2,pH2,1,1,pH2,1,2,pH2,2,1,pH2,2,2.Figure 1
HEMS take the probability of action 1 in the next game stage.
(a)
HEMS take the probability of action 1 in the next game stage and the relationship ofα (b)
HEMS take the probability of action 1 in the next game stage and the relationship ofβFigure 2
The expected load demands of the four strategies vary with the number of different game stages.As can be seen from Figure1(a), the probability that HEMS will take action 1 in the next game stage will increase as α increases, no matter what kind of behavior of the attackers and electricity suppliers are in the previous game stage. If the HEMS is active during the previous game stage, the probability that the HEMS will take action 1 in the next game stage is significantly greater than the probability that the HEMS was idle during the previous game stage. If the RTPA is active during the previous game stage, the probability that HEMS will take action 1 in the next game stage is also greater than the probability that RTPA was idle during the previous game stage, the reason is that in the game process, HEMS has a memory depth of 1 under combining with a MPZDS, which can memorize the behavioral state of the three game players in the previous game stage to optimize the behavior selection of HEMS in the next game stage. In the previous game stage, if the behavior states of HEMS and RTPA remain unchanged, when the ES is active, the probability that HEMS takes action 1 in the next game stage is greater than the probability that ES was idle state in the previous game stage, the reason is that the dispatching effect of ES on load demand of electrical customers aims to make the power supply and demand balance, and this shows that ES can optimize the behavior selection of HEMS in the next game stage by participating in game. Similarly, as can also be seen from Figure 1(b), if the RTPA is active in the previous game stage, the probability that the HEMS will take action 1 in the next game stage is significantly greater than the probability that RTPA was idle in the previous game stage. Although the probability that HEMS takes action 1 in the next game stage decreases slightly when the HEMS was active in the previous game stage, but the mean value of the probability which is still greater or equal to the average of the HEMS was idle state in the previous game stage, and pH1,1,1+pH2,2,2=1, pH1,1,2+pH2,2,1=1, pH1,2,1+pH2,1,2=1, and pH1,2,2+pH2,1,1=1, and the relations of pH1,1,1 and pH2,2,2, pH1,1,2 and pH2,2,1, pH1,2,1 and pH2,1,2, and pH1,2,2 and pH2,1,1 are shown to be complementary.


## 5.2. Expected Load Demand Reduction Rate of Electrical Customers

Assuming that the number of repeated game phases is 300 round and the game period isΔT=5min, we, respectively, simulate the four kinds of single-memory strategies under different game stage numbers to get the game sequence of the corresponding strategies and calculate the expected load demand. The four kinds of single-memory strategies are Multiperson Zero-Determinant Strategies (MPZDS), Random Strategies (RS), Win-Stay-Lose-Shift Strategies [25] (WSLS), and Stochastic Cooperator Strategies [26] (SCS). Figure 2 shows the comparison between the four strategies and the expected load demand in stable state under different game stage numbers.As can be seen from Figure2, expected load demand reduction rate of MPZDS was 5.4%, expected load demand reduction rate of RS was 38.5%, expected load demand reduction rate of WSLS was 36.9%, and expected load demand reduction rate of SCS was 32.2% compared with the expected load demand of stable state. The expected load demand of MPZDS is closer to the expected load demand under stable state, because HEMS takes the behavioral state “1” with a high probability in combination with MPZDS under the condition of RTPA, to achieve stable control effect of the total load demand of electrical customers, and ES also allocates a certain amount of power to the electrical customers in order to balance the power supply and demand. Electrical customers do not need to turn off schedulable or nonschedulable loads too much while ensuring electricity consumption satisfaction under the MPZDS, but RS, WSLS, and SCS greatly reduces the total load demand of electrical customers by shutting off schedulable or nonschedulable loads, although it achieves the goal of preventing RTPA to a certain extent; it also greatly reduces the satisfaction with electricity consumption of electrical customers.Figure3 compares the expected load demand of four kinds of single-memory strategy and stable state under RTPA which occurs at different periods of time. Among them, the trough hours take i=272 (22: 00-24: 00), flat hours take i=210 (11: 00-18: 00), and peak hours take i=123 (07: 00-11: 00). As can be seen from Figure 3, if the RTPA occurs in the trough period, the expected load demand reduction rates of MPZDS, RS, WSLS, and SCS are 14.8%, 44.6%, 43.1%, and 39%, respectively; if RTPA occurs in the flat period, the expected load demand reduction rates of MPZDS, RS, WSLS, and SCS are 5.4%, 38.5%, 36.9%, and 32.2%, respectively; if RTPA occurs in the peak period, the expected load demand reduction rates of MPZDS, RS, WSLS, and SCS are 6.2%, 39%, 37.4%, and 32.8%, respectively. From the above analysis, we can see that MPZDS has better attack prevention effect than RS, WSLS, and SCS in view of RTPA maliciously increasing the total load demand of electrical customers; the reason is that, due to the influence of incentive factor α and control factor β and the electricity supplier’s scheduling, the probability that HEMS is active in the next game stage is greater than that of RS, WSLS, and SCS by taking MPZDS. In addition, Figure 3 shows that the expected load demand reduction rate during flat period is lower than the expected load demand reduction rates during peak period and trough period under the premise of HEMS combined with MPZDS; the reason is that the total load demand of electrical customers during flat period is stable and the actual load demand is lower than that during trough period and peak period; HEMS does not need to turn off a large number of schedulable or nonschedulable loads, so as to achieve a better load demand scheduling solution.Figure 3
Comparing expected load demands of different periods.


## 5.3. Power Transmission Line Protection Rate

It is assumed that the RTPA occurs during the peak period and the number of power transmission lines is 84. The expected load demand under a stable state is averaged out, and the maximum loadability of each transmission line is89.34MW, Figures 4(a), 4(b), and 4(c), respectively, when Lmax<89.34MW, Lmax=89.34MW, and Lmax>89.34MW, the protection rate of the power transmission line in four strategies.Figure 4
Power transmission line protection rates.
(a)
L m a x < 89.34 M W (b)
L m a x = 89.34 M W (c)
L m a x > 89.34 M WFrom Figures4(a), 4(b), and 4(c), if the maximum loadability of each transmission line is less than 89.34MW, the average protection ratios of power transmission lines under MPZDS, RS, WSLS, and SCS are, respectively, 44.8%, 29.6%, 31.3 %, and 31%, and the maximum power transmission line protection rates of MPZDS, RS, WSLS, and SCS are, respectively, 49.6%, 32%, 46.6%, and 39.7%, and the minimum power transmission line protection rates of MPZDS, RS, WSLS, and SCS are, respectively, 32.4%, 24.7%, 22.3%, and 22.1%; if the maximum loadability of each transmission line is equal to 89.34MW, the average protection ratios of power transmission lines under MPZDS, RS, WSLS, and SCS are, respectively, 94.8%, 79.1%, 80.7%, and 80.3%, and the maximum power transmission line protection rates of MPZDS, RS, WSLS, and SCS are, respectively, 99.6%, 82%, 96.6%, and 89.7%, and the minimum power transmission line protection rates of MPZDS, RS, WSLS, and SCS are, respectively, 82.4%, 72.4%, 68.1%, and 67.8%; if the maximum loadability of each transmission line is greater than 89.34MW, the average protection ratios of power transmission lines under MPZDS, RS, WSLS, and SCS are, respectively, 99.9%, 99.4%, 99.4%, and 99.4%, and the maximum power transmission line protection rates of MPZDS, RS, WSLS, and SCS are, respectively, 100%, 99.9%, 100%, and 100%, and the minimum power transmission line protection rates of MPZDS, RS, WSLS, and SCS are, respectively, 99.9%, 97.8%, 95.9%, and 95.7%. Therefore, the protection effect of MPZDS on power transmission lines is obviously better than that of RS, WSLS, and SCS, and the protection rate of power transmission lines increases with the increase of the maximum loadability of single transmission line. The reason is that when HEMS is combined with MPZDS, the active state is taken with a high probability, so that the expected load demand of electrical customers is closer to the expected load demand in stable state; however, RS, WSLS, and SCS significantly reduce the electricity load demand of electrical customers at the expense of sacrificing the electricity satisfaction of electrical customers, so that the expected load demand of electrical customers deviates excessively from the expected load demands under stable state.


## 6. Conclusions

Aiming at the problem that RTPA increases the total load demand of electrical customers, we propose a defensive strategy of real-time price attack based on MPZDS. In order to achieve the goal of stabilizing the total load demand of electrical customers, firstly, according to the game relationship between RTPA, HEMS, and ES, the behavior characteristics of the three players are, respectively, defined. Secondly, we analyze eight kinds of game situations among the three players and get the total load demand of electrical customers. Finally, we combine the MPZDS for safety analysis. Experimental results show that the proposed method of the paper has a lower expected load demand reduction rate and can better protect the safety of power transmission lines. In the future research work, in addition to considering the natural factors such as new energy, it will also consider the impact of collaborative real-time price attack on load demand of electrical customers and electricity market. At the same time, new defense methods such as machine learning or intelligent judgment will also be considered in defensive strategies of real-time price attack [27, 28].



---

*Source: 6956072-2019-07-16.xml*