Adversarial attack and defense methods for neural network based state estimation in smart grid

IET Renewable Power GenerationEarly View ORIGINAL RESEARCH PAPEROpen Access Adversarial attack and defense methods for neural network based state estimation in smart grid Jiwei Tian, Corresponding Author Jiwei Tian tianjiwei2016@163.com orcid.org/0000-0002-1485-7465 ATC Navigation College, Air Force Engineering University, Xi'an, China Correspondence Jiwei Tian, ATC Navigation College, Air Force Engineering University, No. 1, Changle East Road, Baqiao District, Xi'an City, Shaanxi Province, China 710043. Email: tianjiwei2016@163.comSearch for more papers by this authorBuhong Wang, Buhong Wang Information and Navigation College, Air Force Engineering University, Xi'an, ChinaSearch for more papers by this authorJing Li, Jing Li School of Design and Art, Henan University of Technology, Zhengzhou, ChinaSearch for more papers by this authorCharalambos Konstantinou, Charalambos Konstantinou orcid.org/0000-0002-3825-3930 KAUST, Thuwal, Saudi ArabiaSearch for more papers by this author Jiwei Tian, Corresponding Author Jiwei Tian tianjiwei2016@163.com orcid.org/0000-0002-1485-7465 ATC Navigation College, Air Force Engineering University, Xi'an, China Correspondence Jiwei Tian, ATC Navigation College, Air Force Engineering University, No. 1, Changle East Road, Baqiao District, Xi'an City, Shaanxi Province, China 710043. Email: tianjiwei2016@163.comSearch for more papers by this authorBuhong Wang, Buhong Wang Information and Navigation College, Air Force Engineering University, Xi'an, ChinaSearch for more papers by this authorJing Li, Jing Li School of Design and Art, Henan University of Technology, Zhengzhou, ChinaSearch for more papers by this authorCharalambos Konstantinou, Charalambos Konstantinou orcid.org/0000-0002-3825-3930 KAUST, Thuwal, Saudi ArabiaSearch for more papers by this author First published: 21 November 2021 https://doi.org/10.1049/rpg2.12334Citations: 1AboutSectionsPDF ToolsRequest permissionExport citationAdd to favoritesTrack citation ShareShare Give accessShare full text accessShare full-text accessPlease review our Terms and Conditions of Use and check box below to share full-text version of article.I have read and accept the Wiley Online Library Terms and Conditions of UseShareable LinkUse the link below to share a full-text version of this article with your friends and colleagues. Learn more.Copy URL Share a linkShare onFacebookTwitterLinked InRedditWechat Abstract Deep learning has been recently used in safety-critical cyber-physical systems (CPS) such as the smart grid. The security assessment of such learning-based methods within CPS algorithms, however, is still an open problem. Despite existing research on adversarial attacks against deep learning models, only few works are concerned about safety-critical energy CPS, especially the state estimation routine. This paper investigates security issues of neural network based state estimation in the smart grid. Specifically, the problem of adversarial attacks against neural network based state estimation is analysed and an efficient adversarial attack method is proposed. To thwart this attack, two defense methods based on protection and adversarial training, respectively, are proposed further. The experiments demonstrate that the proposed attack method poses a major threat to neural network based state estimation models. In addition, our results present that defense methods can improve the ability of neural network models to defend against such adversarial attacks. 1 INTRODUCTION The smart grid can be seen as the largest internet-of-things (IoT) deployment, since it is based on smart IoT devices and intelligent decision-making algorithms to achieve low-loss, efficient, and environmentally-friendly power control [1-3]. At the same time, the smart grid is also facing increasing threats from cyber-attacks [4-7]. For example, the cyber-attack on Ukraine's power system in 2015 caused hundreds of thousands of electricity customers to lose power. Among the attack threats facing the power grid, cyber-attacks on state estimation have attracted a lot of research attention because estimated variables are direct inputs to critical applications in the other energy management system routines (e.g. contingency analysis, economic dispatch) [8]. Typical cyber-attacks against state estimation include false data injection attacks [9-13], load redistribution attacks [14, 15], data framing attacks [16], and topology attacks [17]. These attacks are all against weighted least squares (WLS)-based state estimation. The optimization-oriented WLS scheme for non-linear (AC) state estimation requires many iterations, and often fails to converge, due to the increasing system size [18]. In addition, it is impractical for real-time estimation [19]. Recently, neural networks (NNs) have been explored to perform various tasks related to state estimation: detection of topological errors [20], generation of pseudo-measurements [21], and real-time state estimation [18, 19, 22]. Particularly, it has been shown that NNs can provide highly accurate state estimation but have much lower computational burden than conventional WLS methods. However, the security implications of utilizing NNs for state estimation are rarely considered. From the perspective of active defense, their security threats should be studied in depth to design more secure models, so that they can be practically applied in actual systems. NNs are extremely vulnerable to a form of attack called ‘adversarial example’ [23-25]. Adversarial examples are inputs that an attacker deliberately crafts to cause the deep learning model to operate as not intended. Although research in the field of adversarial machine learning has been carried out for more than a decade [26], most work has focused on classification problems in the image, speech, and text fields [27-31]. Deep learning is increasingly used in cyber-physical systems (CPS) [32, 33], but its security in CPS is still a new topic [34-36]. In the field of energy CPS, research on machine learning (ML) security has only begun to appear recently [37-39]. The vulnerabilities of current ML algorithms proposed in power systems are first shown in reference[40], and a recent survey is presented in reference [41]. Concerning NN-based classification tasks in power gird, existing works analyse security issues of grid events classification [42], N − 1 security classification [43], power quality signal classification [40], [44], energy theft detection [45, 46], non-intrusive load monitoring [47] and false data injection attack detection [48, 49]. In the area of NN-based regression tasks in power grid, research has mainly been focused on security issues of load forecasting. The potential vulnerabilities in load forecasting algorithms by injected perturbations into input temperatures are studied in reference[50]. A domain-specific framework to evaluate security and resilience of load forecasting algorithms is proposed in reference[51]. The experiments showed that NN-based load predictions may suffer worst-case attacks even when only part of the network is compromised [51]. From a defense perspective, adversarial training [52], and game theory [53] are, respectively, adopted to design resilient load forecasting systems. In addition, poisoning attack (implemented during the training phase) on load forecasting is explored in reference[54]. To our best of knowledge, unlike load forecasting, there is little research on security issues of NN-based state estimation. The first study of adversarial attack against NN-based state estimation is performed in reference[55]. However, the iterative optimization algorithms used to carry out such attacks are likely not suitable for large-scale power systems in the real world, and corresponding defense methods have not been considered at all. To address these problems, this paper considers designing efficient adversarial attack crafting methods and exploring corresponding defense strategies. To our knowledge, this is the first study on adversarial attacks against NN-based state estimation from a defense perspective, which calls for more attention on the research about security of NNs in safety-critical applications. The overall flowchart is presented in Figure 1, and the main contributions of this work are summarized as follows: We propose an efficient adversarial attack crafting method based on forward derivative. The proposed attack crafting method takes into account multiple factors such as the magnitude of the input elements, the impact of the attack on the multiple regression output, and the number of measurement meters allowed to be controlled. In addition, the attack method is efficient so that corresponding defense methods such as adversarial training can be carried out based on this method. We proposed two defense methods to thwart adversarial attacks against state estimation: 1. Utilizing traditional security technologies, the protection-based defense method protects important meters under resource constraints to achieve maximum defense effects. 2. Using techniques in the field of adversarial ML, the adversarial training-based defense method improves the robustness of NN-based state estimation models, thereby reducing the impact of adversarial attacks. Both defense methods can improve the ability of NN models to defend against adversarial attacks to a certain extent. FIGURE 1Open in figure viewerPowerPoint The overall flowchart of the paper The rest of this paper is organized as follows. In Section 2, the NN-based state estimation and proposed adversarial attack crafting method are introduced. The proposed protection and adversarial training-based defense methods are presented in Section 3. In Section 4, the proposed attack and defense methods are evaluated for various case studies. The conclusion and future work are provided in Section 5. 2 ADVERSARIAL ATTACKS AGAINST NEURAL NETWORK BASED STATE ESTIMATION 2.1 Neural network based state estimation Formally, a NN-based state estimation model represents a function F : X → Y , where X is an input vector and Y is a regression output vector. Unlike the discrete outputs of classification problems [56], the outputs of regression problems are continuous. Given certain training samples { ( x i , y i ) } i = 1 N , the above regression function is computed by minimizing the expected loss E P ( R ( F ( x ) − y ) ) . The squared loss function R ( F ( x ) − y ) = ∥ F ( x ) − y ∥ 2 2 is often used to measure the difference between F(x) and y. 2.2 Attack model and objective In our attack model, we propose a threat model as follows: We assume that the attackers have knowledge of the NN-based state estimation model parameters, and use this knowledge to carry out adversarial attacks. In practice, the related knowledge can be obtained either because of insider threats [57], or because attackers can use various methods, such as eavesdropping on network traffic and destroying database systems [58]. We assume that the attackers have some resources to maliciously control some energy-CPS meters. This can be achieved through a variety of cyber-attacks such as man-in-the-middle attack, Trojan-horse attack, firmware modification attacks [59], etc. Although the above assumptions may not always represent the practical cases, they enable us to explore the robustness and vulnerabilities of the NN-based state estimation under the worst-case scenario. In addition, the worst-case scenario can guide, in a more structural way in terms of probable attack impacts, the design of defense strategies. 2.3 Forward derivative-based adversarial attack A forward derivative-based adversarial attack method is proposed for power systems state estimation. The forward derivative is defined as the Jacobian matrix of F: J F X = ∂ F X ∂ X = ∂ F j X ∂ X i i ∈ 1 ⋯ M , j ∈ 1 ⋯ N , (1)where M and N represent the dimensions of the input and output, respectively. The gradient of the forward derivative calculation is similar to the gradient of the backpropagation calculation, but there are two important differences: we directly use the derivative of the network instead of its cost function, and we differentiate in regard to the input vector rather than model parameters. The forward derivative approach is chosen because it allows us to find the input components that cause model outputs to change significantly [56]. To derive the forward derivative in (1), the recursive process is started from the first hidden layer of the network. ∂ L k X ∂ X i = f k , q W k , q · L k − 1 + b k , q X i q ∈ 1 ⋯ m k , (2)where Lk, f k , q represent the output vector and the qth activation function of kth hidden layer ( k ∈ 1 ⋯ n + 1 ), respectively. The connections between the kth layer and the previous layer are represented in W k , q . b k , q is the bias for neuron q of layer k. Based on the chain rule, a series of formulas can be written for k ≥ 2 : ∂ L k X ∂ X i | q ∈ 1 ⋯ m k = W k , q · ∂ L k − 1 ∂ X i × ∂ f k , q ∂ X i W k , q · L k − 1 + b k , q . (3) Then ∂ L n ∂ X i can be expressed and the jth output neuron computes: F j X = f n + 1 , j W n + 1 , j · L n + b n + 1 , j . (4) Thus, J F [ i , j ] ( X ) can be obtained based on the chain rule: ∂ F j X ∂ X i = W n + 1 , j · ∂ L n ∂ X i × ∂ f n + 1 , j ∂ X i W n + 1 , j · L n + b n + 1 , j . (5) Based on the derived forward derivative, we can add some perturbation leading to significant changes in regression outputs. Unlike only one regression output in load forecasting [50-53], there are usually multiple outputs in regression for state estimation. To compare the cumulative effect of each input disturbance on all outputs, we compute the row sum of the absolute matrix of J F ( X ) , J F s ( X ) = ∑ j = 1 N a b s ( J F ( X ) ( : , j ) ) . (6) The M × 1 vector J F s ( X ) can reflect the impact of each input disturbance on the overall output to some extent. Among various adversarial attack methods developed in the image domain so far, the well-known and popular method Fast Gradient Sign Method (FGSM) adds the constant magnitude perturbation to the original sample to maximize the loss function [24]. Unlike the pixels of images in the range of (0,1), the range of the grid measurements used for state estimation is much larger and each meter (value in the input vector) also has a different range of variation. Based on these conditions, it is more reasonable to express the deviation as a ratio rather than a fixed value. Therefore, we compute J F s i ( X ) = J F s ( X ) · | X | , (7)where | X | represents the absolute vector of M × 1 input. Considering the resource constraints of attackers, the number of meters allowed to be perturbed is also limited. The larger the element (meter) in J F s i ( X ) , the greater the impact on overall regression output. Sophisticated attackers will always choose meters with greater influence to carry out adversarial attacks in order to achieve a high impact on the state estimation with low effort. The method proposed for generating adversarial samples is summarized in Algorithm 1. Note that this method is very efficient since the iterative optimization process is not required (in image domain, C&W [60] and PGD [61] are iterative attack methods and time consuming). ALGORITHM 1. Forward derivative-based adversarial attack crafting for NN-based state estimation Input: x: An original input measurement vector; F: The NN-based state estimation model; γ: The ratio of meters attackers can perturb; θ: The scaling ratio made to measurements. Output: x ′ , The adversarial example. 1: Initialization: x ′ ← x . 2: Calculate the forward derivative w.r.t x: J F ( x ) = ∂ F ( x ) ∂ x = [ ∂ F j ( x ) ∂ x i ] . 3: Calculate the metric J F s ( x ) : J F s ( x ) = ∑ j = 1 N a b s ( J F ( x ) ( : , j ) ) . 4: Calculate the metric J F s i ( x ) : J F s i ( x ) = J F s ( x ) · | x | . 5: Find set Υ: the largest γ ratio elements of J F s i ( x ) 6: for i in Υ do 7: x i ′ = x i × θ 8: end for 9: returnThe generated adversarial sample x ′ . 2.4 Random scaling attack In random scaling attack (RSA) [52], the attacker randomly selects a proportion γ of all meters and modifies the selected true measurements via a constant scaling factor θ as in Algorithm 1. The RSA method for generating adversarial samples is given in Algorithm 2. For the above two attack methods, the scaling ratio θ and attack ratio γ will determine the final attack effect. We analyse the related influences in the experiments (Section 4). Note that although the generated adversarial examples by the above two attack methods might raise detection flags within power system operation routines, a smaller number of attack meters and a lower attack ratio can still lead towards implementing stealthy attacks (the threshold-based detection method cannot detect attacks as long as the residual does not exceed the threshold). Furthermore, as mentioned earlier, the proposed attack methods are mainly for exploring vulnerabilities and worst-case scenarios of the model, which can provide guidance for the corresponding defense strategies. ALGORITHM 2. Random scaling attack crafting for NN-based state estimation Input: x: An original input measurement vector; γ: The ratio of meters attackers can perturb; θ: The scaling ratio made to measurements. Output: x ′ , The adversarial example. 1: Initialization: x ′ ← x . 2: Find set Υ:Randomly choose γ ratio of meters 3: for i in Υ do 4: x i ′ = x i × θ 5: end for 6: returnThe generated adversarial sample x ′ . 3 DEFENSE METHODS FOR NEURAL NETWORK BASED STATE ESTIMATION 3.1 Protection-based defense method For false data injection attacks against WLS state estimation, protection strategies based on carefully chosen meters are explored in [62]. Similarly as in the case of attack generation, this implies that different sensors also have widely varying importance in defense measures. This phenomenon still exists in NN-based state estimation (see Section 4 for the details). In fact, in Algorithm 1, the proposed adversarial attack method ranks the importance of the meters. In addition, during our experiments, the significance of some meters remains relatively consistent in different measurement samples. Therefore, we can record the importance ranking of each sensor in different input samples and calculate the order of overall importance of each meter. Based on the overall importance order, we can choose the more vital meters to protect under resource constraints11 In reality, large-scale power grids have a large number of sensors, and defense resources are always limited. Therefore, it is unrealistic to protect all sensors. . The related methodologies and techniques for protecting meters include data encryption, authentication, access control, and auditing. The method for calculating the overall importance order is given in Algorithm 3. Note that although different meters can have a varying degree of importance over time (power systems are dynamic systems), the overall importance order derived by Algorithm 3 is global and comprehensive (based on a set of input measurements vectors). Therefore, the derived importance order reflects the importance of each meter from the overall perspective. ALGORITHM 3. Calculating the overall importance order of meters Input: X: A set of input measurement vectors (number,N). F: The NN-based state estimation model. Output: S, The overall importance order of meters. 1: for x in X do 2: Calculate the forward derivative w.r.t x: J F ( x ) = ∂ F ( x ) ∂ x = [ ∂ F j ( x ) ∂ x i ] . 3: Calculate the metric J F s ( x ) : J F s ( x ) = ∑ j = 1 N a b s ( J F ( x ) ( : , j ) ) . 4: Calculate the metric J F s i ( x ) : J F s i ( x ) = J F s ( x ) · | x | . 5: Sort the elements in J F s i ( x ) and record the order of each elements: J F s i s ( x ) = a r g s o r t ( J F s i ( x ) ) 6: end for 7: Calculate the overall importance order: S = a r g s o r t ( 1 N ∑ j = 1 N J F s i s ( x ) ) . 8: return The overall importance order of meters S. 3.2 Adversarial training-based defense method Adversarial training is used to improve robustness of a NN model to adversarial attacks [24, 63]. In this method, a large number of adversarial examples generated are used to retrain the model. The basic requirement of this method is to use the strongest possible attack crafting algorithm to generate as many adversarial examples as possible. In references[24] and [64], the results show that adversarial training improves robustness of NNs to adversarial attacks. Adversarial training can provide regularization for NNs and improve their classification accuracy [24]. Here, we also adopt adversarial training as our defense method to improve robustness of NN-based state estimation in power systems. Our proposed adversarial training method is summarized in Algorithm 4. ALGORITHM 4. Defending NN-based state estimation using adversarial training Input: D t r : A training dataset of clean samples. N i t e r : The number of iterations used for training. Output: A robust NN-based state estimation model F Θ , where Θ is a set of learnable parameters of the model F. Training the model F Θ using a training dataset D t r : 1: for 1 , 2 , … , N i t e r do 2: Optimize Θ by training F Θ using D t r . 3: end for 4: Generate a set of adversarial samples X ∼ using Algorithm 1. Retraining the model F Θ using the set of samples X ∼ : 5: for 1 , 2 , … , N i t e r do 6: Update Θ by training F Θ using X ∼ . 7: end for 8: returnA robust NN-based state estimation model F Θ . 4 EXPERIMENTAL RESULTS We conducted experiments against NN-based state estimation algorithms proposed in reference[18]. The corresponding dataset, code, and well-trained feed-forward neural networks (FNN) for the IEEE 118-bus benchmark system are publicly available, which makes it easy to evaluate our attack and defense methods. Specifically, real load data from the 2012 Global Energy Forecasting Competition22 https://www.kaggle.com/c/global-energy-forecasting-competition-2012-load-forecasting/data were used to generate the datasets, where the load series were sub-sampled for size reduction by a factor of 2 for the IEEE 118-bus system [18]. Subsequently, the resultant load instances were normalized to match the scale of power demands in the simulated system. The MATPOWER toolbox [65] was used to solve the AC power flow equations to obtain the ground-truth voltages and measurements. The dataset includes 18,528 voltage-measurement pairs, with 14,822 pairs employed for training and 3706 kept for testing. Each voltage-measurement pair consists of 490 dimensional measurement input and 236 dimensional state output. Three trained NN-based state estimation models are: 6-layer FNN, 8-layer FNN and Prox-linear Net, respectively33 The specific model architecture and parameters can be referred to in [18]. Due to space constraints, they are omitted here. . In the experiments, we consider two NN-based state estimation models: 8-layer FNN and Prox-linear Net (in the following parts of the section, we analyse the results of 8-layer FNN first and then analyse the results of Prox-linear Net). The normalized root mean-square error (RMSE) is adopted as the evaluation metric (the RMSE of normal test data of 8-layer FNN and Prox-linear Net are 0.02 and 0.000344 The well-trained 8-layer FNN and Prox-linear Net models are publicly available and we evaluated the RMSEs based on the provided test data in reference [18]. Although the Prox-linear Net is better than the 8-layer FNN, we considered both the models to evaluate the proposed attack and defense methods. , respectively). 4.1 Impact analysis of adversarial attacks The performance of the forward derivative-based adversarial attack (FAA) and RSA are both affected by two factors: attack ratio γ and scaling ratio θ. In order to better study the influence of related factors, we carry out considerable experiments. The chosen percentage of measurement points (total of 490) being compromised γ is listed in Table 1. For each γ, we explore the attack performance under different scaling ratio θ, which is also listed in Table 1. Each experiment runs on all training measurement instances. The main results are summarized in Figure 2. Figure 2a shows that as the attack ratio γ and the scaling ratio θ increase, the average RMSE increases accordingly. This is in line with our intuitive idea that the larger the perturbation, the greater the error in the regression results. However, as pointed out in reference[23], the influence of randomly generated noise (even large) on classification tasks is much smaller than that of the specific perturbation, which can be referred to as an adversarial example. This observation still exists in the NN-based state estimation tasks: under the same attack parameter configuration, the RMSE of RSA (Figure 2b) is much smaller than that of FAA (Figure 2a), implying the RSA has much less impact than the forward derivative-based adversarial attack. Therefore, our proposed FAA method searches meters that have a large impact on the regression results first and then applies perturbations. In addition, from Figure 2a, attackers can cause a non-negligible impact on state estimation results even the attack ratio γ is small (the RMSE of clean data is only 0.02). The similar results of Prox-linear Net are provided in Figure 3, which implies that the state estimation model based on different NN structures is still vulnerable to adversarial attacks. Besides, as shown in Figure 4, most of the RMSEs of 8-layer FNN are bigger than those of Prox-linear Net, which is consistent with the original model performance. TABLE 1. Parameter setting for adversarial attacks Attack number (attack ratio γ) 1( < 1 % ) 2( < 1 % ) 3( < 1 % ) 4(1%) 24(5%) 49(10%) 73(15%) 98(20%) 122(25%) 147(30%) Scaling ratio θ 0.5 0.6 0.7 0.8 0.9 1.1 1.2 1.3 1.4 1.5 FIGURE 2Open in figure viewerPowerPoint RMSE of (a) FAA and (b) RSA for varying values of attack ratio γ and scaling ratio θ (8-layer FNN) FIGURE 3Open in figure viewerPowerPoint RMSE of (a) FAA and (b) RSA for varying values of attack ratio γ and scaling ratio θ (Prox-linear Net) FIGURE 4Open in figure viewerPowerPoint Difference of RMSE between Prox-linear Net and 8-layer FNN for varying values of attack ratio γ and scaling ratio θ (negative values mean that RMSEs of 8-layer FNN are bigger than those of Prox-linear Net) 4.2 Evaluation of protection-based defense The protection-based defense method focuses on the fundamental elimination of the possibility of injecting bad data based on security measures such as encryption, authentication and access control. Due to the limited resources from the defense perspective, power grid utilities cannot protect against all meters. As shown in Figure 5, based on the same scaling ratio, the impact of attacking different meters will be significantly different. At this point, how to choose important meters for defense is crucial. FIGURE 5Open in figure viewerPowerPoint Comparison of attacking two different meters respectively (8-layer FNN, scaling ratio: 1.5) Based on Algorithm 3, defenders can derive the overall importance order of meters S. For 8-layer FNN, the overall importance order of meters S is shown in Figure 6. In Figure 6, the meters are ranked based on importance from 1 to 490. The smaller the meter's sequence value, the more important the sensor is. Besides, we also rank the meters based on magnitude from 1 to 490 and calculate difference between two ranks. In Figure 6, although there is a degree of relationship between importance and magnitude (the curve of ‘Sequence Difference’ fluctuates up and down with 0 as the center), it is not reasonable to consider magnitude only (some values of ‘Sequence Difference’ are large). The Algorithm 3 considering both magnitude and gradient provides a more reasonable way to rank the overall importance order of meters, and choose related meters to protect, accordingly. The similar results of Prox-linear Net are provided in Figure 7. In addition, we compare the importance orders of 8-layer FNN and Prox-linear Net. The result is shown in Figure 8. The overall importance orders of 8-layer FNN and Prox-linear Net are quite different. This indicates that in different NN-based state estimation models, the meters that require priority in terms of protection are likely to be different, even if their selection does not lead to better model performance when there is no attack. The reason may be that models with different structures learn different representations, causing the same meter to have different effects in the model. Therefore, the protection strategy is for a specific NN model. In other words, for the deployed NN-based state estimation model, the specific protection strategy is chosen to achieve the maximum defense effect. FI