A Diversity Index based Scoring Framework for Identifying
Smart Meters Launching Stealthy Data Falsification Attacks
Shameek Bhattacharjee

Praveen Madhavarapu

Sajal K. Das

Western Michigan University
Kalamazoo, MI, USA
shameek.bhattacharjee@wmich.edu

Missouri Univ. of Sc. & Tech.
Rolla, MO, USA
vmcx3@umsystem.edu

Missouri Univ. of Sc. & Tech.
Rolla, MO, USA
sdas@mst.edu

ABSTRACT
A challenging problem in Advanced Metering Infrastructure (AMI)of smart grids is the identification of smart meters under the control
of a stealthy adversary, that inject very low margins of stealthy data
falsification. The problem is challenging due to wide legitimate variation in both individual and aggregate trends in real world power
consumption data, making such stealthy attacks unrecognizableby existing approaches. In this paper, via proposed modified diversity index scoring metric, we propose a novel information-theory
inspired data driven device anomaly classification framework to
identify compromised meters launching low margins of stealthy
data falsification attacks. Specifically, we draw a parallelism between the effects of data falsification attacks and ecological balance
disruptions and identify required mathematical modifications in existing Renyi Entropy and Hill’s Diversity Entropy measures. These
modifications such as expected self-similarity with weighted abundance shifts across various temporal scales, and diversity order areappropriately embedded in our resulting framework. The resulting
diversity index score is used to classify smart meters launching additive, deductive, and alternating switching attack types with highsensitivity (as low as 100W) compared to the existing works that
perform poorly at margins of false data below 400W. Our proposed
theory is validated with two different real smart meter datasetsfrom USA and Ireland. Experimental results demonstrate successful
detection sensitivity from very low to high margins of false data,thus reducing undetectable strategy space of attacks in AMI for anadversary having complete knowledge of our method.

ACM Reference Format:
Shameek Bhattacharjee, Praveen Madhavarapu, and Sajal K. Das. 2021. A
Diversity Index based Scoring Framework for Identifying Smart Meters
Launching Stealthy Data Falsification Attacks. In Proceedings of the 2021
ACM Asia Conference on Computer and Communications Security (ASIA
CCS ’21), June 7–11, 2021, Hong Kong, Hong Kong. ACM, , USA, 14 pages.
https://doi.org/10.1145/3433210.3437527

1

INTRODUCTION

As illustrated in Figure 1, the Advanced Metering Infrastructure(AMI) in a smart grid is composed of a collection of smart meters(end-point IoT devices) that collect power/energy consumption datafrom customers [19]. The smart meters periodically send data via ahierarchical communication network to various data management
servers belonging to the utility company. These servers processdata from smart meters for critical operations such as automatedbilling, load forecast, daily and critical peak shifts, and automateddemand response [32]. Therefore, the integrity of the data fromindividual smart meters is pivotal to the success of the smart grid.

CCS CONCEPTS• Computing methodologies → Machine learning; Artificial
intelligence; • Security and privacy → Trust frameworks; Intrusion detection systems; • Hardware → Smart grid;

KEYWORDS
Anomaly Detection; Interpretable ML based Security; Explainable
AI based Security; Information Theory; Data Falsification Attacks;
Smart Meters; IoT security; Bio-Inspired ML Approaches
Permission to make digital or hard copies of all or part of this work for personal orclassroom use is granted without fee provided that copies are not made or distributedfor profit or commercial advantage and that copies bear this notice and the full citationon the first page. Copyrights for components of this work owned by others than theauthor(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or
republish, to post on servers or to redistribute to lists, requires prior specific permission
and/or a fee. Request permissions from permissions@acm.org.
ASIA CCS ’21, June 7–11, 2021, Hong Kong, Hong Kong
© 2021 Copyright held by the owner/author(s). Publication rights licensed to Association for Computing Machinery.
ACM ISBN 978-1-4503-8287-8/21/06. . . $15.00
https://doi.org/10.1145/3433210.3437527

Figure 1: AMI Network Architecture
However, orchestrated cyber or physical attacks on smart meterdata are getting increasingly likely as the smart meters are being connected to web-based Energy Management portals [32] andsmart home networks at the customer’s end [13]. An orchestrated
cyber attack can compromise several smart meters connected tothe same feeder and then spoof false power consumption readings.
Real evidence of orchestrated large scale physical attacks on smart
meters was reported in Puerto Rico [33, 34] where several hundreds of smart meters were tampered via an optical probe toolkit
by collusion of customers and utility insiders, thus false power
consumption data inflicted huge losses to the concerned utility.

1.1

Motivation and Key Challenges

Identification of those smart meters involved in the injection offalse power consumption data is a key security challenge. The parameter quantifying the extent of falsification from original dataper meter device is termed as ‘average margin of attack strength’.
Lower margins of attack strength are stealthier and hence harder todetect. Additionally, the parameter that quantifies the total percentage of such compromised smart meters in a micro-grid is termed
as ‘attack scale’. Orchestrated and coordinated attacks often havelarger attack scales compared to the isolated attacks. Moreover, asmart adversary can find a cheaper exploit to compromise smart
meters, thereby allowing even a lower margin attack to have a significant impact on the utility when compared to adversary’s total
attack cost. Orchestrated attacks are usually launched by organized
and stealthy adversaries (business competitors, organized cybercriminals). They expect to lower the margins of data falsification
per meter by hiding behind the randomness of smart meter data,such that meters are not easily caught. Rival nation states may bemotivated to launch organized attacks, since meter data dictate thegeneration and distribution of electricity to critical infrastructures.
Our analysis of existing works in smart meter data falsification found that most methods fail to identify meters when theattack margins are below 400W, regardless of the attack types andstrategies (elaborated in the threat model section). Additionally,
the possibility of physical attacks causing data falsification ( optical laser attacks [34] and acoustic transduction attacks [5, 6])render cryptography based and network traffic based intrusion detection methods on IoT devices inadequate. Cybersecurity practices
such as static analysis, and signed software updates do not protect against a sensor recording false data since the physical attacks
influence the output of sensor hardware that is trusted by software/firmware [6, 23]. Furthermore, several studies [31, 41] havenoted that embedded/hardware/in-situ security of smart metersthat provide some protection against physical attacks, is not costeffective due to the large scale nature of meter deployment andvariety in commodity hardware. Therefore, providing a device level
data driven behavioral anomaly scoring technique is necessary not
only as an extra level of security, but as a principle approach fortrusting the data from a distributed set of IoT devices (e.g. smart
meters), which motivates our approach.
However, non-typical but benign conditions can cause changes
in data due to weather conditions, and seasons; and low margin
data falsification attacks can easily hide behind such randomness.
Our anomaly based detection approach should distinguish between
such events and changes that are caused due to attacks. Finally, themethod has to generalize across various attacks and datasets.

1.2

Contributions of This WorkIn this paper, we propose a novel information-theoretic anomaly
scoring framework, called Modified Diversity Index Scoring, that captures smart meters launching additive, deductive, and alternating
switching attack types across a wide range of very low to very highmargins of attack strengths and attack scales, while also lowering
false alarms and missed detection, compared to existing approaches,for various stealthy attack strategies.
Specifically, we first establish an analogy between the intelligent data falsification attacks in smart meters and the monitoringof ecological balance of species distributions in a geographical region. Next, we show that information-theoretic approaches, such
as Renyi and Tsallis Entropies (popular in ecology), Shannon’s
Entropy and Kullback-Leibler Divergence common in computer

security; are not sufficient to address this problem. Thereafter, bystudying the effects of various attack types on the probability ofrelative abundance of each discretized space of the random variableof power consumption, we identify the need for modifications to
the existing information theoretic measures. To this end, we introduce modifications to the concept of Renyi Entropy and Hill’sDiversity Entropy by embedding a notion of a weighted expected
self-similarity mapping of a smart meter IoT device across multiple
temporal scales. Next, we embed an appropriate order of the entropy
and a weighted relative abundance vector to capture subtle drifts
in the horizontal, vertical and incline directions in the probability
space, thereby resulting in a diversity index score. The higher thediversity index score, the more likely is the meter launching data
falsification attacks. Thereafter, we offer a supervised approach to
the learn the parameters of our proposed model that maximizes
the separation of diversity index scores between the set of labeledcompromised and honest meters, accompanied by cross-validation.
We validate the proposed framework with multiple full year real
datasets, demonstrating its generalization across a wide range ofattack strengths, scales, types, and strategies, across seasons. Experimental results show that our method exhibits lower false alarms
and missed detection even when the average attack strengths permeter lower than 400𝑊 (which causes evasion in previous defenses)
for both Texas Dataset (200 meters) and Irish Dataset (1300 meters).
Specifically, we show that model generalizes to successfully detect
deductive, alternating switching attacks and strategies that were
not used to train the model. A comparison with existing works
exhibits improved performance in terms of reduced undetectable attack strategy space when the attacker has knowledge of our method.
We also provide a tradeoff between impact of missed attacks versus
cost of base rate false alarms (when there no attacks in the test set).
The paper is organized as follows: Sec. 3 and Sec. 4 describe
the system and threat models along with the datasets used. Sec. 5
presents the proposed method, while Sec. 6 discusses the optimal
choice of parameters and threshold choice. Section 7 describes
experimental results and the final section offers conclusions.

2

RELATED WORKS

Existing approaches for detecting smart meters launching data falsification attack can be broadly divided into three categories, (i) classical machine learning, (ii) information-theoretic, (iii) consensusbased statistical approaches.
The classical machine learning methods use SVMs [10], decision/regression trees (DRT) [9], and neural networks [4, 12]. In [10],the problem was investigated using SVM with a radial basis network
when 𝛿𝑎𝑣𝑔 = 450𝑊 , but the percentage of compromised meters assumed was just 1%. In the DRT method [9], the approach does notparameterize the attack strategy space fully, and the attack strengthand scales are unclear. Surveyed by[8], the false alarm rates reported by most neural network based methods are much higher
even as they do not generalize for an unbounded attack strategy
spaces and low profile attacks.
Information-theoretic approaches proposed in [1, 14] use
Kullback-Leibler divergence to classify compromised meters, and is
a competing approach to our model. Hence, we will show how ourattacks perform under these defenses and our solution’s detection
performance compare with these approaches.

Consensus-based approaches used classical statistics [25, 26],
time series [18], robust statistics and density-based learning [3] toidentify such smart meters. We chose to compare with [3], since it
outperforms the others, and parameterizes the attack strategy spacewith attack scales and strengths. State estimation based methods are
not used since they depend on putting extra monitoring hardwarein the higher layers of a smart grid. Note that [2] applies to stealthy
attacks at a micro-grid level and but not at the meter level, thus it
does not feature in our comparisons. The comparison of our methodwith existing research is provided in Sec 7.4.

3

SYSTEM MODEL AND DATASETS

Here we present the AMI architecture as a proof of concept for an
IoT network, the real AMI datasets used, and the rationale behind
their choice for validation of our proposed framework.
AMI Architecture: Let us consider a typical AMI micro-grid,where each house is equipped with a smart meter that records
aggregate power consumption data (from all appliances inside thehome) and periodically (e,g., hourly) sends them to the utility company over the AMI communication network. The AMI network
typically consists of a Neighborhood Area Network (NaN) Gateway that aggregates data from multiple smart meters. Data frommultiple NaN gateways are aggregated by a Field Area Network(FAN) gateway and sent to the utility wide area network. The FAN
gateway may also host edge computing capabilities.
Let 𝑁 be the total number of smart meters in a micro-grid, suchthat 𝑖 ∈ {1, 𝑁 } uniquely identifies a smart meter ID. Formally, theactual power consumption of a 𝑖-th house at the 𝑡-th ‘time slot’𝑖 (𝑡), where 𝑡 is slotted ‘hourly’. If a meter is not
is denoted by 𝑃𝑎𝑐𝑡compromised, then the actual power consumption is equal to theadvertised power consumption 𝑃𝑡𝑖 sent to the utility.
Choice of Datasets: We have used two real AMI datasets to validate the proposed framework. The first dataset is Ireland Social
Sciences Data Archives [30] containing 5000 meters from six regions in the city of Dublin, Ireland, collected between 2009-2010.
Three out of these six regions, have more than 1000 smart meters.
The rationale for choosing this dataset is to investigate the scalability of our framework for large micro-grids. The second dataset
is Pecan Street Project [29] containing hourly power consumption
data from 215 houses from a Solar village in Texas, USA, collected
between 2014-2016. Hence, we have chosen two datasets that are inherently different in terms of their geography, climate, randomness,and extreme difference in sizes.

4

THREAT MODEL

We assume an organized adversary that orchestrates data falsification attacks from multiple smart meters via cyber or physicalexploit [2]. Smart meters receive power consumption from various
appliances via the Home Area Network (HAN) and sends it to theutility side via the Neighbourhood Area Network. Either the (i) input to the smart meter, (ii) the power consumption data at rest inside
the smart meter, (iii) or data in flight may be falsified. An example
of falsifying power consumption data at rest is the Puerto-Rico Grid
Attack of 2012, where hundreds of smart meter’s optical ports weremanipulated using laser probes by utility insiders [33, 34], causing
the smart meters to record lower than actual power consumption.

Similarly, load altering attacks reported in [20], have shown thepossibility to change the inputs from appliance loads to the smart
meter. Similarly, the data in flight from multiple smart meters to
the NaN gateway may be falsified by a traditional man-in-the middle attack. Finally, another possibility is an organized adversarythat controls a set of smart meters like a Botnet, collect data fromintercepted smart meters, and inject advanced data falsification
strategies, that we discuss under the stealthy attack strategies.
Our approach is agnostic of the exploit used to falsify the data.
Of course, depending on the exploit the attack scales, strengths,
and strategies will vary. Our intention is to capture various kinds ofdata falsification attack realizations instead of a specific one, since
exploits tend to evolve over time and just because an attack has notbeen realized before, does not mean they will not be experienced infuture. We capture this generic data falsification attack landscapeby parameterizing the attack strategy space; taking into accountthe full range attack scales, strengths, strategy combinations in thissection. The following features characterize our threat model:
Attack Scale: The fraction of compromised meters, 𝜌𝑚𝑎𝑙 = 𝑀
𝑁,
is the attack scale, where 𝑀 is the number of unique smart meters compromised by an organized adversary in a given network.
Traditional use of Kullback-Leibler Divergence (KLD) model withstatistical aggregates work well, if 𝜌𝑚𝑎𝑙 % [1, 21] are smaller. However, resilience against higher 𝜌𝑚𝑎𝑙 has been reported only whenassociated margins of false data per meter is too high (which facilitates easier detection). However, if the attack budget is high, or acreative adversary finds a cheaper exploit to compromise a meter,or the network size is smaller, then the attack budget constraint
does not automatically imply a lower fraction of compromised meters [1]. This is because, in reality, the value of 𝑀 depends also on
the creativity of its exploit, and the micro-grid size 𝑁 . Given large
values of 𝜌𝑚𝑎𝑙 are possible in the real world, we take into accounta wide variation of 𝜌𝑚𝑎𝑙 between 0.10 to 0.90.
Average Margin of Attack Strength: Average margin of false
data is the average extent of falsification introduced per meter.
We observed that in most previous works, the average margin offalse data is not parameterized as a variable except in two recent
works [1, 3], which report that these methods completely fail todetect meters when their average margin of false data is 𝛿𝑎𝑣𝑔 < 400.
This happens because the standard deviation of data streams arehigh (430W-480W in AMI applications) due to randomness of human activity, making it difficult for previous methods to achieve
success. We have included a real case study in Appendix A, showingthat attack strengths as low as 100W per meter create a significant
attack impact on the AMI utility. In our model, we consider an unbounded 𝛿𝑎𝑣𝑔 value to show that our method reduces undetectablestrategy space of attack strength.
Attack Types: We consider three different attack types. The ad𝑖 (𝑡) representing
versary seeks to falsify original data points 𝑃𝑎𝑐𝑡actual energy consumption at time 𝑡 by some factor 𝛿𝑡 , where𝛿𝑡 ∈ [𝛿𝑚𝑖𝑛 , 𝛿𝑚𝑎𝑥 ] and the long term average value of 𝛿𝑡 is 𝛿𝑎𝑣𝑔
(avg. margin of attack strength).
(i) Additive Attacks: Here the smart meters seek to increase the data𝑖 (𝑡) + 𝛿 . Motivationfrom its original values, such that 𝑃 𝑖 (𝑡) = 𝑃𝑎𝑐𝑡𝑡of such attacks are discussed in [20].

4000

0.14
0.12

3000

Probability

Stealthy Attack Distribution Strategies: Now we focus on‘how’ false data is introduced in the smart meters data streams.
Apart from a non-stealthy random bias, we analyze our solution
against four stealthy strategies, viz. (i) the data order aware, (ii)incremental ramp, (iii) KLD minimization (iv) persistent strategies.
AMI applications are not real time systems; they can tolerate somedelay. Therefore, if there is some timing delay due to coordination
for the stealthy strategies, it is still practical. We assume that areasonably organized attacker will have an idea of the data distributions and mechanisms used by usual anomaly detectors, and craft
the following strategies accordingly:(i) Data Order Aware Strategy: It is a stealthy falsification strategy
that minimizes the chance of detection against mechanisms utilizing
proximity (e.g., Euclidean 𝐿2 distance) between the reported andoriginal data distribution, while keeping the same 𝛿𝑎𝑣𝑔 . Additionally,
this strategy makes sure that the maximum and minimum values
in the original and falsified distribution are not different, to prevent
obvious statistical outliers.
The following strategy is implemented in the following manner: At any time slot 𝑡, the adversary sorts the actual recorded
data vector from its compromised set of devices such that
(1)
(𝑚)
(𝑀)
𝑃𝑡 (𝑎𝑐𝑡) ≤, · · · , 𝑃𝑡 (𝑎𝑐𝑡), ≤ 𝑃𝑡 (𝑎𝑐𝑡); as well as its corre1
sponding bias vector 𝛿𝑡 (𝑚𝑖𝑛) ≤, · · · , ≤ 𝛿𝑡𝑀 (𝑚𝑎𝑥). Under an additive attack, the minimum actual data is changed with the highest 𝛿𝑡 (𝑚𝑎𝑥), while the maximum observed data is modified withlowest 𝛿𝑡 (𝑚𝑖𝑛), and so on like an inverse matching, such that
(1)(𝑀)
𝑃𝑡 (𝑎𝑐𝑡) + 𝛿𝑡 (𝑚𝑎𝑥), · · · , 𝑃𝑡 (𝑎𝑐𝑡) + 𝛿𝑡 (𝑚𝑖𝑛), subject to the factthat it does not violate bounds on the historical distribution. For
a deductive attack, the maximum recorded data is modified bymatching with the maximum bias 𝛿𝑡 (𝑚𝑎𝑥), while the lowest actual
recorded data is altered with the lowest 𝛿𝑡 (𝑚𝑖𝑛). For alternating
switching attack, the additive and deductive attacks alternate withthe strategy mentioned above.
In Fig. 2(a), the blue line corresponds to the non-attacked valueof compromised meters. The yellow and red lines correspond to arealization of falsified data under a data order aware and non-dataorder aware strategy with same 𝛿𝑎𝑣𝑔 =200W and 𝜌𝑚𝑎𝑙 = 40% for‘deductive’ attacks from Texas dataset. The same revenue impact isachieved with both strategies, but chances of detection (using proximity/distance/similarity) are smaller in data order aware strategy.
The width of the interval of 𝛿𝑡 ∈ [𝛿𝑚𝑖𝑛 , 𝛿𝑚𝑎𝑥 ] is known as theaperture of attack. The aperture is varied as necessary to minimize
the euclidean and KLD.
(ii) Incremental/ Ramp / Boil-frog Strategy: This strategy involves avery gradual increase in of 𝛿𝑡 bias over time, until intended 𝛿𝑎𝑣𝑔
is reached. This attack strategy is termed as boil-frog in AI security and ramp attack in cyber physical system (CPS) security. The
strategy causes all temporal metrics to record minimal changes thatevolve over time to bypass detection.
(iii) KLD Minimizing Strategy: The falsified data is injected in amanner which minimizes the KLD, while preserving the target
𝛿𝑎𝑣𝑔 . Fig. 2(b) shows an illustration for a single meter where theadversary crafts a distribution (bold red line) that minimizes the
KLD; thus being closer to the actual data distribution (blue line)
than to a uniform random bias attack (gray line), even when the𝛿𝑎𝑣𝑔 = 200 for both attack strategies.
(iv) Persistent Strategies: In Section 7.5, we provide a list of strategies launched by an adversary that knows our defense model andlaunches evasion attacks. We show performance under such evasion attacks, my showing the extent to which undetectable strategy
space is reduced, and break even time of adversary.
Meter Reading (W)(ii) Deductive Attacks: Smart meters seek to decrease the data from𝑖 (𝑡) − 𝛿 ; this is equivalent
its original values, such that 𝑃 𝑖 (𝑡) = 𝑃𝑎𝑐𝑡𝑡to electric theft and the most commonly seen attack type [34].
(iii) Alternating Switching: In such an attack, every compromised
meter alternates between launching additive and deductive attacks
with the same margin of false data at different times of the day totake advantage of dynamic pricing/demand response of electricity.
When the prices are high (due to higher demand), it launches a deductive attack, while compensating with an equal margin additive
attack when the pricing is low (due to lower demand), causing themean consumption trends from individual compromised meters
practically unchanged. This is device level equivalent to a camouflage attack reported in [2] from two sets of meters in the sametime, thus blinding a micro-grid level anomaly detector. However,our variation of camouflage attack is launched from the same end
point meter to camouflage the end device (meter) level detectors.

NO AttackNon Data Order Aware Attack
Data Order Aware Attack

2000

No AttackAttack withb Minimum KLD Strategy
Attack with uniform distribution

0.1
0.08
0.06
0.04

1000

0.02
0

0
0

10

20

30

40

50

60

70

80

0

500

Smart Meter ID

(a)

1000

1500

2000

2500

3000

Smart Meter Readings(b)

Figure 2: (a) Data Order Aware (b) KLD Minimizing
Attacker Knowledge and Assumptions: We are aware that adversaries will have full knowledge of our defense mechanism whenpublished. In Section 7.5, we showed the performance of our method
in terms of reduction in undetectable attack strategy, assuming theadversary has knowledge of historical data and our method.
Our base assumption is that the defender has a significant portionof training set that is not attacked. As for this paper, we have only
focused on evasion attacks that occur in the testing phase andignored possibility of poisoning attacks. This is because reducing
the undetectable attack strategy space itself is quite a challenging
problem. Active poisoning attacks will be part of our future work.
Finally, since CPS application security is still a new field, datasets
containing real attacks are unavailable. Therefore, we have parameterized the entire attack strategy space in terms of attack strength,
scale, strategies, and types, and reported the failure bounds. Thisshould alleviate concerns over non-availability of real attacks.

5

PROPOSED FRAMEWORKThe technical contribution of the paper is divided into three parts:(i) Theoretical Intuition ((Sec 5.1 to 5.4)): which describes a highlevel intuition of the proposed theory; (ii) Proposed Methodology(Sec 5.5 to 5.9): which describes our proposed information-theoretic
framework for the trust scoring; (iii) Parameter Optimization andTheoretical Intuition

In this subsection, we provide a theoretical intuition of why ourframework is required and at a high level why it works.
Discretizing Data Distributions: The range of dataset fromthe smallest (𝐶𝑚𝑖𝑛 ) to the highest recorded value (𝐶𝑚𝑎𝑥 ) is gathered over all devices. The continuous range [𝐶𝑚𝑖𝑛 , 𝐶𝑚𝑎𝑥 ] of reported data is divided into 𝑅 number of discrete partitions of equal
width termed as species width (denoted by 𝑠𝑤 watts). Each of these
discrete partitions is individually visualized as distinct outcome
categories known as ‘species’ or bins. The terms outcome category
and species ID are used interchangeably.
Each outcome category is uniquely identified by a species ID
𝑠 ∈ {1, 𝑅}. The formation of species categories and corresponding
species width 𝑠𝑤 is common for all devices, whose value needs tobe optimized. For AMI, if 𝑠𝑤 = 100𝑊 , then 0W-100W forms thefirst species (𝑠 = 1), 101W-200W forms the second species and soon. The total number of species 𝑅, depends on the species width
𝑠𝑤 and the data range. The optimal species width is a parameterthat affects classification will be discussed later.
Probability of Relative Abundance of Species: We visualize the species as a random variable associated with a probability.
This probability simply indicates the probability of reported data
belonging to one of these species categories, and is termed as the‘probability of abundance’ 𝑝𝑠𝑖 of each species. Mathematically, we
use the Bayesian interpretation of posterior relative abundance tocalculate each of the 𝑝𝑠𝑖 :𝑝𝑠𝑖 =𝜂 (𝑠)𝑖 + 1
Í𝑅
𝑅 + 𝑠=1
𝜂 (𝑠)𝑖(1)

is the probability of abundance of the s-th species in 𝑖-th meter,where 𝜂 (𝑠) is the total frequency of datapoints observed in that 𝑆,
Í𝑅 𝑖
and 𝑠=1
𝑝𝑠 = 1. Mathematically, for a given smart meter 𝑖, theprobability of relative abundance of species is a column vector isdenoted as 𝒑, such that 𝒑𝑇 is the corresponding row vector 𝒑 𝑻 ={𝑝 𝑖1, · · · , 𝑝𝑠𝑖 , · · · , 𝑝𝑅𝑖 } collectively denote the relative abundances ofall species for a given meter. From now on, since the following
method applies to any meter 𝑖, we will drop the suffix 𝑖 from ournotations for simplicity.

where, 𝑅 is the number of bins in the discretized data distribution
and 𝑠 denotes any such bin, 𝑝𝑠 is the probability of observing a
datapoint in the s-th bin, 𝑞𝑠 denotes the same observed in a different
time or space. Both [1, 14] use various bin partitions and then use
KLD between the two distributions before and after an attack tofind compromised meters.
In this paper, for the context of small margins of false data insmart metering data, we verify how KLD based methods, that has
its roots in Shannon entropy perform. Shannon Entropy views
each species as distinct without relationship among them and also
treats each species as contributing equally to the overall score.
However, in reality, such measures cannot track subtle changes inthem. Fig. 3(a), shows the result where we partition the data into
several small partitions and take the Kullback-Leibler Divergenceof the distributions over all the partitions before and after the attack
for a margin of false data of 200W. This resulted in scores that are
not linearly separable as shown in Fig. 3(a).
6Compromised Meter
Honest Meter

0.6

Hill Diversity Index

5.1

Weaknesses of KLD Approach: Information theory plays a central role in most learning and classification methods for security.
However, in this section, we show that the typical informationtheoretic approaches, do not work effectively in classifying devices
that launch data falsification due to the high variance, shifting
trends, low margins of attack, data order aware strategy for all
attack types.
The 𝐻 𝑖 and the 𝐾𝐿𝐷 𝑖 denote the Shannon entropy and KullbackLeibler Divergence of a smart meter 𝑖:
!
𝑅
𝑅
Õ
Õ
𝑞𝑠
𝐻 =−
𝑝𝑠 𝑙𝑜𝑔(𝑝𝑠 );𝐾𝐿𝐷 = −
𝑝𝑠 𝑙𝑜𝑔
(2)
𝑝𝑠
𝑠=1
𝑠=1

KL Distance

Trade-offs (Sec 6): describes the trade-offs associated with choosing
various optimal parameters in the proposed methodology.
We first discretize the power consumption random variable into
bins/categories/species, find the relative abundances, and introduce
the notion of Renyi Entropy and Hill’s diversity index. We discuss
why KLD fails to detect attacks in our threat model. Then, weshow that Hill’s index is not enough, but a promising starting point
for solving our problem. Subsequently, we show how very low tohigh strength margins of attacks of various attack types, introduce
different kinds of subtle shifts in the distribution of species andidentify the ‘factors’ that need to be tracked/added into the existing
Diversity and Renyi entropy. Then, using a bottom-up approach
we propose the trust scoring classification framework that embeds
those factors such that produce linearly separable scores, that can
indicate smart meters whose data is falsified. Finally, we discuss thelearning of parameters from data which maximizes the separation
between scores of compromised and honest labeled meters.

0.4

0.2

0

0

50

100

150

200

Compromised Meter
Honest Meter

5
4
3
2
1
0

0

50

100

Smart Meter ID

Smart Meter ID

(a)

(b)

150

200

Figure 3: Failure to Classify (𝛿𝑎𝑣𝑔 = 200W, Texas Data, Additive): (a) KLD (b) Hill’s Diversity Index
5.1.1 Renyi Entropy and Hill’s Diversity Entropy. The Renyi entropy introduces the notion of ‘order’ 𝑞 in the entropy measure.
The ‘order’ mathematically allows one to embed the importance of
certain rare or abundant outcome categories (or species) in the 𝒑.
As a special case, the Renyi entropy of 𝑞 → 1 converges to Shannon
entropy. The Renyi entropy is mathematically defined as:
𝑅
Õ

1
𝑞
𝐻𝑞 (𝒑) =
𝑙𝑛
𝑝 (𝑠)
(3)
1−𝑞
𝑠=1
The exponential of the Renyi entropy of the q-th order is also
known as the Hill’s Diversity index of the order 𝑞 ∈ I for such a
community and is given by:
𝑅
Õ
 1
1−𝑞
𝑞
𝑒 𝐻𝑞 = 𝐷 (𝑞) =
𝑝 (𝑠)
(4)
𝑠=1

where 𝑞 is known as ‘order of diversity’, which controls the sensitivity of the diversity index measure to the most common or therarest species. A diversity index of 𝑞 = 0 is completely insensitive
to species frequencies and treats all species equally. All species
𝑞 < 1 favors disproportionately the rarer species. All species 𝑞 > 1
disproportionately favors the most common species. If 𝑞 = 1, thediversity index proportionally favors species relative abundances.
While this offers more math provisions for capturing subtle changes,
the Fig. 3(b), shows no visible difference still in the attacked andnon-attacked meters for 𝛿𝑎𝑣𝑔 = 200. Now let us discuss the effectof each attack on the species distribution.
5.1.2 Effect of Data Falsification: A Diversity Parallelism. Now
we provide intuition on the effect over the dynamics of species
abundance, under data falsification, with a low 𝛿𝑎𝑣𝑔 and data orderaware strategies. At the same time, the effect of intermediate andhigh 𝛿𝑎𝑣𝑔 cannot be ignored, given that the defender has no ideaon the value of 𝛿𝑎𝑣𝑔 in the real world. We show how additive,
deductive, switching attacks, introduce subtle shifts in various
directions. We assign some nomenclature for these shifts, whichneeds to be accounted for in the trust classification method forsuccessful identification.

Probability

0.25

No AttackAfter Attack

0.2
0.15
0.1
0.05
0

0

5

10

15

20

25

30

Species ID

(a)
0.25

No AttackAfter Attack

Probability

0.2
0.15
0.1

decrease in the rarity of species. Any further increase in 𝛿𝑎𝑣𝑔 isgoing to make the distribution, less concentrated.
Additive Attacks: Fig. 4(b), shows an illustration for an additive
attack with 𝛿𝑎𝑣𝑔 = 120𝑊 . The blue and pink bars correspond to𝒑 before and after attacks. Here, the rarer species 1-10 almost disappear, while the most of the abundant species (11-16) become
even more abundant. Between species ids 20 − 30 (which are rarer),
there is a no vertical increase in probabilities, barring a few witha slight increase. This gives the opposite effect compared to the
deductive attack. The datapoints after attack become more concentrated into a smaller number of species categories because of anincrease in the most abundant species, while decreases are seen
in rarer species, which needs to be captured. However, when 𝛿𝑎𝑣𝑔
is just large enough, this trend is reversed, and the distribution
becomes less concentrated again (though in the opposite direction),and there is a small vertical change in each species, but spannedover a large number of rarer species (horizontal shift). This effectis shown in Fig. 5(b), for an additive attack of 𝛿𝑎𝑣𝑔 = 240𝑊 thatcauses small increases in the probabilities across many rare species,
which needs to be captured.
Alternating Switching Attacks: Fig. 5(a), is the trickiest attack
type and shows two important but minor shifts in the distributions
before and after an alternating switching attack of 𝛿𝑎𝑣𝑔 = 180𝑊 .
The shift is in the intermediately rare species on either side of abundant species (resultant incline shift). In contrast, if there is an additive attack of high magnitude then the relative abundance of
higher species will increase and lower species will decrease. However, the net effect in the change of relative abundance may cancel
out. Therefore, it’s true that relative abundance that measures thevertical heights of the species change is not enough. We also needa measure that captures the horizontal shifts and incline shifts.

0.05

0.4

0.2
5

10

15

20

25

30

35

Species ID(b)

Figure 4: Effects of Attacks on Species Distributions: (a) Deductive (b) Additive
Deductive Attacks: Fig. 4(a), shows an illustrative example of adeductive attack with a 𝛿𝑎𝑣𝑔 = 140𝑊 . The ‘blue bars’ denote theprobabilities of relative abundance ‘before the attack’. The higher
the blue bars, the least rare (or most abundant) the species, while
the species with smaller blue bars, are more rare (or less abundant).
The ‘pink bars’ in Fig. 4(a), denote the new probabilities of relative
abundance of each species ‘after the attack’. A comparison of blue
and pink bars shows that there is a minimal horizontal shift in thespecies distribution. Certain species such as species IDs 1,2,3,4 thatare intermediately rare before the attack become more abundant
after the attack. However, this increase in abundance is contributedby a mixture of both abundant and rare species categories between
5 and 20, which experience a slight decrease in their probabilities ofabundance (vertical changes). Hence, we conclude that change inrarer species is significant and can be visualized as an incline shift(change in intermediately rare species) which includes a mixture ofhorizontal (increased quantity diff aka number) and vertical shifts(increased quality difference aka probability) in the rarer species.
The concentration of the resultant distribution is less, due to a

No AttackAfter Attack

0.15

Probability

0

Probability

0

0.1
0.05
0

0.3
0.2
0.1
0

0

5

10

15

20

25

30

35

40

No AttackAfter Attack

0

5

10

15

20

Species ID

Species ID

(a)

(b)

25

30

35

Figure 5: Effects of Attacks on Species Distributions: (a) Alternate Switching Attack (b) Large 𝛿𝑎𝑣𝑔 Additive
5.1.3 Summary of Conclusions:. From the above study, we
make the following conclusions: (i) Small 𝛿𝑎𝑣𝑔 causes changes ina small number of intermediately rarer species. (ii) Higher 𝛿𝑎𝑣𝑔
causes small changes in a large number of rarer species. (iii) The
vertical shift in terms of the change in probabilities of abundanceis the third thing that needs to be kept track of (introduced later as
expected self-similarity).
From the above, it is clear that the number of rarer species experiencing small changes is a critical factor that needs to be embeddedin the scoring framework. This is loosely referred to as ‘quantity’ ofrare species since the adversary can craft an attack introducing verysmall vertical changes across all/large numbers of species. Additionally, the rareness of a particular species itself is important, because,for very low 𝛿𝑎𝑣𝑔 , the intermediately rare species are likely to get
more effected than the rarest of species. This is loosely referred toas ’quality’ of the change in species.
For lower 𝛿𝑎𝑣𝑔 , the horizontal shifts are less pronounced but
vertical and incline shifts (intermediate rare species) are most important and vice-versa. The quantity of rarer species and quality
depicting the extent of change both needs to be embedded, which
introduces a notion of dual weights that needs to be embedded inthis existing diversity index measures.
5.1.4 Functional Form of Modified Hill’s Index:. Following theabove conclusions, we introduce modifications into existing Hill’sDiversity Index measure in a way that embeds new variables
that can track changes in the number of rarer species (horizontal changes), the change in the individual species abundance withitself (vertical changes), in a way that gives magnified importanceto those changes observed in intermediately rare species (incline
shifts) as compared to other species. All three factors ensure thatthe resultant modified diversity score (say 𝑟𝐷) can separate the compromised meters from honest ones. Mathematically, the original
equation of Hills Diversity is given by the following:

 1
𝑞
𝑞
𝑞 1−𝑞
𝐷 (𝑞) = 𝑝 1 + · · · + 𝑝𝑠 + · · · + 𝑝𝑅
(5)
Every term in the summation in Eqn. 5 dictates the contributionof each species category 𝑖, to the diversity of a meter under consideration. Considering the conclusions drawn from the summarized
effects of attacks, we need to add atleast two more factors in Eqn. 5
to each term in the summation of the following function:

window (where the diversity index is being calculated) comparedto past windows. The past may be previous years’s history or ashorter term history of a set of previous consecutive time windows.
For smart city application context, we use consecutive time
windows, given the observation that shifting trends in data, candiametrically change the self similarity of species without presenceof attacks over yearly time horizons.
To build D, the simplest approach would be an absolute difference
between the relative abundance of each species category between
the current and the previous time window. Mathematically, letmatrix p(𝑓 − 1) denote the species abundance in previous time
window 𝑓 − 1 for the 𝑖-th meter and the same at the current time
window 𝑓 is denoted by p∗ (𝑓 ). Then, the most simple self similarity
matrix could be S(f) = |𝑝 (𝑓 − 1) − 𝑝 ∗ (𝑓 )|, where:
𝑝 1
𝑝 ∗
0
...
0 
0
...
0 

 1
∗
 0 𝑝2 . . .

0
0
𝑝
.
.
.
0 
∗
2

𝒑 (𝑓 − 1) = 𝒑 (𝑓 ) = 

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
. . 
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.



0
0
0
. . . 𝑝𝑅  𝑅×𝑅
0. . . 𝑝𝑅∗  𝑅×𝑅

However, we found two problems with this approach. (1) this will
fail to detect incremental ramp or boil-frog attack strategies, thatcause very small vertical changes over time. Hence, we need to lookover a longer time horizon for ‘sustained’ vertical changes. (2) there
could be false alarms, since some of the meters may show a higher
change in the legitimate difference of relative abundance in species
without attacks in pairs of windows. Without any transformation,
it creates a higher change in the eventual trust score even underbenign changes, which needs to be avoided.
0.8

(6)

5.2

Modified Diversity Index based Trust Score

5.2.1 Forming Species Self Similarity Matrix:. We build asquare matrix 𝑫 of RxR dimensions known as the species selfsimilarity matrix, where only diagonal entries are non-zero, andquantifies the effective level of similarity (or difference) of therelative abundance of a species with itself between the current time

0.8
)

0.2
0

0.6

s

0.4

(

where 𝑥𝑠 and 𝑦𝑠 are analogous to weights, that embed the effect of
the number of rare species, and vertical and incline changes as a
combination respectively. Hence, at the end of the day, we would
seek to get a formula for a modified diversity index score which
confirms to the functional form in Eqn 6. We will later verify, how
the final diversity index score confirms to this basic functional form.
While building the functional form, we have to be aware of how
different attacks, might exploit vulnerabilities, and also take steps
to reduce false alarms.
Intuition on Diversity order q: We know that all 𝑝𝑠 is between
0 and 1. Any 𝑝𝑠 when powered by 𝑞 > 1, the species with higher
probabilities (abundant) show a lower resultant decrease than forlower probability species, that show a higher decrease for the same𝑞. This is however, reversed when 𝑞 < 1; where the rarer species
see more increase compared to abundant species, when poweredby 𝑞 < 1. Since the data falsification attack has relationships withnumber as well as change of probabilities in the rarer species, thisgives the intuition that optimal value of 𝑞 might be between 0 and
1, which we will verify later.

1

0.6
Probaility


 1
𝑞
𝑞
𝑞 1−𝑞
𝑟𝐷 (𝑞) = 𝑥 1 .𝑦1 .𝑝 1 + · · · + 𝑥𝑠 .𝑦𝑠 .𝑝𝑠 + · · · + 𝑥𝑅 .𝑦𝑅 .𝑝𝑅

0.4
0.2

0

0.05

0.1

0.15
s

0.2

0.25

0.3

0

0

0.2

0.4

(f)

0.6
s

(a)

0.8

1

(f)

(b)

Figure 6: (a) Distribution of Benign Sample ∇𝑠 (𝑓 ) (b) The 𝜙transformation function (Texas Dataset)This gives an intuition that once an idea on the bounds of legitimate vertical changes is learned, changes beyond that can be overweighed, while changes below those bounds can be discounted.
These two aspects are embedded in the following way: Let thedifference between relative abundance vector between any twoconsecutive windows be denoted by 𝜖𝑠 (𝑘) = 𝑝𝑠 (𝑘 − 1) − 𝑝𝑠 (𝑘), a
shorter term similarity. Then we keep a long term memory of 𝜖𝑠for each species represented by:𝑓Õ𝜖𝑠 (𝑘)(7)∇𝑠 (𝑓 ) =𝑘=𝑓 −𝐹

such that ∇𝑠 (𝑓 ) keeps the cumulative sum of the differences
observed between pairs of time windows for a sliding frame containing 𝐹 previous windows. When there are no attacks, ∇𝑠 (𝑓 ) has
no increasing trend (see Fig. 7(a)) and the values are typically verysmall (See Fig. 6(a)). Infact, across an appropriate frame length (F),the ∇𝑠 flattens out (blue lines in Figs. 7(a) and 7(b)). In contrast, forincremental attacks, there is a small monotonic increasing trendin ∇𝑠 (green and red lines in Fig. 7(a) and 7(b) respectively). For allother strategies, the average ∇𝑠 is larger, under attacks.
0.25

0.3
0.25

0.2
(f)

No AttackConstant Attack
Incremental Attack

0.15
0.1

Incremental Attack Starts

Incremental Attack
No Attack

0.1
0.05

0.05
0
1

0.15

s

s

(f)

0.2

2

3

4

5

F (Frame Number)

6

7

8

0
0

2

4

6

8

10

f (Window Number)

(a)

(b)

Figure 7: (a) Effect of Varying Frame Length (b) Reference
Frame Tracking under Incremental Ramp Strategy
The species self-similarity matrix is given by 𝑫 (𝑓 ) such thateach diagonal element is computed through a function of the form(𝜙 (∇𝑠 (𝑓 )), such that the diagonal elements in 𝑫 (𝑓 ) is a mappingthat takes the ∇𝑠 across the frame within each species as the input
and mathematically written as: |𝜙 ( ∇1 )
0
...
0 

 0
𝜙 ( ∇2 ) . . .
0 𝑫 (𝑓 ) = 
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
. . . .
 00
. . . 𝜙 ( ∇𝑅 )  𝑅×𝑅


where each entry
𝜙 ( ∇𝑠 ) =1;(1 + 𝐴𝑏 𝑒 −𝐵𝑏 (∇𝑠 ) ) 1/𝜈𝜙 ( ∇𝑠 ) ∈ [0, 1](8)

is a generalized sigmoidal function which inputs the vertical changeover a frame of length 𝐹 at a time window 𝑓 . The 𝜙 transformation
produces the necessary weighing that reduces the false alarm rate
while not sacrificing missed detection. Here the 𝐵𝑏 is a growth rate
parameter controlling the value of ∇𝑠 for which the 𝜙 (.) function
reaches its max value, while 𝜈 is a displacement parameter that
controls the value of ∇𝑠 , where the 𝜙 (.) function enters the exponential growth phase. The 𝐴𝑏 is a parameter that decides the initial
y-intercept, when ∇𝑠 is zero. Fig. 6(b), shows the 𝜙 function.
5.2.2 Expectation of Temporal Self-Similarity:. Now we quantify the overall average change in the similarity of the 𝑖-th meter.
Let 𝒑 denote the probability abundance vector of species calculated
over a time window in near history (ideally just before attack starts)and the 𝑫 is a probabilistic measure related to that 𝑝 (given thedesign of 𝑫), so that we get something similar to a second-order
expectation where the random variable is itself a probability vector
𝒑. Mathematically, we do the following operation:
h
i
h i
h i
𝐸 (𝐷)
= 𝑫
𝒑
(9)
𝑅𝑥1

𝑅×𝑅

𝑅𝑥1

where 𝐸 (𝐷) is an Rx1 matrix where each element represents the
expectation (average) change in the self similarity (in terms ofprobability of species abundances) of the corresponding species
between over this time frame. Each element of the 𝐸 (𝐷) is of theform (𝜙 (∇𝑠 ) ∗ 𝑝𝑠 ), which gives an idea on the index of verticalchange within each species 𝑠 between two time frames. Let us call𝒑 as the reference probability vector.
Now it could be tricky to get the correct reference vector belonging to a frame just before attacks, especially if incremental attack
strategy inflicted. However, ∇𝑠 (𝑓 ) also allows us to pinpoint thisby backtracking the ∇𝑠 (𝑓 ) variation (See Fig. 7(b)) and the 𝑝 is built
from before the window just before the change-point of ∇𝑠 (𝑓 ).

5.2.3 Diversity Order Embedding:. From theoretical intuition,
one requirement was to magnify changes in intermediately rarer
species, which we accomplish here. We add the order into the expectation of similarity in a similar way that appears as a powerin the Hill’s Diversity Index, in order to achieve the embedding of
non-uniform vertical change such that we get the following:[𝑀] 𝑅𝑥1 = [𝐸 (𝐷)]𝑞𝑞  𝜙 (∇1 ).𝑝 1 




..


.  


𝑞
𝑴 =  𝜙 (∇𝑠 ).𝑝𝑠 


..




.

𝑞 


 𝜙 (∇𝑅 ).𝑝𝑅 

 𝑅×1

(10)

(11)

5.2.4 Magnifying quantity of species with changes:. From theoretical intuition, we need to finally ensure that very small incremental changes but happening in many unique IDs of rare species, has
importance in the resultant functional form of modified diversity
index that we are striving to achieve. We put more emphasis on theshifts in rarer species as a weight to each of the species in the 𝑅 × 1
matrix, [𝐸 (𝐷)]𝑞 . Note that we need a scalar value for the diversity
index trust score and the quantity weight matrix needs to be a 1 × 𝑅
matrix for the scalar to exist. Hence, we seek to design a weightvector that is 1 × 𝑅 dimension.
h ih i[𝑊 ] 1𝑥𝑅 = 𝐽
− 𝒑𝑻
(12)
1𝑥𝑅

1𝑥𝑅

where [𝐽 ] = [1 . . . 1] 1𝑥𝑅 is a matrix containing all 1’s for R columns,
and the intuition is that one minus a rare value will be a high value
and too many of these occurrences will push the resulting scalar toa higher portion in the number line. Hence, the resulting weightfactor is given by:
𝑾 = (1 − 𝑝 1 ) . . . (1 − 𝑝𝑠 ) . . . (1 − 𝑝𝑅 ) 1×𝑅(13)
5.2.5 Final Modified Diversity Index Trust Score:. The diversity
index based trust score of the q-th order for a smart meter 𝑖 is givenby the multiplication of 𝑾 and 𝑴. The reason they are multiplied
is to achieve the functional form we had earlier in Eqn. 6, as weshall see next.
h i𝑞 
𝑇 𝑅𝑖 (𝑞) = 𝑊 × 𝑀 = ([𝐽 ] − [𝒑 𝑻 ]) × 𝑫𝒑
(14)
One can verify how Eqn. 14 confirms to the original functional
form of mathematical abstraction of a diversity index score for
identifying data falsification. By plugging Eqn. 11 and Eqn. 13 into
Eqn. 14, we get a scalar value due to matrix multiplications ofdimensions 1XR and Rx1, which gives:𝑞𝑞

TR𝑖 (𝑞) = (1 − 𝑝 1 ).(𝜙 (∇1 ))𝑞 .𝑝 1 + · · · + (1 − 𝑝𝑅 ).(𝜙 (∇𝑅 ))𝑞 .𝑝𝑅
If we assume (1 − 𝑝𝑠 ) = 𝑥𝑠 and (𝜙 (∇𝑠 ))𝑞 = 𝑦𝑠 , then the above
reduces to the desired abstraction of the mathematical functional
form, we expressed as required earlier in Eqn. 6.
Hence, to conclude the modified diversity index score of a meter𝑖 that can detect compromised meters is:
h i𝑞 
𝑟𝐷 𝑖 = ([𝐽 ] − [𝒑 𝑻 ]) × 𝑫𝒑
(15)

1 in thewhere 𝑟 𝐷 𝑖 > 0, if 𝑞 > 0. The whole exponent factor of 1−𝑞
original functional form is ignored since it does not provide any
added classification advantage as far tracking changes. Anotherimportant point is the nature of change in diversity score after theattack is launched, and its effect on the final distribution of 𝑟𝐷 𝑖
values of compromised versus honest meters. Due to the nature of
Eqn. 15, where changes in each species are added up, the meters
launching data falsification will experience an increase in the diversity scores after the attack. In contrast, the non-compromised
meters will exhibit a lower diversity score than the compromised
meters. We will verify this in the experimental results section.

6

PARAMETER LEARNING AND THRESHOLD

Now that we have the architecture of our base model, we need toprovide a generalizable way of learning various parameter values
given any dataset. Our approach towards this is a supervised one,
where we divide the training set into two parts: first, without any
attacks; the second containing attacks from a subset of meters wechoose and program them to simulate a limited set of attacks. Our
method learns parameters according to a target objective functionthat maximizes the difference between the diversity index scoresof the honest and malicious classes in the training set. Later on, we
use cross-validation set to find a threshold and then apply it on atesting set for performance evaluation.

6.1

Training Set Details

We use the full year of 2014 as the training set for Texas dataset.
The attack starts after the end of 6-th month. The malicious classlabels contain the following attack features: An additive attack with𝛿𝑎𝑣𝑔 = 100W, 𝜌𝑚𝑎𝑙 = 30%, with an incremental ramp strategy that
increases by 20W every 15 days. The idea is that if it detects forthe smallest and slowest moving attack, it will be able to detect
anything stronger. Other parts of the threat model are not used fortraining, since we need to verify that our method is generalizable
to detect ‘mutated’ and ‘unknown’ attack realizations that it was
not trained on.

6.2

Decision Variables

The controllable decision variables are namely 𝐴𝑏 , 𝐵𝑏 , 𝜈, 𝑠𝑤, 𝑞 and𝐹 which are candidates for optimization. Among these, parametersstrongly related to the dataset are 𝐵𝑏 and 𝜈, others are weakly related to the dataset. Note that the 𝛿𝑎𝑣𝑔 and 𝜌𝑚𝑎𝑙 are uncontrollable
decision variables which are beyond defenders knowledge. However, it is known that if we observe a linear separability between
diversity index scores of a compromised and honest set of devices,
for a lower 𝛿𝑎𝑣𝑔 , it will automatically hold for higher 𝛿𝑎𝑣𝑔 valuesby virtue of our scoring design. Therefore, during learning, we
train with only select candidates of 𝛿𝑎𝑣𝑔 that are below the desired
𝑑𝑙𝑏 . For tractability of search space,lower bound of sensitivity 𝛿𝑎𝑣𝑔
we partition the candidate species widths and candidate 𝛿𝑎𝑣𝑔 into
discrete partitions with upper and lower bounds 𝛿 1 and 𝛿𝑃 .

6.3

Objective (Error) Function

The objective function (or the error/loss function) should maximize
the separation between compromised and honest devices, in termsof the distribution of their diversity index scores. Hence, we used
the squared difference of average of diversity index scores between
the compromised and honest sets in the training set. Intuitively,that combination of parameters/decision variables that maximizes
this objective function is the optimal parameter set.
 Í (𝑟𝐷 ℎ ) Í (𝑟𝐷𝑚 )  2(16)−
𝑒 = 𝑚𝑎𝑥
𝑁 −𝑀
𝑀
s.t.
s.t.

𝐴𝑏 > 0;0 < 𝑞 < ∞;0 < 𝐵𝑏 < 1;𝑑𝑙𝑏
𝑤 1 < 𝑠𝑤 ≤ 𝛿𝑎𝑣𝑔;

0<𝜈 <1
1 ≤ 𝐹 < 𝐹𝑚𝑎𝑥

It might seem that there too many variables to optimize. However, in reality, the search space of 𝑠𝑤, 𝑞, 𝐴𝑏 turns out to be bounded
and small, once we apply the following pruning logic and design
considerations: The candidate species width 𝑠𝑤 is upper bounded
𝑑𝑙𝑏 , which isby the desired lower bound sensitivity of attacks 𝛿𝑎𝑣𝑔
small, making the 𝑠𝑤 range limited. Furthermore, given the roleof the Renyi diversity order, we can prune the search space ofdiversity order to 𝑞 ∈ (0, 1]; an explanation provided earlier in theoretical intuition. Appendix C.2 shows that this intuition matches
the outcome of the optimization.
The optimization can be solved using a grid search; or an efficient method like gradient descent which scales well when there
are many parameters with a wide search space. For gradient descent
to work, the error function needs to be transformed into a convex
function. Our objective function is a concave function with a global
maxima. Such functions can be converted into a convex function
using the negative logarithm of the original objective function, andthen apply gradient descent. However, accuracy depends on thesmoothness of the convex function. In our implementation, thenumber of parameters is limited, and has a smaller search spaceeither by design or through pruning. Hence, we solved our optimization, using a grid coordinate search method. The explainabilityof the relationship of each parameter with the objective (error)
function is shown in detail in Appendix C.
For Texas data, we found the following (near) optimal parametervalues: 𝜈 = 0.05, 𝐵𝑏 = 0.1, 𝑞 = 0.55, 𝑠𝑤 = 100, 𝐴𝑏 = 0.3. To crosscheck for parameter values for a different dataset, we repeatedthis process for over the Irish dataset. The first 7 months of thedataset were used as training set, and attack labels were introduced
after the end of the 3rd month, using the same attack features as
the Texas dataset. We solved the parameters separately and found𝜈 = 0.03, 𝐵𝑏 = 0.12, 𝑞 = 0.5, 𝑠𝑤 = 100, 𝐴𝑏 = 0.3, 𝐹 = 8 and window
length is 15 days. We can observe that 𝜈 and 𝐵𝑏 are slightly different(due to dataset specifics), while other parameters are closer due totheir relationship with attack model and underlying theory.

6.4

Threshold Selection

Cross-validation ascertains whether the optimal values generalize
well or not to maximize the linear separation of scores, and also
learn a classification threshold that generalizes during the testing
set. We use a Receiver Operating Characteristics (ROC) curve to get
the full spectrum of possibilities of false alarm (FA) to true positive(TP) rates. From this, based on the defender’s desirable maximum
tolerable false alarm rate, the corresponding threshold giving that
FA rate is chosen, and then applied to the testing set for security
performance evaluation.
Cross-validation Dataset: For Texas Dataset, we used 2015, partitioned into 12 partitions for cross-validation. For Irish dataset,
we used 6 partitions, starting from the 8-th month of 2009. We

0.6
0.4
0.2
0

7.2.1 Generalizing against untrained Attacks:. We firstshow performance under previously unseen attack types (deductive
and alternating switching) across varying 𝛿𝑎𝑣𝑔 values and the new𝜌𝑚𝑎𝑙 = 40%; threats which did not feature in the training phase,using a data order aware strategy.

0.6
0.4
0.2

0.6

0.8

1

0

0.2

0.4

0.6

0.8

(a)

(b)

Figure 8: ROC in Cross-validation: (a) Texas (b) Irish

EXPERIMENTAL VALIDATIONThis section includes cross-validation and testing set results of both
Texas and Irish datasets for the smart metering application. The
experimental result section is divided into the following subsections:(i) Attack Implementation on Test set description; (ii) Performance
results (iii) Cost Benefit Analysis (iv) Comparison with other works(v) Verifying robustness to ramp strategy and attack scales (vi)
Performance under complete knowledge of defense mechanism

7.1

0.5

0.5

1

False Positive Rate

Attack Implementation on Testing Set

For each attack type, and strategy (discussed in the threat model) we
did the following: For the Texas dataset, the 2016 year’s data (having
a duration of a year), we had five attack start points interspersed
approximately by two months to cover the entire testset duration.
Similarly, for the Irish dataset, the final five months of the 2010 data
were used as a test set, with two attack start points interspersed
in a two-month duration. This is done to show that regardless ofthe start point of attack, the reported missed detection is unbiased.
Hence, five (or two) versions of the attacked testing set are obtainedfor each attack type for Texas (and Irish) datasets respectively.
In each version, we had six different sets of compromised meters
per attack scale value (to remove compromised meter selection bias),
making a total of 30 (or 12) versions. Each such version is attacked
with the indicated several different 𝛿𝑎𝑣𝑔 (from the compromised
ones of course), and then fed to the diversity index model. Then,the final result on missed detection and false alarms is reported by
combining the results from all these versions. For reporting baseline
false alarm rate (where there are no attacks throughout the year or
test duration), we counted the false alarms accordingly. Additionally,note that we have parameterized the space of attack strengths andscales covering all possible values. There is no availability of real
attack dataset in this area, but our implementation included thegold standards for performance evaluation covering any gaps thatmight otherwise exist. Note that deductive, alternating switchingattacks attack types, KLD minimizing strategies were not used for

Misdetection Rate

0.4

False Positive Rate

Maximum Tolerable FA = 2%
Maximum Tolerable FA = 5%
Maximum Tolerable FA = 8%
Maximum Tolerable FA = 10%

0.4
0.3
0.2
0.1
0
100

200

300
400
Margin of False Data

500

0.4

Maximum Tolerable FA = 2%
Maximum Tolerable FA = 5%
Maximum Tolerable FA = 8%
Maximum Tolerable FA = 10%

0.3
0.2
0.1

0
100 150 200 250 300 350 400 450 500 550 600

600

Margin of False Data

(a)

(b)

Figure 9: Deductive Attacks: MD rates over Varying Max. Allowable FA (𝜌𝑚𝑎𝑙 = 40%): (a) Texas Data (b) Irish Data
Figs. 9(a) and 9(b) show the MD rates across various 𝛿𝑎𝑣𝑔 against‘deductive attacks’ under the Texas and Irish datasets respectively.
Each line corresponds to a performance given by different thresholds corresponding to that particular tolerable base FA rate. Similarly, Figs. 10(a) and Fig. 10(b) show the MD rate ‘alternating switching attacks’ for Irish and Texas datasets, respectively. To verify, that
performance is also valid over the additive attacks and ramp strategy (on which we trained our model) please refer to Appendix D
and Appendix E. We will show in Sec. 7.4, at these 𝛿𝑎𝑣𝑔 values our
missed detection rates are much smaller than to previous research.
0.5

0.5
Misdetection Rate

0.2

Performance Results

Instead of ROC curve, we show (i) missed detection (MD) rates
across a wide 𝛿𝑎𝑣𝑔 range, for different thresholds based on user’s
tolerable FA rate; (ii) the base rate FA, which is false alarm rate,
when there are no attacks throughout the test set; because mostcompanies have a concern on lowering FA rates (because the priorprobability of an actual attack is low). The ROC curve from crossvalidation, is used to pick four corresponding thresholds that gave
2%,5%,8% and 10% FA rate; which are then applied to the test set.

0

0

7

7.2

Misdetection Rate1
0.8

training. We put these in test set only to understand whether the
method generalizes to previously unseen attacks.

0.4
0.3

Maximum Tolerable FA = 2%
Maximum Tolerable FA = 5%
Maximum Tolerable FA = 8%
Maximum Tolerable FA = 10%

0.2
0.1
0
100 150 200 250 300 350 400 450 500 550 600

Margin of False Data(a)

Misdetection Rate1
0.8
True Positive Rate

True Positive Rateaverage the parameter outputs to provide more accurate estimate of model prediction performance. For Texas dataset, we got:
𝜈 = 0.04, 𝐵𝑏 = 0.08, 𝑞 = 0.55, 𝑠𝑤 = 100, 𝐴𝑏 = 0.29, while For Irish
dataset, we got 𝜈 = 0.03, 𝐵𝑏 = 0.1, 𝑞 = 0.5, 𝑠𝑤 = 100, 𝐴𝑏 = 0.31.
We used these values to retrofit in the model and generated thediversity index scores of both classes. Then thresholds are varied
according to desired false alarm rate.
ROC Curve: Figs. 8(a) and 8(b), shows the ROC curve under a𝛿𝑎𝑣𝑔 = 100 from cross-validation, with an AUC of 0.89 and 0.93
respectively. In general, the ROC curves for various 𝛿𝑎𝑣𝑔 can beplotted. A utility can use his desired maximum allowable false alarm
rate and find the corresponding threshold using this ROC.

Maximum Tolerable FA = 2%
Maximum Tolerable FA = 5%
Maximum Tolerable FA = 8%
Maximum Tolerable FA = 10%

0.4
0.3
0.2
0.1
0
100

200

300

400

Margin of False Data

500

600

(b)

Figure 10: Alternating Switching: MD rates over Varying
Max. Allowable FA(𝜌𝑚𝑎𝑙 = 40%): (a) Irish Data (b) Texas Data
Performance against untrained KLD minimization strategy
Fig.11(a) at tolerable FA rate of 10%, is shown for the 3 attack
types. The performance is slightly worse compared to the data order aware strategy. The increase in mis-detection rate on averagefor the KLD minimizing strategy across all attack types and 𝛿𝑎𝑣𝑔
values, is 7.3% keeping the same FA rate. The Fig. 11(b) shows that
our method scales well and is invariant to changing 𝜌𝑚𝑎𝑙 .

0.2

0.15

0.1
Misdetection
False Alarm

0.05

0.1

0
150

200

250

300

350

400

Margin of False Data

450

0

10

20

30

40

50

60

70

80

90

Percentage of Compromised Smart Meters ( mal)

(a)

(b)

Figure 11: Performance: (a) KLD Minimizing Strategy (b) Invariance to Attack Scales
7.2.2 False Alarm Performance:. A concern on anomaly
based scoring frameworks are false alarms and their costs. A summary of base rate false alarm performance in the testset is included inTable 1. The Texas dataset has more shifting trends (due to renewable penetration), thus it has more base rate FA than Irish dataset.
Table 1: Base Rate False Alarm Percentages in test set
Tolerable FA Threshold
2%
5%
8%
10%

Texas Test Set FA
2.60%
6.25%
9.37%
10.93%

1

Tolerable FA Threshold
2%
10%
2%
10%

𝛿𝑎𝑣𝑔

𝑁 𝑒𝑡𝐵𝑒𝑛𝑒 𝑓 𝑖𝑡 : Irish

𝑁 𝑒𝑡𝐵𝑒𝑛𝑒 𝑓 𝑖𝑡 : Texas

100
100
400
400

+ 21,219.12+ 16,922.34+ 141,686.64+ 138,441.06+ 4,868.88+ 5,597.16+ 30,413.04+ 32,087.47

Error Rate

Cost Benefit Usability of our Performance

Here we analyze the costs of MD and FA rates from the perspectiveof real life usability. Once inferred as attacked, an audit trail is doneby utilities on each device for confirmation. According to [37], audit
inspections are billed for a median cost of 𝐶𝐴 = $141 per device,while [36] reported the average time to inspect each meter device
is 55-65 minutes. Audits are an annual affair in many companies
and our test set is also for one year. There are two options foraudit for a utility: (1) a utility wide audit (expensive), (2) an audit
on those devices detected as positive (less costly). Let different
utilities have different tolerable false alarm that vary between 2%
to 10%. There is a loss due to audit on false alarms but a gain for
detecting compromised meters successfully. We consider here only
the monetary value per Kilo Watts hour (KWH) of electricity thatis falsified. The effective profit/loss per year can be calculated as:𝛿𝑎𝑣𝑔 × 𝜂 × 𝐸 × 365
× (𝑀 − 𝑚𝑑)(17)
𝑁 𝑃𝑟𝑜 𝑓 𝑖𝑡 =1000
where 𝑀 is the number of meters compromised, 𝑚𝑑 is number ofmissed detections, 𝜂 is the number of reports/day, 𝐸 = $0.12 per
KWH is average cost of electricity in USA (could be as high as $0.38
in some states). On the other hand, the cost of false alarms per year
is: 𝐿 = 𝐶𝐴 ∗ 𝑓 𝑎 and 𝑁 𝑒𝑡𝐵𝑒𝑛𝑒 𝑓 𝑖𝑡 = 𝑁 𝑃𝑟𝑜 𝑓 𝑖𝑡 − 𝐿 where 𝐶𝐴 is thecost of audit/meter, and 𝑓 𝑎 is the number of false alarms.
In Table 2, we provide the practical implication of our performance under user tolerable false alarm rates of 2% and 10%, with𝜌𝑚𝑎𝑙 = 40%, in terms of monetary benefit. Given the numbers, a 2%
tolerable FA is more profitable for Irish data, while 10% tolerable FA
is more profitable for Texas data, for the same 𝛿𝑎𝑣𝑔 . Since the difference in losses is not drastic, our recommendation for utilities is tochoose 10% tolerable rate, since it will give much lower MD when
attack actually occurs. Since the Irish data has a large micro-grid,
the benefit is large, underscoring that the benefit is scalable.
Table 2: Profit/Loss Per Year with our Framework

MD Proposed
MD [3]MD [1]MD [9]

0.8
0.6
0.4
0.2
0
100

200

300

400

500

Margin of False Data

600

700

800

Figure 12: Performance Comparison with Existing Research

7.5

Quantifying Impact of Evasion Attacks
with Knowledge of our Model

Now we analyze how adversary can bypass our detector according to knowledge of our the model.
Knowledge of Species Width and Threshold: Here adver𝑖sary’s optimal strategy is to use a tailor made 𝛿𝑎𝑣𝑔
for eachcompromised meter, that just refrains from crossing the threshold(by back calculating deviation from the original diversity index
just before attacks start). The breakdown 𝛿𝑎𝑣𝑔 has to be at-least
equal to or less than the optimal species width, else it will change
species memberships, to help the classification. The Fig. 13(a)shows the average upper bound evasion 𝛿𝑎𝑣𝑔 is around 76 watts
across randomized compromised meter sets. Fig. 13(b) is a CDF
which proves that 84% of those 383 compromised meters have amaximum evasion 𝛿𝑎𝑣𝑔 of less than 100W (93% of them are below
150W). The median absolute deviation of this upper bound is about
31 Watts. This is enough to show that we reduce the impact of theundetected attack, even when attacker has this knowledge.
77
avg

7.3

Irish Test Set FA
2.11%
5.33%
8.86%
10.58%

Comparison with Previous Research

We compare our performance with 3 categories of existing methods:(i) classical ML, (ii) information-theoretic, (iii) statistical learning.
Classical ML uses SVMs [10], decision/regression trees (DRT) [9].
The [10] outperforms [9], hence we compare our work with [10].
For information theoretic approaches [1, 14], we chose to comparewith [1] (though mainly it showed the Texas data results) since
it reports for various 𝛿𝑎𝑣𝑔 unlike [14]. Statistical learning based
method [3] outperform [18, 25, 26] and hence is chosen for comparison. The Fig. 12, shows a comparison of our method with existing
works under our threat model (assuming deductive attacks overIrish dataset since its common to all previous works). We can observe that the MD rate of our method (blue- solid line) is much
lower compared to other works especially for lower 𝛿𝑎𝑣𝑔 , with athreshold corresponding to 10% acceptable FA. This is fair comparison since Refs [1, 3, 10] have FA rates ≥ 10%, even for attacks with𝛿𝑎𝑣𝑔 > 350W (See Appendix B).

Avg Evasion

0.05
100

7.4

0.15

1
0.8

76.5

CDF

Additive Attack
Deductive Attack
Alternate Switching Attack

0.2

Error Rate

Misdetection Rate

0.25

76

0.4
0.2

75.5
75
100

0.6

200

300

400

500

600

Number of Compromised Meters

(a)

700

800

0
0

20

40

60

80 100 120 140 160 180

Maximum Evasive avg(b)

Figure 13: Performance Under Evasion Attacks: (a) Upper
Bound breakdown 𝛿𝑎𝑣𝑔 (b) CDF of evasion (𝑀 = 383)

Knowledge of Frame Size and Sigmoid parameters: The
adversary can learn the exact frame length and sigmoidal parameters 𝜈 and 𝐵𝑏 , to make sure that the total ∇𝑠 for each species
across the frame 𝐹 is below the point that fails to trigger the
exponential growth of the 𝜙 function, with an incremental attack.
The increment per day can be lesser than intended 𝛿𝑎𝑣𝑔 divided bywindow length multiplied by 𝐹 or an increment/day such that ∇𝑠 issafely below 0.5. With this, we found that it takes 120 days to reach𝛿𝑎𝑣𝑔 of 100W. To reach 400W (where existing works have highMD), it will need 480 days. Thus, we reduce attacker’s break-eventime. The knowledge of 𝐴𝑏 , does not help the adversary in evasion.
Knowledge of 𝑞: If adversary knows the exact value of 𝑞, understands the role of 𝑞, it can introduce an attack, avoiding theintermediate rare species magnification, by distributing the false
data points onto the other rarer species, by introducing a proportional change (assuming knowledge of historical distribution) in
the all species equally (See Fig. 14(a)). However, note that 𝑞 is justone of parameters, and our method also looks at number of species
that contain changes however small, with the 𝑊 matrix and alsouses an expectation of the similarity 𝐷 (𝑓 ) across all the species,
and these gets added up. Hence, an adversary with this attack will
not be able to evade detection. The proof is shown in Fig. 14(b).
0.4

1.2
Honest Meter
Compromised Meter

1
Diversity Index

Probability

0.3
No AttackAttack

0.2

0.1

0.8
0.6
0.4
0.2

0

0
0

5

10

15

Species ID

20

25

30

(a)

0

50

100

150

200

Smart meter ID(b)

Figure 14: (a) Attack Crafted Using 𝑞 knowledge (b) Corresponding Performance under 𝑞 knowledge attack

8

CONCLUSIONSIn this paper, we offered a novel information-theoretic anomaly
scoring technique that showed successful detection of smart meters
launching data falsification with very low to high attack strengths
and attack scales are possible, using AMI as proof of concept. The
proposed method’s accuracy generalizes well across two different
datasets, with completely different years of data collection, countries, sizes of micro-grids. The conclusion is that the method is a wayof inferring security status in terms of data integrity where inherent
variances are higher than impactful attack strengths. Additionally,
we conclude that for a cognizant attacker, the undetectable strategy
space in smart energy AMI is reduced from what was achieved byprevious works, without a drastic increase in false alarms. As partof future work, we will study how to strengthen the model undertraining data poisoning attacks and give theoretical estimationsof expectation of change in diversity index score as a function ofvarious attack parameters, and check on whether retraining overthe untrained attacks improves missed detection performance.

ACKNOWLEDGMENTS
This work is funded by NSF grants SATC-2030611, SATC-2030624,
OAC-2017289

REFERENCES[1] S. Bhattacharjee, A. Thakur, S. Silvestri, S.K. Das, “Statistical Security Incident Forensics
against Data Falsification in Smart Grid Advanced Metering Infrastructure", ACM CODASPY,
pp. 35-45, 2017.
[2] S. Bhattacharjee, S.K. Das, “Detection and Forensics under Stealthy Data Falsification in Smart
Metering Infrastructure", IEEE Transactions on Dependable and Secure Computing, Early access,2018 (DOI: 10.1109/TDSC.2018.2889729).
[3] S. Bhattacharjee, A. Thakur, S. K. Das “Towards Fast and Semi-supervised Identification of
Smart Meters Launching Data Falsification Attacks" ACM ASIA CCS, pp. 173-185, 2018.
[4] B. C. Costa, B. L. Alberto, A. M. Portela, W. Maduro, and E. O. Eler, “Fraud detection in electric
power distribution networks using an ANN based knowledge-discovery process," Int. J. Artif.
Intell. Appl., vol. 4, no. 6, pp. 17-23, 2013.
[5] K. Foo, “Ghost Talk: Mitigating EMI signal injection attacks against analog sensors" IEEE
Symposium on Security and Privacy (Oakland, CA), May 2013.
[6] K. Fu, W. Xu, “Risks of Trusting the Physics of Sensors", Communications of the ACM, Vol. 61
No. 2, pp., 20-23, Feb 2018.
[7] S.-C. Huang, Y.-L. Lo, and C.-N. Lu, “Non-technical loss detection using state estimation andanalysis of variance", IEEE Trans. on Power Systems, Vol. 28(3), pp. 2959-2966, Aug. 2013.
[8] R. Jiang , R. Lu, Y. Wang, J. Luo, C. Shen, and X. Shen, “Energy-Theft detection issues foradvanced metering infrastructure in smart grids", Tsinghua Science and Technology, Vol. 19(2),pp. 105-120, April 2014.
[9] A, Jindal, A. Dua, K. Kaur, M. Singh, N. Kumar, S. Mishra, “Decision Tree and SVM-Based Data
Analytics for Theft Detection in Smart Grid" IEEE Trans. Industrial Informatics, 12(3), 2016.
[10] P. Jokar, N. Arianpoo, V. Leung, “Electricity Theft Detection in AMI Using Customers’ Consumption Patterns", IEEE Trans. on Smart Grid, Vol. 7(1), 2016.
[11] L. Jost, B. Tungurahua “Entropy and Diversity”, Wiley Synthesizing Ecology. Vol. 113(2), pp.
363-375, 2006.
[12] K. Khanna, B. K. Panigrahi and A. Joshi, “AI-based approach to identify compromised metersin data integrity attacks on smart grid," IET Generation, Transmission & Distribution, vol. 12,
no. 5, pp. 1052-1066, 2018.
[13] T. Koppel, “Lights Out: A Cyberattack, A Nation Unprepared, Surviving the Aftermath",
Crown Publishers, New York, 2015.
[14] V. B. Krishna, K. Lee, G. A. Weaver, R. K. Iyer and W. H. Sanders, “F-DETA: A Frameworkfor Detecting Electricity Theft Attacks in Smart Grids," IEEE/IFIP on Dependable Systems andNetworks (DSN), 2016, pp. 407-418.
[15] C.-H. Lo and N. Ansari, “CONSUMER: A novel hybrid intrusion detection system for distribution networks in smart grid", IEEE Trans. on Emerging Topics in Computing, 1(1):33-44, 2013.
[16] T. Leinster, M. Meckes, “Maximizing Diversity in Biology and beyond", Entropy, Vol 18, 2016.
[17] S. McLaughlin, D. Podkuiko, P. McDaniel, “Energy theft in the advanced metering infrastructure", Proc. of Critical information infrastructures security (CRITIS’09), Springer-Verlag, pp.
176-187, 2009.
[18] D. Mashima, A. Alvaro, “Evaluating Electricity Theft Detectors in Smart Grid Networks",
Springer Heidelberg, pp. 210-229, 2012.
[19] R. Mohassel, A. Fung, F. Mohammadi, K. Raahemifar, “A survey on Advanced Metering Infrastructure", Elsevier Journal of Electrical Power & Energy Systems, 63:473-484, Dec. 2014.
[20] A. Rad, A.L. Garcia, “Distributed internet-based load altering attacks against smart power
grids", IEEE Trans. on Smart Grids, 2(4):667-674, Dec. 2011.
[21] A. S. Rawat, P. Anand, H. Chen and P. K. Varshney, "Collaborative Spectrum Sensing in the
Presence of Byzantine Attacks in Cognitive Radio Networks," in IEEE Transactions on Signal
Processing, vol. 59, no. 2, pp. 774-786, Feb. 2011.
[22] S. Salinas, M. Li, and P. Li, “Privacy-preserving energy theft detection in smart grids: A P2P
computing approach," IEEE J. Sel. Areas Commun., vol. 31, no. 9, pp. 257-267, Sep. 2013.
[23] T. Trippel, O. Weisse, W. Xu, P. Honeyman and K. Fu, “WALNUT: Waging Doubt on the Integrity of MEMS Accelerometers with Acoustic Injection Attacks," IEEE European Symposium
on Security and Privacy (EuroS & P), pp. 3-18, 2017.
[24] R. Sevlian and R. Rajagopal, “Value of aggregation in smart grids", IEEE SmartGridComm, pp.
714-719, Oct. 2013.
[25] E. Werley, S. Angelos, O. Saavedra, O. Cortes, A. Souza, “Detection and Identification of Abnormalities in Customer Consumptions in Power Distribution Systems", IEEE Trans. on Power
Delivery, Vol. 26(4), 2011.
[26] W. Yu, D. Griffith, L. Ge, S. Bhattarai, N. Golmie, “An integrated detection system against false
data injection attacks in the Smart Grid, Security and Commun. Networks, 8(2): 91-109, 2015.
[27] W. Zhang, S. K. Das and Y. Liu, "A Trust Based Framework for Secure Data Aggregation in
Wireless Sensor Networks," 2006 3rd Annual IEEE Communications Society on Sensor and
Ad Hoc Communications and Networks, Reston, VA, 2006, pp. 60-69.
[28] M. Sinai, N. Partush, S. Yadid, E. Yahav, “Exploiting Social Navigation", Black Hat, 2015.
[29] https://www.smartgrid.gov/project/pecan_street_project_inc_energy_internet _demonstration.html[30] Irish Social Science Data Archives, Available at: http://www.ucd.ie/issda/data/[31] https://www.epri.com/#/pages/product/000000000001026553/[32] http://energy.gov/sites/prod/files/oeprod/DocumentsandMedia/14AMI_System_Security_Requirements_updated.pdf[33] https://krebsonsecurity.com/2012/04/fbi-smart-meter-hacks-likely-to-spread/[34] https://www.maximintegrated.com/content/dam/files/design/technical-documents/whitepapers/smart-grid-security-recent-history-demonstrates.pdf[35] https://web.archive.org/web/20170131081134/https://www.smartgrid.gov/files/The_Smart_Grid_Promise_DemandSide_Management_201003.pdf[36] https://www.ovoenergy.com/help/smart-meter-installation#how-long-will-i-have-to-waituntil-i-get-my-smart-meter[37] https://www.fairwarning.com/blog/power-audit-trail-data-security/[38] https://www.miningreview.com/top-stories/research-study-quantifies-energy-theft-losses/[39] https://www.npr.org/sections/money/2011/10/27/141766341/the-price-of-electricity-in-yourstate[40] https://electricitywizard.com.au/electricity/electricity-cost/how-much-does-electricity-cost[41] https://www.cnet.com/news/money-trumps-security-in-smart-meter-rollouts-experts-say/[42] https://resource.optimalnetworks.com/blog/2015/01/07/cost-security-audit

APPENDIXAIMPACT CASE STUDY OF LOW MARGINSOF FALSE DATA

The impact of an attack on the utility not only depends on the
average margin of false data (𝛿𝑎𝑣𝑔 ), but also to the (i) real tampering
incidence relative to the network sizes, (ii) time until the detection of
the attacked meter, and (iii) widely varying prices across countries.
The size of real world utility’s metering network and the realistic
number of tampered meters are large. For example [38], real meter
audits from three countries (USA, Canada, Australia) for utility
deployments reported the rates/numbers of tampered meters, andthe corresponding sample and population losses. The USA study(APS Arizona) is the most statistically scientific study that involved
one utility with 868,000 meters. This study concluded 0.72% was thetamper rate with high statistical certainty, given the observations
from the representative samples used, which comes to about 6000
meters of that utility. An additional 1% was believed to be tampered
indirectly through diversion. On the other hand, the Canadian study
did not reveal the total number of meters, but involved several
utilities across the country to find 1.36% of tampered meters. In areal utility network, the raw number is quite large which makes
total losses very high. Such studies are done once in decades due tothe effort and cost, hence the detection times are also very large [38].
Consider the average margin as 𝛿𝑎𝑣𝑔 = 100W which is seemingly
very low. The average electricity price in USA is 𝐸 = $0.12/kwH [39].
If this attack goes undetected for just a year (i.e., 𝐷 = 365 days), thedollar loss/year is calculated as:

performance is better for Irish dataset, due to high shifting trends.
Most of the papers used as a random bias scaling attack. However, these numbers degrade especially when subjected a stealthierthreat model. The [1], also shows seemingly good numbers but
the 𝛿𝑎𝑣𝑔 is much higher in this case. This work fails at very high𝛿𝑎𝑣𝑔 as well as low 𝛿𝑎𝑣𝑔 ≤ 500𝑊 . In [10], the 𝜌𝑚𝑎𝑙 considered was
extremely small, so the detection rate reported was inflated. Whenthe 𝜌𝑚𝑎𝑙 was increased, the detection rate success dropped (i.e.,the missed detection rate increased greatly). Specifically, under our
threat model with The false alarm rates reported by previous works
do not report the base rate false alarms. The [3] shows that classifier
at 300W, has a high missed detection rate of 30%, and drops to 52%
at 250W, but with high false alarm rates of 29% and 40%. If the FA
is bounded like in our method, these missed detection rates will bemuch higher than what were reported. This is true for all the otherworks as well. In contrast, our method produces a missed detection
rate in the range of 15%-22% regardless of attack type for a false
alarm rate of 10% when the attack strength is lower than a factorof 4 (at 𝛿𝑎𝑣𝑔 = 100) in competing approaches.
Table 3: Self Reported Performance of Previous works
Parameter
FA
MD

𝛿𝑎𝑣𝑔
𝜌𝑚𝑎𝑙

CPBETD [10]
29%
24%
400W
0.72%
1 mo

C.1

1 mo

Effect of 𝐵𝑏 on error function

4
Error Function (e)PERFORMANCE OF PREVIOUS WORKS≤ 40%

Figs. 15(a) shows the relationship of our objective (error) function
and growth parameter 𝐵𝑏 , keeping other factors constant. Fig.15(b)is proof of corresponding convex equivalent that allows gradientdescent to be applied to quickly converge to the optimal values.

Error Function (e)B

Entropy [1]11%8%
700-800W

Since it is difficult to visualize more than 3 dimensions, we show
the effect of each parameter by varying only that while keeping
constant other decision variables (both controlled and uncontrolled).
This will prove that the error function has a global maxima and an
equivalent convex function with global minima is achievable. We
prove that for each parameter, an optimal answer is possible. All
figures shown in this section of appendix is corresponding to the
Irish training dataset.

0.15

The table 3 includes the self reported numbers of what attack parameters they used and their corresponding reported performance.
However, these numbers are not directly comparable in a fair manner with the our method. The [14] is not included in the table sincethe 𝛿𝑎𝑣𝑔 assumed are too high and is easier to detect comparedto other methods. The [22], is not included in the table, since themethod used a synthetic dataset of just 25 users, which does notmake the missed detection rate fairly comparable with ours orother methods. We found that for most existing methods, the FA

ARMA [18]
33%
28%
NA
NA
1 mo

EXPLAINABILITY OF DIFFERENTPARAMETERS FOR OPTIMIZATION

𝑇 𝐼 = (𝛿𝑎𝑣𝑔 ∗ 𝑀 ∗ 𝜂 ∗ 𝐷 ∗ 𝐸)/1000
where 𝜂 = 24 is the reports/day (typically hourly). (Note that wechose 𝛿𝑎𝑣𝑔 with data sampling rates equal with existing works forfair performance comparison). Under the USA (Arizona) study, theloss will be $660, 048/year at the corresponding lower bound tampering rate (0.72%), for 𝐸 = $0.12/kWH, 𝛿𝑎𝑣𝑔 = 100W. Including thediversion and theft together (at 1.72%), the loss/year is $1, 569, 399
at 𝛿𝑎𝑣𝑔 = 100. At 𝛿𝑎𝑣𝑔 = 350𝑊 , where previous methods fail, theloss/year is 5, 492, 898. It is easy to conclude that the lower boundimpact of stealthy margins is very expensive. Furthermore, in many
countries, the cost of electricity is not so cheap. For example, in Australia, where 𝐸 = $0.34/kwH [40], the same attack will cost threetimes more. However, these rates are only the tip of an iceberg. For
example, in Puerto-Rico AMI utility attack (PREPA), about 10% of
the meters were found compromised [33] among about 1.3 million
customers, facing a $400 million loss.

10% − 60%
≤ 10 days

TTD

C

FGKT [3]29%-5%
30%-4%
300W-700W

0.1

0.05

0

0

0.1

0.2

0.3

Growth Rate (Bb)

(a)

0.4

0.5

0.6

3.5
3
2.5
2
1.5

0

0.1

0.2

0.3

0.4

0.5

0.6

Growth Rate (Bb)(b)

Figure 15: Effect of 𝐵𝑏 (a) Objective Function ; (b) Convex
Equivalent of objective 𝑒

Effect of 𝜈 and 𝑞 on error function

Figs. 16(a) and 16(b) show the objective function with changing
displacement parameter 𝜈 and diversity order 𝑞, proving that aunique global optimal exists for each of these parameters.
0.15

0

0

0.01

0.02

0.03

0.04

0.05

0.05

0

0.06

0

0.2

0.4

0.6

0.8

1

1.2

q

Point of Growth ( )

(a)

0.25
0.2
s

Figure 16: Objective Function: (a) Effect of 𝜈, (b) Effect of 𝑞

C.3

Effect of Species Width 𝑠𝑤

Error function (e)

0.15

0.1
avg
avg

0
60

70

80

90

100

110

120

= 60
= 100

130

0.1

0.05

0.05
0
1

0

2

3

4

5

F (Frame Number)

6

7

8

0

5

10

15

20

25

30

Window Size (Days)

(a)

(b)

Figure 18: (a) Optimal Frame Length (b) Optimal Window
Size

D

ADDITIVE ATTACKS IN TEST SET

The following figures Figs. 19(a) and 19(b) show performance in thetesting set for additive attacks (which were trained on) for Texasand Irish datasets respectively. The conclusion is that they have aslightly less mis-detection rate, because these attacks were trainedon during parameter learning.

140

Species Width

0.4
0.4

From the perspective of our objective function, if the species
width is too small, the legitimate changes will cause the diversity
index score of honest meters to increase, thus making the difference
between the compromised and the honest set minimum. However, if
the species width is too high, then our model will miss the changes
in the compromised set, and both sets will have the same rangeof diversity index score, but in a different region of the diversity
index axis. Therefore, some intermediate optimal value would exist
for 𝑠𝑤 which would produce a global maxima, showing the optima
𝑑𝑙𝑏
exists. This illustration is shown in Fig. 17, where 50 < 𝑠𝑤 ≤ 𝛿𝑎𝑣𝑔
and given this constraint, we can see that for all 𝛿𝑎𝑣𝑔 equal to or
below this, the objective function is maximized at 𝑠𝑤 = 100.

Optimal Frame Length and Window Size

We felt that the explainability of this is not apparent, if explainedwith the objective function. Hence, we use a different y axis, whichindirectly effects our objective function. The Fig. 18(a), shows how
to find get an optimal frame length 𝐹 for our model. The frame
length should be atleast 5 and atmost 8, keeping in mind the incremental change. Above 𝐹 = 8, the answers tend to become suboptimal (since legit changes creep into the ∇𝑠 (𝑓 )), although theperformance degradation is very slow. However, keeping in mind
the threat of incremental attacks, we used the upper bound of optimal frame length 𝐹 = 8. Information theoretic approaches andentropies are steady state measures and involve probabilities of

Misdetection Rate

Figure 17: Species width versus objective function

C.4

No AttackConstant Attack
Incremental Attack

0.15
0.1

The optimal species width depends on 𝛿𝑎𝑣𝑔 which is unknown
to the defender. Hence, there is no practical way of scientificallycalculating the species width in the optimal sense. Additionally, thefalse alarm is subjective to utility, and we need to be able to bypass
this difficulty. The upper bound of the species width has to be equal
to or lower than the initial desired lower bound (dlb) of 𝛿𝑎𝑣𝑔 thatthe defender seeks to detect, else the method will not work. As longas the problem is in this constrained space, the following is true:

0.05

0.15

0.3

(b)

Error Function (e)

0.05

0.1

Maximum Tolerable FA rate = 2%
Maximum Tolerable FA rate = 5%
Maximum Tolerable FA rate = 8%
Maximum Tolerable FA rate = 10%

0.3

Misdetection Rate

0.1

0.2
0.1
0
100

200

300

400

500

600

Maximum Tolerable FA rate = 2%
Maximum Tolerable FA rate = 5%
Maximum Tolerable FA rate = 8%
Maximum Tolerable FA rate = 10%

0.3
0.2
0.1

0
100 150 200 250 300 350 400 450 500 550 600

Margin of False Data

Margin of False Data

(a)

(b)

Figure 19: Additive Attacks: MD rates over Varying Max. Allowable FA (𝜌𝑚𝑎𝑙 = 40%): (a) Texas Dataset (b) Irish Dataset

E

RAMP ATTACK SENSITIVITY ANALYSIS

Fig. 20, shows the performance for under incremental ramp attack
strategy for various 𝛿𝑎𝑣𝑔 for Texas dataset for all three attack types(shown for tolerable FA of 10%).
0.3

Misdetection Rate

Error Function (e)Error Function (e)0.15

categories to converge to their true value and therefore an adequate minimum time window is necessary. However, if the time
window is too large the detection of meters will be delayed. Hence,
we studied the effect of the time window sizes on the modified
diversity index score, keeping other factors fixed. Fig. 18(b), showsthat the optimal window size that maximizes the error function is awindow length of 15 days (from cross-validation). The errors tend
to increase slightly with increasing window length, because in alarge time window, legitimate shifting trends in the consumption
mimic attacks; thus increasing the false alarms slightly. Hence, forall results, we keep a sliding frame size as 15 days.

(f)

C.2

0.25
Additive
Deductive
Alternate Switching

0.2
0.15
0.1
0.05
0
100

150

200

250

300

350

400

Margin of False Data

Figure 20: Performance against Ramp Attack Strategy