2/20/2024

1Research Background1

Countermeasures against Data Poisoning
Attacks against Machine Learning Models
with Multi-Sensor Inputs• Popularity of Machine Learning• Used in various fields such as anomaly
detection in factories and farms• Machine learning-based applications
are also of interest.

Shyam Maisuria• Smart healthcare, smart grid,smart water treatment, smart

Murata Laboratory, Graduate School of Information Science and Technology, Osaka University

home , etc.

• Concerned about attacks onmachine learning models[1] Adedeji, K.B.; Hamam, Y. Cyber-Physical Systems for Water Supply Network Management: Basics, Challenges, and Roadmap. Sustainability 2020, 12, 9555.
https://doi.org/10.3390/su12229555

12/13

2

Attack risk in systems using multiple sensors

Poisoning Attack Scenario• In the case of a system using multiple sensors, the risk of a hostile sample ishigher if the sensor-based attack could also be misinterpreted by the machinelearning model.

• The attacker can cause service
disruption by altering the treatment
process.

•• Can also release untreated water tothe rivers and streams

Attackers may use vulnerable sensors to attack

Research Objectives
Verification of the possibility of attacks in cases where an attacker can monitor andtamper with some of the input features.
Propose countermeasures to the above attacks• To prevent this a machine learningmodel is trained to detect thisattacks.

Approach
Attack: Create a machine learning model that can generate attacks by inputting someof the input features.
Countermeasures: Find the attacked features and exclude and discriminate them

• What if attacker compromises themachine learning model itself

3/13

3

4/13

4

How to Evaluate Attacks

Generation of attacks• Dataset: Secure Water Treatment (Swat) dataset[4]• Total of 6 different water treatment processors• 51 sensors and actuators• The attacker installs an attacker
generator model that changes theoutput of sensors.

Metric Used in this experiment• The attack generator model is•

Anomaly Score was used as a metric•

Dt is prediction error••

S is anomaly score
H and L are 90 and 20 percentile values of Dt

trained in advance by using theknowledge of the attacker[4]Jonguk Kim, Jeong Han Yun, and Hyoung Chun Kim. Anomaly detection for industrial control systems using sequence-to-sequence neural networks. Lect.
Notes Comput. Sci. (including Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinformatics), 11980 LNCS:3–18, 2020.

5/13

56/13

61

2/20/2024

Anomaly Scores of Target Model

Countermeasures -Overview• Main idea is to identify and remove compromised sensors before the model isupdated.
• How to do this? We use a Robust Model.
Robust Model:••

We use a “robust model” that doesn’t heavily depend on specific features
It allows us to calculate results without relying on certain sensor data

Detecting Compromised Sensors:••

Gradients calculated using attacker-generated values differ from those without compromisedvaluesEnables detection and identification of compromised sensor devices by comparing gradients.

Architecture of Robust Model
7/13

7

8/13

8

Countermeasures- Training the Robust Model

Countermeasures- Detection• Training Process:• Compromised Sensor Detection:• We check if gradients calculated using a sensor’s values are outliers.
• Each sensor gets a score, measuring how different its gradients are fromthe rest.

•••

We create multiple trainers, each excluding features frompotentially compromised sensors
Calculated gradients are combined, outliers are removed, andthe model is updated.

• Cumulative scores are tracked over time, and sensors exceeding athreshold are flagged.

Below equation determines which gradient to exclude

9/13

910/13

10Countermeasure-Results
Countermeasure-Results• Shows the cumulative anomaly score• Detection threshold 10

Target Model• Show which sensor group is hacked• Helps prevent further impact frompoisoning attack on machine learningmodel
Robust Model

11/13

1112/13122

2/20/2024

Conclusion

How to train the generator2• Machine learning algorithms are vulnerable to data poisoning (compromising•Input for Attack Generator
Model: Hacked sensor data

data collection), including Deep Learning systems.

•Output from Attack Generator• Poisoning attack is generated using part of hacked sensors.

Model: Generated poisoned• We propose a robust model as countermeasure and successfully detected hacked

datasensors.

•Training Objective: Minimize• Robust model takes a lot of time to trainloss function A to degrade• Future task is to implement measures to shorter the training timetarget model accuracy

13/13

13

[2] Luis Mu˜noz-Gonz´alez, Battista Biggio, Ambra Demontis, Andrea Paudice, Vasin Wongrassamee, Emil C. Lupu, and Fabio Roli. Towards poisoning of deep learningalgorithms with back-gradient optimization. AISec 2017 - Proc. 10th ACM Work. Artif. Intell. Secur. co-located with CCS 2017, pages 27–38, 2017.

14/1
614

Countermeasure-Results

Attack Assessment Results• Shows poisoning attack for robust model with countermeasure• Each figure trained for 25, 50 and 75 epochs respectively• Each model is based on each process from Swatdataset• We only focused on process 5• Which has total of 13 sensors
and actuators• Target model is trained using
those 13 sensors and actuators• Model accuracy: 71%• Found attacks: 5/7

15/16

15

16/16

16

Research Background1

Goal of Research• Popularity of Machine Learning• Used in various fields such as anomaly detection in factories and farms• Machine learning-based applications are also of interest.

• Smart healthcare, smart grid, smart water treatment, smart home , etc.

Objective: Investigate data poisoning attacks and their profound impact on machine learningmodels.
Why Focus on Data Poisoning Attacks?
• Concerned about attacks on machine learning models• Hostile samples that carry small noises that misclassify the model• E.g. Traffic sign “Stop” misinterpreted as “speed sign”[1] Gu, Tianyu, et al. "Badnets: Evaluating backdooring attacks on deep neural networks." IEEE Access 7 (2019): 47230-47244.

17



17/1
6Significant Impact: These attacks can severely compromise model integrity, making them acritical concern.
Long-Term Consequences: Once attacked, models may remain vulnerable over an extendedperiod.

Our Research Aims To:
Understand attack strategies that break machine learning models.

Develop effective mitigation and prevention techniques.

Safeguard real-world applications, ensuring their trustworthiness and safety.

18/22

183

2/20/2024

Background- Data Poisoning Attacks1

Presentation Outline• Many Machine Learning(ML) based systems uses multiple sensors• It is possible that some sensors are hacked by the attacker• Background – Data Poisoning Attacks• The ML model can also be attacked from the hacked sensors• Kurniawan et al demonstrated the possibility of this kind of attacks as an adversarial examples

Sensors not hacked• Poisoning Attack• Target ModelSensors hacked• Results• Conclusion• References[1] Ade Kurniawan, Yuichi Ohsita, and Masayuki Murata. Experiments on Adversarial Examples for Deep Learning Model Using Multimodal Sensors. Sensors,
22(22):8642, nov 2022.

19

19/2
2

20/22

20

Background

21

Poisoning Attack

22

Secure Water Treatment Plant (Swat)3• Secure Water Treatment (Swat) is a water treatment site forcybersecurity researchExperiments• 11 days of continuous operation:• 7 days under normal operation• 4 days with attack scenarios• 51 sensors and actuators• Total of 6 different water treatment processors[3]Jonathan Goh, Sridhar Adepu, Khurum Nazir Junejo, and Aditya Mathur. A dataset to support research in the design of secure water treatment systems. Lect. Notes
24/22
Comput. Sci. (including Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinformatics), 10242 LNCS(October):88–99, 2017.

23244

2/20/2024

Attackers' objective

Target Model – Features• To generate data poisoning attacks that evades detection• Objective function = weighted sum loss function••= generator models objective = target models
objective• Alpha(α) controls the importance of each of the objectivefunction• Alpha(α) = low : prioritize evading detection• Alpha(α) = high: prioritize effectiveness of attack
25/22

25

26/22

26

Hacked Sensors

Results

27/22

2728

Anomaly Scores with varied data points

Anomaly Scores with varied alpha values• Alpha = 0.5 is constant• Number of datapoints is constant at 7000sec• As number of datapoints increase anomaly scores decrease• As alpha value increases the anomaly score decreases• Shows poisoning attack is successful if we use more datapoints• Shows poisoning attack is successful when we increase the alpha value

29/22

2930/22305

2/20/2024

Check if Target Model detects Poisoned Attack• Alpha values determine if attack is detected or not• Higher alpha values = attack gets detected and poisoning attack is stronger• Lower alpha values = avoid attack detection and poisoning attack is weaker

31/22

316