DeepLocker

Concealing Targeted Attacks with AI Locksmithing
Dhilung Kirat, Jiyong Jang, Marc Ph. Stoecklin
IBM ResearchDhilung Kirat

Jiyong Jang

Marc Ph. Stoecklin

Cognitive Cyber Security Intelligence (CCSI)IBM Research2

AI-aided attacks
Profile a target
to increase success [2]Craft an email to bypass filters [1]AI

Mutate malware to bypass AV [3, 4]

Execute machine-speed/creative attacks [5, 6][1] S. Palka et al., “Fuzzing Email Filters with Generative Grammars and N-Gram Analysis”, Usenix WOOT 2015[2] A. Singh and V. Thaware, “Wire Me through Machine Learning”, Black Hat USA 2017[3] J. Jung et al., “AVPASS: Automatically Bypassing Android Malware Detection System”, Black Hat USA 2017[4] H. Anderson, “Bot vs. Bot: Evading Machine Learning Malware Detection”, Black Hat USA 2017[5] DARPA Cyber Grand Challenge (CGC), 2016[6] D. Petro and B. Morris, “Weaponizing Machine Learning: Humanity was Overrated Anyway”, DEF CON 20173AI-aided attacks

AI-embedded attack
Profile a target
to increase success [2]Craft an email to bypass filters [1]AI capability embedded inside malware itselfAIMutate malware to bypass AV [3, 4]AI

Execute machine-speed/creative attacks [5, 6][1] S. Palka et al., “Fuzzing Email Filters with Generative Grammars and N-Gram Analysis”, Usenix WOOT 2015[2] A. Singh and V. Thaware, “Wire Me through Machine Learning”, Black Hat USA 2017[3] J. Jung et al., “AVPASS: Automatically Bypassing Android Malware Detection System”, Black Hat USA 2017[4] H. Anderson, “Bot vs. Bot: Evading Machine Learning Malware Detection”, Black Hat USA 2017[5] DARPA Cyber Grand Challenge (CGC), 2016[6] D. Petro and B. Morris, “Weaponizing Machine Learning: Humanity was Overrated Anyway”, DEF CON 20174

Malware concealment – Locksmithing

Checks for sandbox

Decryption
code

1980

1990

Obfuscation

Mutate payload⎯⎯⎯
Polymorphism
Metamorphism

Encryption

Hide payload⎯⎯⎯
Packers

Checks for target

Decryption
code

2000

Evasive malware

Avoid being analyzed⎯⎯⎯
VM check
Processes check

Decryption
code

2010

Targeted attack

Disclose only at a target⎯⎯⎯
Target attributes check5

AI Locksmithing© 2003 Warner Bros. Pictures All Rights Reserved

6Unleashing DeepLocker – AI Locksmithing

7

DeepLocker – OverviewEvading

Benignapplication
Existing
malwarestatic, dynamic, manual analysis

Security analysis

AI
DeepLocker

AI

Benign-looking
malwareTargetattributes
AI-powered concealment

Execution at non-target
Benign behavior

Execution at target
Malicious behavior

Distribution & execution8

DeepLocker Deep Dive9

Traditional targeted attack
Start

No

IsthisthisIsnotaa
sandbox?
target?

Stuxnet
RegKeyExists(“HKLM\SOFTWARE\SIEMENS\STEP7”)

Yes

ExecuteAttack!

End10

AI-powered targeted attack
Start

AIIs this atarget?
ExecuteAttack!

End11

What is a Deep Neural Network (DNN)?
Hidden Layers

Input Layer

x1x2

Output Layer

w1w2wnnf (∑ wi xi )i=1

Sigmoid()tanh()ReLU()yxn

12Deep Convolutional Neural Network

Input Layer

Hidden Layers

Output Layer

AlexNet (2012) [1]
8 layers, 622K neurons, 60 million parameters[1] Krizhevsky, Alex, et. al. "Imagenet classification with deep convolutional neural networks.” NIPS 2012.
13

Target attributes
Audio, visual

Physicalenvironment

Softwareenvironment

TargetSensors

User activity

Geolocation
14

Target detection

Template matching requires atemplate to match to.

Start

ExtractedFace

ReferenceFaceDNNNo

Yes

ExecuteAttack!

End15

Derivation of an unlocking key
Start

DeepLocker

ExecuteAttack!

End16

UnlockingConcealment

DeepLocker – AI-Powered Concealment and Unlocking
Target attributes

Target Concealment

Secret key

Malicious
payloada1a2Payload Concealment

Concealed
payloadencryption

a3

a2

a1

DeepLocker

decryption

a3

Input attributes

Target Detection

Recovered key

Concealed
payload

Payload Unlocking

Malicious
payload17

AI-powered concealment

DeepLocker

No decryption key
available in malware
sample to reverseengineer!

18

Key generation
Unstable key!
Face Detection
Model0.9

10.0

00.8

TargetBucketization

10.100.9

1High-dimensional
facial features

Key19

Key generation

Face Detection
ModelTarget

Key Generation
Model0Class 1 ý

0.01

Class 2 þ

0.8

1Class 3 þ

0.1

0Class 4 ý

0.9

1Class 5 þHigh-dimensional
facial features

Key0.920

Analysis of the key generation model
Target Detection

Dataset: Labeled Faces in the Wild (LFW)
http://vis-www.cs.umass.edu/lfw/21

DeepLocker – AI-powered concealment
3 Malicious Intent Concealment
Payload is fully encrypted concealing
how the final attack is executed

1 Target Class Concealment
Does not reveal what it is looking for (e.g.,faces, organization, or a completely obscureobject specific to the target environment)

2 Target Instance Concealment
If the target class is an individual, it
does not reveal who it is looking for22

Attacking DeepLocker – AI Lock Picking

23Ways to counter

PayloadexecutionAI

AI-poweredconcealment

DeepLocker

24Ways to counter

PayloadexecutionAI

AI-poweredconcealment

Code attestation

Block sensor access

Host-based monitoring

AI usage monitoring

Brute-force key

Brute-force attributes

Deceptive resources

Deceptive attributes

Code analysis

AI lock picking

25

?
26

Reverse engineering AI models

Partialocclusion

Neural attention
modelDebug neuralnetworks

Occlude a portion of the image tosee how the embedding isaffected (deconvnet) [1]

Heatmap using the degree of
excitation of neurons in eachlayer (excitation backprop) [2]Fuzzing for neural networks(coverage-guided fuzzing) [3][1] M. Zeiler and R. Fergus, “Visualizing and understanding convolutional networks,” ECCV 2014[2] J. Zhang et. al., “Top-down neural attention by excitation backprop,” ECCV 2016[3] A. Odena and I. Goodfellow, “TensorFuzz: Debugging neural networks with coverage-guided fuzzing,” arXiv 201827

Takeaways
Rapid democratization of AI has madeAI-powered attacks an imminent threat
DeepLocker is a demonstration of the potential of

AI-embedded attacks

Current defenses will become obsolete andnew defenses are needed28Thank you

Dhilung Kirat

dkirat@us.ibm.com

Jiyong Jangj jang@us.ibm.com

Marc Ph. Stoecklin

mpstoeck@us.ibm.com

IBM Research