JPAAWG 5TH MEETING - 2022

Using AI in Threat Detection
Examples and 2022 Email Threat Review

Severin Walker – Vade Director of Products (ISP)Maxime Meyer

Lead Research Scientist

2Gabriel Loiseau
Research Scientist

Outline● Detecting phishing using Random Forest● Using Deep Learning to improve phishing detection● Phishing detection in production
● Q&A

3

Detecting Phishing Using Random Forest

Supervised Machine Learning - Training

5Supervised Machine Learning - Evaluation• Evaluate on a dataset different from the training data
• Measure metrics (FPR, precision, …)• Ensure non regression

6Supervised Machine Learning - Metrics
TruthPrediction

PhishingSafe● FPR = FP / (FP+TN)
Phishing

True Positive(TP)False Positive (FP)● Recall = TP / (TP+FN)
Safe

7

False Negative(FN)

True Negative (TN)● Precision = TP / (TP+FP)

Supervised Machine Learning - Production• Make prediction on production data
• Measure metrics (speed, memory usage, …)• Evaluation based on emails from FBL

8

Phishing detection with Random ForestRandom Forest1, 2 use an ensemble of Decision Trees

1Breiman, L. (2001). Random Forests. Machine Learning, 45, 5-32. doi: 10.1023/A:1010933404324

92Doyen Sahoo, Chenghao Liu, and Steven C. H. Hoi. Malicious URL Detection using Machine Learning: A Survey. 2017. doi: 10.48550/ARXIV.1701.07179.

Phishing detection with Random Forest

10Feature

Value

Document
Size

145

Number oftags

27"login" in
subdomain

False

TLD".com"...

Phishing detection with Random Forest

11

Phishing detection with Random Forest

12

Phishing detection with Random Forest

Url is HTTPS?
Document
Size < 384 ?
Number oftags < 30 ?

Url is HTTPS ?

Safe

Phishing"login" indomain name?

Safe

Document
Size > 100 ?

...

Safe

Phishing...

...

Document
Size > 120 ?

Safe

Phishing

Safe"Safe"
13

Safe

...

"Phishing"Phishing"Safe"

Decision boundary

Human supervision
Model outputs a "Phishing Risk" between 0% and 100%50% ? 75% ? 90% ?

14Limitations
Random Forest in production generalizes well and gives goodresults but ...

• Still some FPs, quite a lot of FNs• Too many samples sent yearly to analysts (> 250 000)
• New supervised learning technologies seem promising15

Using Deep Learning to Improve Phishing
Detection

Deep Learning and Neural Networks
Dendrites

Inputsbw1
w2

OutputwnNucleus

Axon

Linear Activation
function function

Rosenblatt, F. (1958). The perceptron: a probabilistic model for information storage and organization in the brain. Psychological review, 65(6), 386.

17

Deep Learning and Neural Networks

18

Deep Learning and Neural Networks● Deep Learning can make senseof raw data bygenerating its own "abstract"features● Automatic feature extraction● Cannot be done with classicalmachine learning

19Pros and Cons of Deep Learning
Pros●
●

●

20

Fewer preprocessing
Less expert
knowledge
Can be used with all
numerical data points

Cons●

●
●

More computationalpower (GPU)
Hardly explainable
Requires a lot of data towork properly

Architecture21

Data inputs for Deep Learning
Transform real life objects into numerical values
Images→Pixel values

••

•
•
22

Text→???
Tokenization stepWhat is tokenization ?
• Split text into smaller units (words)• Obtain "meaning" from sentences{ We, fight, Phishing, . }We fight Phishing.
{ W, e, _, f, i, g, h, t, _, P,h, i, s, h, i, n, g, . }{ We, fight, Phi, ##shing, . }23

From text to values• BERT Wordpiece Tokenizer1• Used in other URL-based approaches2
Original URL

http://thisisaphishingurl.com

Tokenized URL[http, :, /, /, this, ##isa, ##phi, ##shing, ##ur, ##l, .,com]

Token indexes
from vocabulary[8299,1024,1013,1013,2023,14268,21850,12227,3126,2140,1012,4012]

1Devlin, Jacob; Chang, Ming-Wei; Lee, Kenton; Toutanova, Kristina (11 October 2018). "BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding". arXiv:1810.04805v2
2Maneriker, Stokes, Lazo, Carutasu, Tajaddodianfar, Gururajan (27 August 2021) "URLTran: Improving Phishing URL Detection Using Transformers". arXiv:2106.05256

24

Embedding Layer
8299

1024

...

4012

Embedding

25

0.023

0.09

...

0.022

0.056

0.022

...

0.039

...

...

...

...

-0.019

0.012

...

0.096

Convolution Layer1_ x 1 + _ x 0 + _ x (-1) +_ x 1 + _ x 0 + _ x (-1) +_ x 1 + _ x 0 + _ x (-1) = ?
1LeCun, Yann; Bengio, Yoshua (1995). "Convolutional networks for images, speech, and time series"

26

Convolution Layer1_ x 1 + _ x 0 + _ x (-1) +_ x 1 + _ x 0 + _ x (-1) +_ x 1 + _ x 0 + _ x (-1) = ?
1LeCun, Yann; Bengio, Yoshua (1995). "Convolutional networks for images, speech, and time series"

27

Average Pooling Layer1

112

45

6

7

83

2

1

01

2

3

4

AveragePooling

3,25

5,25

22

(1+1+5+6)/4 = 3.25

1Y. Lecun, L. Bottou, Y. Bengio and P. Haffner, "Gradient-based learning applied to document recognition," in Proceedings of the IEEE, vol. 86, no. 11, pp. 2278-2324, Nov. 1998, doi: 10.1109/5.726791.

28

Flattening Layer

6-9-8

n6-9-8-3-2-3-30-2-3

Flattening
nxm-2-3-3m

0-229

Activation Functions

Le, H., Pham, Q., Sahoo, D., & Hoi, S. C. (2018). URLNet: Learning a URL representation with deep learning for malicious URL detection. arXiv preprint arXiv:1802.03162.

30

Architecture31Motivations to not use Deep Learning alone• Deep Learning achieves better recall for Phishing detection ...
• … but with a slight decrease of precision• Helps to remove some Random Forest FPs• Combine Random Forest and Deep Learning• Hybrid methods gives better results for phishing detection1

1Sara Afzal· Muhammad Asim · Abdul Rehman Javed · Mirza Omer Beg · Thar Baker (4 March 2021). "URLdeepDetect: A Deep Learning Approach for Detecting Malicious URLs Using SemanticVector Models". Journal of Network and Systems Management (2021) https://doi.org/10.1007/s10922-021-09587-832Best choice to combine decision• Voting• Weighted Averaging• Distinguish cases• Use one after the other (verification step)33

Resulting Pipeline34

Phishing Detection in Production

Experimentation Protocol•

Can this model be used to reduce supervision workload•~ 5000 phishing documents over 6 months are analyzed•

Apply our pipeline for all documents in this period•

Compare the agreements / disagreements

36

Agreements / Disagreements

37

Agreements / Disagreements

38

Focus on FPs
Phishing risk percentage
interval

Number of documents wheresupervision answered SafeUnder 90%100

Between 90% and 95%

19Over 95%

18

Manual analysis:
The 19 and 18 samples are
ONLY FN.
39

Decision Pipeline40Metrics in production•Daily process of 13M documents•

5ms to extract features•

33ms to obtain a Random Forest prediction•15ms to obtain Deep Learning prediction•Between 1ms and 5ms of latency

41

Conclusion
Deep Learning can capture new phishing
Combination of multiple tools helps toreduce supervision workload• 30% of removed documents from supervision• Instant supervision from Deep Learning• Best strategy: classify documents asphishing automatically only when the two models agree••

42

Examples in Recent Trends

43

Examples in Recent Trends

44

Examples in Recent Trends

45

Examples in Recent Trends

46Questions?
For additional questions, please email:maxime.meyer@vadesecure.com
gabriel.loiseau@vadesecure.com