Weaponizing Data Sciencefor Social Engineering:
Automated E2E Spear Phishing on TwitterJohn Seymour | Philip Tully
1#SNAP_R

You care about phishing on social media

2#SNAP_R

TL;DR#SNAP_R
SocialNetworkAutomatedPhishing withReconnaissance

3TwitterProfiles#SNAP_R

Phishing

Offense#SNAP_R

ISO: Demo Volunteers

Tweet%#SNAP_R(before%the%demo%to%get%an%example%tweet!

4#SNAP_R

#whoamiJohn Seymour Philip Tully@_delta_zero @phtully
Data Scientist at ZeroFOX Senior Data Scientist at ZeroFOX
Ph.D. student at UMBC Ph.D. student at University of Edinburgh &Royal Institute of Technology
Researches Malware Datasets Brain Modeling & Artificial Neural Nets

5

#SNAP_R

HighA Novel Phishing Campaign Design
Our$#SNAP_R
Success Rate

Fully+Automated>30%+Accuracy

Spear$PhishingHighly+Manual
45%+Accuracy

Phishing

Low

Mostly+Automated
5?14%+Accuracy

Low

6

Level of Effort

High#SNAP_R

Fooling Humans for 50 Years

1966: ELIZA Chatbot! Joseph Weizenbaum, MIT! Parsing & keyword replacement7

2016: @TayandYou! Microsoft AI! Deep Neural Network#SNAP_R

InfoSec ML Historically Prioritizes Defense

8#SNAP_R

Machine Learning on Offense
Automated Target DiscoveryAutomated Social Spear Phishing
Evaluation and Metrics
Results and DemoWrap Up
Weaponizing Data Science
for Social Engineering:
Automated E2E Spear Phishing on Twitter
9#SNAP_R

Machine Learning on Offense

Weaponizing Data Science
for Social Engineering:
Automated E2E Spear Phishing on Twitter
10#SNAP_RWhy Twitter?
! Bot-friendly API! Colloquial syntax! Shortened links! Trusting culture! Incentivized data disclosure

11#SNAP_RShoutout
Where(Do(the(Phishers(Live?(Collecting(Phishers(Geographic(Locations(from(Automated(Honeypots(Robbie(Gallagher
We’ve+taken+a+novel+approach+to+automating+the+determination+of+a+phishers+geographic+location.+With+the+help+of+Markov+chains,+we+craft+honeypot+responses+to+phishers’+emails+in+an+attempt+to+beat+them+at+their+own+game.+We’ll+examine+the+underlying+concepts,+
implementation+of+the+system+and+reveal+some+results+from+our+ongoing+experiment.

12#SNAP_R

Techniques, Tactics and Procedures! Our ML Tool...
TwitterProfiles#SNAP_R

Phishing

Offense! Shortens payload per unique user
! Auto-tweets at irregular intervals! Triages users wrt value/engagement! Prepends tweets with @mention! Obeys rate limits! We added...
! Post non-phishing posts! Build believable profile

13#SNAP_RDesign Flowis_target(user)

TwitterProfiles

get_timeline(depth)#SNAP_RPhishing14

gen_markov_tweet()

gen_nn_tweet()schedule_tweet_and_sleep()

post_tweet_and_sleep()

Offense#SNAP_R

Automated Target Discovery

Weaponizing Data Science
for Social Engineering:
Automated E2E Spear Phishing on Twitter
15#SNAP_R

Triage of High Value Targets on Twitter! Accessible personal info
! Historical profile posts! Heterogeneous data
! Text, images, urls, stats, dates

16#SNAP_RExtracting Features from
GET users/lookup! Engagement: following/followers! #myFirstTweet! Default settings! Description content! Account age

17#SNAP_R

Clustering Predicts High Value Users

Eric+Schmidt

Eric+Schmidt
1818#SNAP_R

Selecting theBestClustering
Model! Many algorithms! Many hyperparameters! Max avg. score  [-1,..,1]! 0.5-0.7 reasonable structure

19#SNAP_R

Automated Social Spear Phishing

Weaponizing Data Sciencefor Social Engineering:
Automated E2E Spear Phishing on Twitter
20#SNAP_R

Recon and Footprinting for Profiling! Compute histogram of tweet timings(binsize = 1 hour)! Random minute within max hour to tweet! Bag of Words on timeline tweets! Select most commonly occurring nonstopword! We seed the neural network with topics thatthe user frequently posts about

21#SNAP_RLeveraging Markov Models
10.38! Popular for text generation:see /r/SubredditSimulator,
InfosecTalk TitleBot

I
0.62

don’t
1

like
0.54

0.46

infosec

ML

11.

22

1! Calculates pairwise frequency oftokens and uses that to generate
new ones! Based on transition probabilities! Trained using most recent posts on
the user’s timeline#SNAP_R

Training a Recurrent Neural Network! Hosted on Amazon EC2! Trained on g2.2xlarge
instance (65¢ per hour)! Ubuntu (ami-c79b7eac)! Training set > 2M tweets! Took 5.5 days to train

LSTM+=+Long+Short?Term+Memory! 3 layers, ~500 units/layer

Illustration: Chris Olah (@ch402)
LSTMs: Hochreiter & Schmidhuber, 199723

#SNAP_R

Tradeoffs and Caveats
ModelLSTM

Markov Chain

Days

Seconds

High

MediumAvailability
Size

PublicPublicLarge

SmallCaveats• Deeper representation ofnatural language, generalizes
well• Overfits to each user, cancreate temporally irrelevant
tweets• Retraining required for newlanguages• Performs poorly on users withfew tweets

MetricTraining Speed
Accuracy24#SNAP_R

Language and Social Network Agnosticism! Markov models only use content on user’s timeline, whichmeans they can automatically generate content in otherlanguages! For neural nets, you’d only need to scrape data from the target
language and retrain! Both of these methods can also be applied to other socialnetworks
25#SNAP_R

Evaluation and MetricsWeaponizing Data Science
for Social Engineering:
Automated E2E Spear Phishing on Twitter
26#SNAP_R

Here’s a malicious URL...

27#SNAP_R

And, apparently goo.gl lets us shorten it!

28#SNAP_Rgoo.gl also gives us analytics

29#SNAP_R

Results and Demo
Weaponizing Data Science
for Social Engineering:
Automated E2E Spear Phishing on Twitter
30#SNAP_R

Wild Testing #SNAP_R

31

31#SNAP_R

Pilot Experiment! Via #SNAP_R we sent 90 “phishing” posts
out to people using #cat
! After 2 hours, we had 17% clickthrough rate
! After 2 days, we had between 30% and 66%
clickthrough rate

! Inside the Data! goo.gl showed 27 clickthroughs (30%) camefrom a t.co referrer! Unknown referrers might be caused by bots
! With unique locations, clickthrough rate may be
as high as 66%

32#SNAP_R

Man vs. Machine 2 Hour Bake Off
UserMetric
Total Targets
Tweets/minute

Click-throughs
Observations

33

PersonSNAP_R~200819

1.67

6.85

49275

• Copy/Pasting messages todifferent hashtags• Arbitrarily scalable with thenumber of machines#SNAP_R

DEMO of #SNAP_R

34#SNAP_R

Wrap UpWeaponizing Data Sciencefor Social Engineering:
Automated E2E Spear Phishing on Twitter
35#SNAP_R

Potential Use Cases! Social media security awareness
TwitterProfiles#SNAP_RPhishingOffense! Social media security education! Automated internal pentesting! Social engagement! Staff Recruiting

36#SNAP_R

Mitigations
TwitterProfiles! Of course, we’re white hats here…
! But machine learning is rapidly becoming
automated, so black hats would have thiscapability soon.

! Protected accounts are immune totimeline scraping, which defeats the tool
! Bots can be detected#SNAP_R! Standard mitigations apply:
Phishing

37

Offense! Don’t click on links from people you don’tknow! Report! Twitter is pretty good at flagging spamaccounts! Maybe URL shorteners should be responsible
for malware?

#SNAP_R

Black Hat Sound Bytes
TwitterProfiles#SNAP_R

Phishing

38

Offense! Machine learning can be used
offensively to automate spear phishing! Machine-generated grammar is bad, but
Twitter users DGAF
! Abundant personal data is publicly
accessible and effective for socialengineering#SNAP_R?
39John Seymour Philip Tully@_delta_zero @phtullyWe’ll also be at thebooth
immediately after the presentation!

#SNAP_R