1

• Joint work with
─ Freya Gassmann, University of Saarland, Germany
─ Robert Landwirth, FAU of Erlangen-Nuremberg, Germany• Acknowledgments for data gathering and analysis
─ Nadina Hintz, Andreas Luder, Anna Girard, Gaston Pugliese2IntroducRon• Studied math (Russia) & computer science (Germany)• PhD in computer science (2008), Germany─ Access control protocols for wireless sensor networks• Researcher at FAU, Germany
─ Friedrich-Alexander-Universität Erlangen-Nürnberg• Human Factors in Security & Privacy Group─ Group leader
3Agenda• Spear phishing studies
─ Design & ethics
─ Study 1 ! piTalls & lessons learnt
─ Study 2 ! recommendaRons• Role of security awareness• Challenges in patching human vulnerabiliRes
4Technical vs. Human VulnerabiliRes• Technical vulnerabiliRes─ Found ! patch / redesign / accept risk

• Human vulnerabiliRes─ Know how to exploit
─ Do we know how to patch?
• Is security awareness THE soluRon?

5

Spear Phishing• Academic research: > 1000 papers since 2004
• Phishing as a service (PhaaS)─ KnowBe4, PhishMe, Wombat Security, many others
─ PentesRng the humans

6What don’t we know yet?

7Research QuesRons• Email vs. Facebook
─ Diﬀerence in clicking rates?
• Reasons for clicking and not clicking?
─ Why can some people protect themselves beeer than their peers?
─ Would knowing this provide useful informaRon for defenders?

8Study Idea
• Simulated aeack─ Send spear phishing messages with a link
─ Senders: non-exisRng persons
─ Recruit university students for parRcipaRng in the study• Email / Facebook• Measure clicking behavior• Ask them in a follow-up survey why they clicked / did not click
9Message
Hey <receiver’s ﬁrst name>,here are the pictures from the last week:
hep://<IP address>/photocloud/page.php?h=<USER ID>

accessdenied

Please do not share them with people who have notbeen there :-)See you next Rme!
<ﬁrstname of the sender>
10

Ethics: Recruitment
─ Don’t experiment with people without their consent!
─ ParRcipants recruited for a survey about “online behavior”• Not informed beforehand about the real purpose of the study─ IncenRve: win 10x10 EUR online shopping voucher─ Time: August/September 201311

Ethics: ConnecRng Behavior with Survey
Survey should be anonymous ! validity of the answers

send message
with individual link

wait Rll “enough” people clicked

send survey
with individual link

12

Final Design

send message
with individual link

wait 3 weeks

send anonymous survey
ask: clicked or not?

13

Study 1:Clicked
StaRsRcally signiﬁcant diﬀerence
100%
56%
50%

38%
89/15890/240email

Facebook0%
14

Study 1: Survey
Answered survey: 85% (339 out of 398)
100%

50%

45%
20%
179/398

68/339really clicked

reported that clicked

0%15

Study 2: Design Changes
On January 7th, 2014:Hey,
the New Year party was great! here are the pictures:
hep://<IP address>/photocloud/page.php?h=<USER ID>

send message
withindividual link

if clicked ! wait 24h
if did not click ! wait 7 days

send diﬀerent survey links
via email and on Facebook
ask: clicked or not?
16

Study 2: Clicked100%

StaRsRcally signiﬁcant diﬀerence
42.5%

50%

20%
0%

194/975email

119/280
Facebook17

Addressing by Name
Important via email, but not on Facebook?
Disclaimer: Study 1 ≠ Study 2!!!
! Diﬀerent messages
100%

StaRsRcally signiﬁcantNot signiﬁcant

56%
38%

50%

42.5%

20%
0%

Study 1: email

Study 2: email

Study 1: Facebook Study 2: Facebook18

Both Studies:
Factors Not StaRsRcally Correlated to Clicking• Gender of sender• Gender of receiver• Friend request on Facebook• Amount of informaRon on sender’s Facebook proﬁle19

20

Study 1 vs. Study 2: Survey Reliability
100%

50%

45%
20%

25%

Study 1: reported
that clicked

Study 2: actuallyclicked16%0%
Study 1: actuallyclickedStudy 2: reported
that clicked21

Study 1 vs. Study 2: Survey Reliability
100%

50%

45%
20%

25%

Study 1: reported
that clicked

Study 2: actuallyclicked

16%0%Study 1: actuallyclickedStudy 2: reported
that clicked
22

Study 2: Email vs. Facebook Survey Reliability
100%• Email: ok• Facebook: ???

50%

42.5%
20%

15.5%

18%

0%
Email: actuallyclicked

Email: reported that Facebook: actually Facebook: reported
clickedclickedthat clicked23

Reasons for Clicking: Results• Curiosity: 34%

24

“Curiosity”• “I was curious”• “I wanted to see what is there”• “Out of interest”• “I wanted to ﬁnd out more about the pictures”• “I did not know the sender, but wanted to see who is on the pictures”

25

Reasons for Clicking: Results(some people reported mulRple reasons)• Curiosity: 34%• Fits my New Year party: 27%• InvesRgaRon: 17%• Known sender: 16%• Trust into technical context: 11%26

“Trust Into Technical Context”• “My computer blocks access if there is a virus problem”• “I knew, if this was something dangerous, my Kaspersky would protect me”• “I use Firefox and MacOS, so I’m not afraid of the viruses”• “I used Tor Bundle”• “AOer I googled, photocloud seemed to be a clean website”• “I googled the email address […] I found nothing”• “IP came from the university”• “I consider the webmail of the university to be safe”
27

Reasons for Clicking: Results(some people reported mulRple reasons)• Curiosity: 34%• Fits my New Year party: 27%• InvesRgaRon: 17%• Known sender: 16%• Trust into system: 11%• Really pictures of me? 7%28

Reasons for Non-Clicking(some people reported mulRple reasons)• Unknown sender: 51%• Virus / Spam / Phishing / Scam / Fake: 44%• Does not ﬁt my New Year celebraRon: 36%• Does not ﬁt my way of life: 12%• InvesRgaRon: 6%
─ FB proﬁle: 2%29

Did Not Click Because Of Privacy (6%)• “It (the message) seemed to be private”• “I though the message was genuine and wanted protect privacy”• “It said: please don’t click if you don’t know me”• “The message was not for me”• “I did not see any reason to look up private pictures of a stranger who
obviously made a mistake”
30Factors Not StaRsRcally Correlatedwith Reported Clicking• IT security knowledge (self-assessed)• Knowledge that email sender can be spoofed• Knowledge that links can be dangerous

31

Avtude towards ParRcipaRon in the Study(-3=very negaRve, 3=very posiRve)

30%

20%
non-clickers
clickers

10%0%-3-2-1012332

Should Such Studies be Conducted in The Future?
2%

13%

85%

yesnonot sure33

LimitaRons• Study 1 ≠ Study 2
─ Only tentaRve comparisons across two studies!

• Validity of the reasons─ Cannot look into people’s heads at the moment of clicking• “reported clickers” ≠ “real clickers”

34Lesson 1: TargeRng• Curiosity / Interest
─ 78% knew that links can be dangerous• Context
─ Known sender• 82% knew that sender can be spoofed─ Plausibility: situaRon & expectaRons• Facebook: do people noRce that they clicked?
35

Lesson 2: Requirements on Users• Be suspicious:─ Even if you know the sender─ Even if the message ﬁts your current situaRon
─ Even if the message ﬁts your work and life pracRces• Be suspicious of everything!

36

DecepRon Mode

37

Let me introduce…
• Highly trained special agent• A lot of people want to kill him• (Almost) any person in his life can be a traitor• Has to be in decepRon mode in every life situaRon• Does his job excellently• Does not exist "38

Want Your Employees Be Aware of Spear Phishing?
• Want themto be in James Bond mode every Rme they read a message?

accounRngsales

human resources

public relaRons

customer support• Add this to job descripRons• Make sure to pay them adequately

39Being Security Aware: Personal Adventures

40

Personal Example 1: Curiosity / Interest(anonymized)

41

From: john.smith@turner.com
To: zinaida.benenson@fau.deSubject: CNN request -- about your upcoming Black Hat talkZinaida,John at CNN here. I’m the news network’s cybersecurity reporter. Here’s a link to my work, in caseyou’re not familiar with it.
I saw the descripRon of your upcoming Black Hat talk. Your topic looks fantasRc!
Can we get an exclusive look at your research and write the ﬁrst news story about it?
Cheers,John Smithjohn.smith@CNN.com

42

Personal Example 2: Context(anonymized)

43

From: Journal of Experiments (EXPE) exp@editorial-expe.com
To: zinaida.benenson@fau.deSubject: InvitaRon to Peer Review EXPE-M-35-00737
Dear Dr. Benenson, In view of your experRse […][…]If you would like to review this paper, please click this link:
hep://expe.editorial-expe.com/l.asp?i=35189&l=GKXKMQK
If you do not wish to review this paper, please click this link:
hep://expe.editorial-expe.com/l.asp?i=87665&l=6HN7KKBest regards,Editor<name I’ve never heard of>44

From: Journal of Experiments (EXPE) exp@editorial-expe.com
To: zinaida.benenson@fau.deSubject: InvitaRon to Peer Review EXPE-M-35-00737
Dear Dr. Benenson, In view of your experRse […][…]If you would like to review this paper, please click this link:
hep://expe.editorial-expe.com/l.asp?i=35189&l=GKXKMQ

If you do not wish to review this paper, please click this link:
hep://expe.editorial-expe.com/l.asp?i=87665&l=6HN7KKBest regards,Editor<name I’ve never heard of>45

First Click, Then NoRce: Messages to Helpdesk
D. Caputo et al. "Going spear phishing: Exploring embedded training and awareness.“
IEEE Security & Privacy Magazine, 2014

• “I clicked on it inadvertently without thinking and exited Explorer without
reading the link.”• “I just opened this. Then followed link like an idiot. Then killed the processusing Task Manager. Please advise as what to do.”• “I can’t believe I actually clicked on the link! Let me know if there’s
something I need to do to ensure my laptop isn’t infected, or if this is just aprank.”
46

Personal Example 3: An Aeachment(anonymized)

47

From: setup@company-I’m-dealing-with.com
To: zinaid.benenson@fau.deSubject: Message ID:23519-0297:FRT-92362. Workitem Number:
CMPVDM24062016157789020297

Aeachment: aeach/15072016/29375.docx
48Hi, Please see request details below. Please provide the required informaRon
by replying to this email.
Query Reason: Banking details
Workitem Number: CMPVDM24062016157789020297
Created Date: 15-Jul-2016
Name: Zinaida Benenson
Comments: Dear Sir/Madam In order for us to complete the set up of your account within
our system, we need your bank account details to which seelement of your invoices
should be made. Please complete the aeached form in full and return to us, ensuring it has
been signed by an authorized signatory.
49

Lesson 3: PentesRng & Patching Humans• What are the reasons for ineﬀecRveness of an awareness training?
─ Curiosity / interest ! natural & creaRve human traits
─ “This message ﬁts my current situaRon” / “I know the sender” ! useful decisional heurisRcs• What price users pay for an eﬀecRve awareness training?
─ James Bond mode
─ False posiRves? Work slowdown?
─ Breakdown of social relaRonships? Atmosphere of distrust?
─ Embarrassment? Shame? Anger?
50

Feasible User Involvement?
• Report suspicious messages?
─ Be prepared to get “amateur security”!
(Bruce Schneier about “If you see something, say something”)• Reliable indicators for switching into “James Bond mode”
─ False posiRves destroy trust into the indicator─ Digitally sign messages• Non-experts misinterpret meaning / don’t noRce• Can be social engineered into accepRng an invalid signature• Stop sending “phishy” legiRmate messages• Expect mistakes

51

Key Takeaways• Spear phishing: what defense is feasible and beneﬁcial for humans?
─ People won’t and can’t abstain from decisional heurisRcs
─ Don’t require permanent James Bond mode

• PentesRng and patching humans is tricky─ What do you want people to do?
─ Think about consequences for people & for company
─ Always ask consent• Talk to the users
─ Automated observaRon and measurement are not enough─ Ask directly about their experiences, opinions, work pracRces
52

Research & evidence needed!
If your company is interested, please talk to meZinaida Benenson
zinaida.benenson@fau.deThank you! QuesRons?
Please complete the Speaker Feedback Surveys
53