# Presentation: 089044

## Unleashing Your Inner Mother-In-Law:How to do an Adversarial Vulnerability Assessment

- Roger G. Johnston, Ph.D., CPP
- Vulnerability Assessment Team
- Los Alamos National Laboratory
- 505-667-7414     _[rogerj@lanl.gov](mailto:rogerj@lanl.gov)_
- http://pearl1.lanl.gov/seals.default.htm

- LAUR-05-5905

- Presentation for the 51st ASIS International Conference,
- Sept 11-15, 2005, Orlando, Florida

## LANL Vulnerability Assessment Team

- _Physical Security_
-   consulting
-   cargo security
-   tamper detection
-   training & curricula
-   nuclear safeguards
-   vulnerability assessments
-   novel security approaches
-   new tags & seals (patents)
-   unique vuln. assessment lab

-  The VAT has done detailed
-  vulnerability assessments on
-  hundreds of different security
-  devices, systems, & programs

-  The greatest of faults, I should say,
-   is to be conscious of none.
-           -- Thomas Carlyle (1795-1881)

- VAT

## bad guys
hackers
movie critics
peer reviewers
mothers-in-law

- c
- r
- i
- t
- i
- c
- a
- l
- r
- e
- v
- i
- e
- w

- bad guys
- hackers
- movie critics
- peer reviewers
- mothers-in-law

- Fault Finders:  They find problems because they want to find problems!

-  “Two mothers-in-law.”
-     -- Lord John Russell (1832-1900), on being asked what
-         he would consider proper punishment for bigamy.

- Behind every successful man is a proud
- wife and a surprised mother-in-law.
-        -- Hubert H. Humphrey (1911-1978)

- Conscience is a mother-in-law
- whose visit never ends.
-     -- H.L. Mencken (1880-1956)

- I haven’t spoken to my mother-in-law for
- eighteen months.  I don’t like to interrupt her.        		      -- Ken Dodd

## VA vs AVA 

**vulnerability assessment (VA):**               discovering and demonstrating ways to defeat a security device, system, or program.  Should include suggesting counter-measures and security improvements.
**adversarial vulnerability assessment (AVA):**  doing a more effective VA from first principles by truly wanting to find security problems, by thinking like the bad guys, and by letting them (not the good guys) define the security issues.

- VA vs AVA

- He that wrestles with us strengthens our skill.  Our antagonist is our helper.
-          -- Edmund Burke  (1729-1797)

- d
- e
- f
- i
- n
- i
- t
- i
- o
- n

## Other Reasons for Doing an AVA

-   mental rehearsal
-   fresh perspectives
-   fun/relieves tedium
-   increased alertness
-   bluffing (don’t underestimate)
-   enhanced sense of professionalism
-   educational/professional development for security staff
-   can involve other members of the organization, thus
-     increasing employees’ security awareness
-   can help justify additional resources for security

- Without deviation from the norm,
- progress is not possible.
-           -- Frank Zappa (1940-1993)

- benef
- i
- t
- s

## Security is Difficult!

- We need to recognize that security is difficult and there are no guarantees of success.
- Especially because complacency, over-confidence, wishful thinking, and arrogance    are not compatible with good security.

- Confidence is that feeling you sometimes have before you fully understand the situation.
-   	                   -- Anonymous

- not
- easy

## Why Security is Difficult

- The traditional performance measure for security is pathological:  success is often defined as nothing happening.
- Cost/Benefit analysis is difficult.
- There are few meaningful standards, fundamental principles, metrics, models, or theories.
- Society & employees often do not              like security.

- We spend all our time searching for security, and then we hate it when we get it.
-               -- John Steinbeck  (1902-1968)

- not
- easy

## 
Security managers & personnel aren’t always                      creative or proactive, but adversaries may be.

Adversaries and their resources are usually unknown to security managers, yet the adversaries understand the security systems. 

Objectives are often remarkably vague. 

- Security managers & personnel aren’t always                      creative or proactive, but adversaries may be.
- Adversaries and their resources are usually unknown to security managers, yet the adversaries understand the security systems.
- Objectives are often remarkably vague.

- You’ve got to be very careful if you don’t know       where you are going, because you might not get there.                                  			       -- Yogi Berra

- not
- easy

- Why Security is Difficult (con’t)

## 
Effective security management is highly multi-disciplinary:  engineering, computer science, psychology, sociology, management, economics, communication, & law.

Adversaries can attack at one point, but security managers may need to protect extended assets.

Adversaries need exploit only one or a small number of vulnerabilities, but security mangers must identify, prioritize, & manage many vulnerabilities, including unknown ones.

- Effective security management is highly multi-disciplinary:  engineering, computer science, psychology, sociology, management, economics, communication, & law.
- Adversaries can attack at one point, but security managers may need to protect extended assets.
- Adversaries need exploit only one or a small number of vulnerabilities, but security mangers must identify, prioritize, & manage many vulnerabilities, including unknown ones.

- not
- easy

- Why Security is Difficult (con’t)

- Evil is easy, and has infinite forms.
-            -- Blaise Pascal (1623-1662)

-  Evil will always triumph because good is dumb.
-    -- Rick Moranis, as Dark Helmet in *Spaceballs* (1987)

## 
Everything is a compromise & a tradeoff. 

Security functions are often tedious.

Security personnel have trouble identifying security vulnerabilities because they don’t want them to exist.  

   (It’s hard to think like the bad guys if you devote your career to being a good guy.)

- Everything is a compromise & a tradeoff.
- Security functions are often tedious.
- Security personnel have trouble identifying security vulnerabilities because they don’t want them to exist.
-    (It’s hard to think like the bad guys if you devote your career to being a good guy.)

- There is always more spirit in attack than in defense.
- 		-- Titus Livius (59 BC)

- not
- easy

- Why Security is Difficult (con’t)

## 
Physical Security scarcely a “field” at all!

	-  	You can’t (for the most part) get a degree in it.

	-  	Not widely attracting young people, females,                                         	the best & the brightest.

 	-  	Few peer-review, scholarly journals or R&D conferences.

 	-  	Lots of snake oil salesmen.

 	- 	Shortage of models, fundamental principles, metrics, rigor, standards, 
         guidelines, critical thinking, & creativity.

  	- 	Overly macho and often dominated by bureaucrats, committees, 		groupthink, “old boys” networks, linear/concrete/wishful thinkers.

- Physical Security scarcely a “field” at all!
- 	-  	You can’t (for the most part) get a degree in it.
- 	-  	Not widely attracting young people, females,                                         	the best & the brightest.
-  	-  	Few peer-review, scholarly journals or R&D conferences.
-  	-  	Lots of snake oil salesmen.
-  	- 	Shortage of models, fundamental principles, metrics, rigor, standards,
-          guidelines, critical thinking, & creativity.
-   	- 	Overly macho and often dominated by bureaucrats, committees, 		groupthink, “old boys” networks, linear/concrete/wishful thinkers.

- phys
- sec

- Why Security is Difficult (con’t)

- Is it ignorance or apathy?  I don’t know and I don’t care.    -- Jimmy Buffet

## Major Tools for Improving Security

- Security Survey (SS)
- Risk Management (RM)
- Design Basis Threat (DBT)
- Adversarial Vulnerability Assessment (AVA)

- “Who are you and how did you get in here?”
- ‘I’m a locksmith and I’m a locksmith.’
-      -- Leslie Nielsen as Lt. Frank Drebin, *Police Squad*

- tools

## SSs, RM, DBT, & AVAs

- Not really the same thing because they produce different results.
- The task of identifying Threats & Vulnerabilities, done as part of RM or DBT, is typically not really an AVA.
- SSs, RM, DBT were major breakthroughs & are still useful...   But they are not enough!

- Men do not like to admit to even momentary imperfection.
- My husband forgot the code to turn off the alarm.  When the
- police came, he wouldn't admit he'd forgotten the code...
- he turned himself in.            -- Rita Rudner

- tools

## Security Survey

- Basically a management walk around.
- Walk the spaces, looking for security problems.
- A checklist is often used.

- We made too many wrong mistakes.
-   	          -- Yogi Berra

- SS

## Limitations of Security Surveys

- Binary
- Close-ended
- Often unimaginative
- Not focused on adversaries
- Overly focused on the check list
- Does not encourage new countermeasures
- Expectation that problems will leap out at you

- 0 1

- It's better to be looked over than overlooked.
-        -- Mae West, *Belle of the Nineties*, 1934

- SS

## Risk Management

- Similar to Risk Management Techniques in other fields.
- Identify Assets, Threats & Vulnerabilities, Adversaries, Consequences, Safeguards & Countermeasures.
- Assign relative priorities and probabilities.     (Generate lots of tables.)
- Field your resources appropriately.

- RM

- If we don't succeed, we run the risk of failure.
- 		-- Dan Quayle

## Design Basis Threat

- “Design Basis Threat” is similar to Risk Management.
- DBT means “design your security to deal with the current real-world threats, adversaries, & their resources”.
- In practice, DBT tends to focus more on hardware and infrastructure than Risk Management does.

- A hypothetical paradox: what would happen in a battle between an Enterprise security team, who always get killed soon after appearing, and a squad of Imperial Stormtroopers, who can't hit the broad side of a planet?             -- Tom Galloway

- DBT

## Limitations of RM & DBT

- There is rarely any guidance on how to determine the Threats & Vulnerabilities other than looking at past security incidents.  But that is being reactive, not proactive.  Not good enough post-9/11, in a rapidly changing world, or for dealing with rare catastrophic events.
- Still binary & close-ended

- I skate to where the puck is going to be, not where it has been.
-                  		        -- Wayne Gretzky

- RM
- &
- DBT

## 

Often done unimaginatively 

Typically dominated by groupthink & bureaucrats

Not done from the perspective of the adversaries

The attack probabilities are usually a fantasy

Suffers from overconfidence in tables and the
	“fallacy of precision”

- Often done unimaginatively
- Typically dominated by groupthink & bureaucrats
- Not done from the perspective of the adversaries
- The attack probabilities are usually a fantasy
- Suffers from overconfidence in tables and the
- 	“fallacy of precision”

- 3.14159265359

- There's no sense in being precise when you don't even know what you're talking about.
-             -- John von Neumann (1903-1957)

- More Limitations of RM & DBT

- RM
- &
- DBT

## Tendency to let the good guys and existing security measures define the adversaries & attack modes

Often used to justify the status quo--typically does not encourage new countermeasures

Ignores simple/cheap countermeasures when the attack probabilities are judged (rightly or wrongly) to be low or zero

- Often used to justify the status quo--typically does not encourage new countermeasures
- Ignores simple/cheap countermeasures when the attack probabilities are judged (rightly or wrongly) to be low or zero

- It isn't that they can't see the solution.
- It is that they can't see the problem.
- 	-- G.K. Chesterton,  *The Scandal*
*	**    **of Father Brown * (1935)

- More Limitations of RM & DBT

- RM
- &
- DBT

## Adversarial Vulnerability Assessment

- Perform a mental coordinate transformation                         and pretend to be the bad guys.  (This is                       a lot harder to do than one might think.)
- Gleefully look for trouble, rather than seeking to reassure yourself that everything is fine.
- Unlike other techniques, don’t let the good guys define the problem or its parameters.

- It is sometimes expedient to forget who we are.
- 	        -- Publilius Syrus (~42 BC)

- AVA

## Example:  Open Window

**risk management:**  ignore if not envisioned as part of a specific threat or attack from a likely adversary;  otherwise, design procedure to close & lock window.
**AVA:**  Oh boy, an open window!                What mischief can this lead to?

- You can observe a lot by just watching.
- 	                -- Yogi Berra

- compare

## 
WR Floyd, Security Surveys (1995). 

JF Broder, Risk Analysis and the Security Survey (1999).

CA Roper, Risk management for Security Professionals (1999).

ML Garcia, The Design and Evaluation of Physical Protection Systems (2001). 

- WR Floyd, *Security Surveys *(1995).
- JF Broder, *Risk Analysis and the Security Survey* (1999).
- CA Roper, *Risk management for Security Professionals* (1999).
- ML Garcia, *The Design and Evaluation of Physical Protection Systems* (2001).

**Recommended References**

- Recommended References:  Conventional Security Surveys & Risk Management

- more
-  info

## 
K Hogan, The Psychology of Persuasion (1996).

D Goleman, Vital Lies Simple Truths: The Psychology of Self Deception (1996).

DL Smith, Why We Lie: The Evolutionary Roots of Deception and the Unconscious Mind (2004).

KD Mitnick, WL Simon, & S Wozniak, The Art of Deception: Controlling the Human Element of Security (2002).

T Hoving, False Impressions (1997).

- K Hogan, *The Psychology of Persuasion* (1996).
- D Goleman, *Vital Lies Simple Truths: The Psychology of Self Deception* (1996).
- DL Smith, *Why We Lie: The Evolutionary Roots of Deception and the Unconscious Mind* (2004).
- KD Mitnick, WL Simon, & S Wozniak, *The Art of Deception: Controlling the Human Element of Security* (2002).
- T Hoving, *False Impressions* (1997).

- Recommended References:  Social Engineering, Fakery, Wishful Thinking, & Deception

- more
-  info

## AVA Steps

- Fully understand the device, system, or program and how it is REALLY used.      Talk to the low-level users.
- Play with it.
**Brainstorm--anything goes!**
- Play with it some more.

- I hope you believe you understand what you
- think I said, but I’m not sure you realize that
- what you’ve heard is not what I meant.
- 	      -- Richard Nixon (1913-1994)

- A
- V
- A
- s
- t
- e
- p
- s

## 
 	Edit & prioritize potential attacks.

 	Partially develop some attacks.

 	Determine feasibility of the attacks.

 	Devise countermeasures.

-  	Edit & prioritize potential attacks.
-  	Partially develop some attacks.
-  	Determine feasibility of the attacks.
-  	Devise countermeasures.

- In theory there is no difference between
- theory and practice.  In practice there is.
- 	 	-- Yogi Berra

- A
- V
- A
- s
- t
- e
- p
- s

- AVA Steps

## 
Perfect attacks.

Demonstrate attacks.

Rigorously test attacks.

Rigorously test countermeasures.

- Perfect attacks.
- Demonstrate attacks.
- Rigorously test attacks.
- Rigorously test countermeasures.

- A
- V
- A
- s
- t
- e
- p
- s

- AVA Steps

- After the meek inherit the Earth, I think we
- should just kick their butts and take it from them.      		     -- Jim Rosenburg

## 
You must be more creative and imaginative than your adversaries!

They only need to stumble upon one vulnerability, but you have to worry about all of them!

- You must be more creative and imaginative than your adversaries!
- They only need to stumble upon one vulnerability, but you have to worry about all of them!

- Effective Brainstormingis the Key!

- B
- S
- i
- n
- g

- Sanity is a one trick pony--all you have is rational thought.       But when you're good and loony, the sky's the limit!                   				 -- The Tick

## The Need for Creativity

-     Due to the rapid changes in the complexity of both technology and organizations over the past two decades, historical data has become less significant.  Risk measurement and the identification of consequences require a combination of experience, skills, imagination,   and creativity.  This emphasis on subjective measurements is borne out in practice...
-           -- David McNamee, *Business Risk Management*, (1998), p. 43

- The future ain't what it used to be.  -- Yogi Berra

- imaginat
- ion

- It’s a poor sort of memory that only works backwards.
-      -- Lewis Carroll (1832-1898), *Alice in Wonderland*

## Delaying Judgment

-     Nothing can inhibit and stifle the creative process more--and on this there is unanimous agreement among all creative individuals and investigators of creativity--than critical judgment applied to the emerging idea at the beginning stages of the creative process. ... More ideas have been prematurely rejected by a stringent evaluative attitude than would be warranted by any inherent weakness or absurdity in them.  The longer one can linger with the idea with judgment held in abeyance, the better the chances all its details and ramifications [can emerge].
- 	--  Eugene Raudsepp, *Managing Creative Scientists and Engineers* (1963).

- Keep the possibility phase completely separate from the practicality phase!

- B
- S
- i
- n
- g

- We all know your idea is crazy.  The question is,
- is it crazy enough?      -- Niels Bohr (1885-1962)

## Individuals are creative, not groups...

but the right group dynamics can energize, egg-on, & fertilize individuals...

and a group is usually necessary to fully explore attacks & countermeasures.

- but the right group dynamics can energize, egg-on, & fertilize individuals...
- and a group is usually necessary to fully explore attacks & countermeasures.

- Realities of Creativity

- Could Hamlet have been written by committee, or the Mona Lisa painted by a club?  Could the New Testament have been composed as a conference report?  Creative ideas don't spring from groups.  They spring from individuals.
-                           -- Alfred Whitney Griswold (1885-1959)

- B
- S
- i
- n
- g

##   Individuals must be given ownership of their original idea 
   & should be personally recognized for their creativity.

	The group environment needs to be: 	
diverse
high-energy 
urgent but not stressful 
humorous, joyful, & fun
cohesive but not too cohesive
competitive in a friendly & respectful way
enthusiastic about individual differences & eccentricities

  Every idea, no matter how wacky or stupid, gets written
   down & treated as a gem.

-    & should be personally recognized for their creativity.
- 	The group environment needs to be:
- diverse
- high-energy
- urgent but not stressful
- humorous, joyful, & fun
- cohesive but not too cohesive
- competitive in a friendly & respectful way
- enthusiastic about individual differences & eccentricities
-   Every idea, no matter how wacky or stupid, gets written
-    down & treated as a gem.

- Realities of Brainstorming

- A conference is a gathering of
- important people who singly
- can do nothing, but together can
- decide that nothing can be done.
-          -- Fred Allen  (1894-1956)

- B
- S
- i
- n
- g

##  	Authority figures should not be involved, or at  	least should not act like authority figures.

	A good model:  comedy writing.

- 	A good model:  comedy writing.

- Realities of Brainstorming

- My parents didn't want to move to Florida,
- but they turned 60, and it was the law.
- 	                  -- Jerry Seinfeld

- B
- S
- i
- n
- g

## Brainstorming  -  Sid Caesar

-    -- Sid Caesar and Eddy Friedfeld, *Caesar’s Hours: My Life in Comedy, with Love and Laughter* (2003), p 123.

- B
- S
- i
- n
- g

- To stimulate creativity, one must develop the childlike inclination for play and the childlike desire for recognition.
- 		       -- Albert Einstein (1879-1955)

## Brainstorming  -  Sid Caesar (con’t)

-    -- Sid Caesar and Eddy Friedfeld, *Caesar’s Hours:  My Life in Comedy, with Love and Laughter* (2003), pp 121-122.

- Brainstorming  -  Sid Caesar (con’t)

- B
- S
- i
- n
- g

- You’re only given a little spark
- of madness.  You mustn’t lose it.
-                    -- Robin Williams

## Security Brainstorming Tips

- Pay close attention to explicit or unstated assumptions, and to security features that are widely praised or admired.  These are often the source of serious vulnerabilities.
- Concentrate on the 2nd and 3rd best attacks or countermeasures.  You are likely overlooking something that would make them the best solutions.
- If there is widespread agreement about the efficacy of an attack or countermeasure, re-examine.  Something important was probably overlooked.

- If everybody is thinking alike,
- then nobody is thinking.
-    -- George S. Patton (1885-1945)

- t
- i
- p
- s

## Security Brainstorming Tips

- Quantity breeds quality.
- With all ideas:  elaborate, expand, modify, subvert, exaggerate, & combine with other ideas.  Pursue hunches & intuition.
- Keep a written record of ideas.  Flip charts?
- The best ideas come late, and when you are not thinking about the problem.

- The best way to have a good
- idea is to have lots of ideas.
-       -- Linus Pauling (1901-1994)

- Out of nowhere the idea will appear.  It
- will come to you when you least expect it.
-    -- James Webb Young, *A Technique for Producing Ideas*

- t
- i
- p
- s

## Security Brainstorming Tips

- Think about which rules could be broken to provide better security, or to execute better attacks.
- Think backwards:  How can we make security completely ineffective?  How can our attacks fail miserably?  How do we as the bad guys escape from the facility _after_ completing our attack?
- Pursue what is interesting, controversial,                   contrarian, exciting, or silly.
- Solve the problem that isn’t here yet.

- t
- i
- p
- s

- Now that the world is getting over the initial shock, and the war against terrorism has begun, what now for bridal retailers?                    	-- Actual editorial in the trade magazine *Bridal Buyer*

## Security Brainstorming Tips

- Mentally remove some security devices, measures, or personnel.  Then consider the implications.
- What if Albert Einstein, Chris Rock, or 		Frankenstein were in charge of security?       What would they do differently?
- What if Godzilla, your car mechanic, or the Boston Philharmonic Orchestra were the bad guys?  How would they attack?
- Draw lots of diagrams.

- t
- i
- p
- s

- If people don’t want to come to the ballpark, how are you going to stop them?          -- Yogi Berra

## Security Brainstorming Tips

- Develop and explore models, metaphors, & analogies.
- Terminology constrains our thinking.  Rename everything in your own (and/or silly) words, and think about them in light of the new terminology.
- Consider different verbs for what the bad guys might want to accomplish:  attack, steal, demolish, embarrass, tag, terminate, uncover, purify, whistleblow, poison, etc.

- t
- i
- p
- s

- The problem with the French,
- is that they don’t have a word for
- ‘entrepreneur’.  -- George W. Bush

- Why is it the ‘opposite’ sex,
- and not the ‘other’ sex?    -- Anon

## Security Brainstorming Tips

- Picture attacks from the ceiling and the floor.  How would attacks work if everything was underwater, or if gravity stopped working?
- Ridicule existing security measures & strategies.
- Explore extremes:  best & worst case scenarios for both the good guys and the bad guys.
- Be wary of fault tree analysis!  Most security failures are NOT the result of a sequence of stochastic failures at different points in the system.

- t
- i
- p
- s

- There’s a fine line between fishing and
- just standing on the shore like an idiot.
-                                -- Steven Wright

## Security Brainstorming Tips

- How will the bad guys *feel* during the attack?  Try to imagine their satisfaction if they succeed.
- How would the bad guys attack if they had infinite resources?  Almost no resources?
- What would the security look like if it had infinite resources?  Almost no resources?
- What questions would a 10-year old ask?
- What questions would your mother-in-law ask?

- t
- i
- p
- s

- I think the inventor of the piñata may
- have had some unresolved donkey issues.
-                                  -- Dan Johnson

## AVA Personnel

- Ideally, involve outsiders!

- To see what is in front of one's nose needs a constant struggle.
-    -- George Orwell (1903-1950)

- Vulnerabilities are often
- obvious to outsiders:

- people

## AVA Personnel (con’t)

- Also involve smart, hands-on, creative people inside your organization, including those who are not associated with security.
- _Seek_:  nonconformists, wise guys, trouble makers, smart alecks, schemers, organizational critics, loophole finders, questioners of tradition and authority, outside-the-box thinkers, artists, hackers, tinkerers, problem solvers, & techno-nerds.

- AVA Personnel (con’t)

- people

- Some people see things that are and ask, Why?
- Some people dream of things that never were and ask, Why not?
- Some people have to go to work and don’t have time for all that.
-                                                         -- George Carlin

## 
TM Amabile, “How to Kill Creativity”, Harvard Business Review, September 1, 1998.

A Hargadon, “Brainstorming Groups in Context”, Administrative Science Quarterly, December 1, 1996.

R van Oech, A Whack on the Side of the Head: How You Can be More Creative (1998).

RL Ackoff and S Rovin, Beating the System: Using Creativity to Outsmart Bureaucracies (2005).

D Dauten, The Laughing Warriors:  How to Enjoy Killing the Status Quo (2003).

M Michalko, Cracking Creativity: The Secrets of Creative Genius (2001).

- TM Amabile, “How to Kill Creativity”, *Harvard Business Review*, September 1, 1998.
- A Hargadon, “Brainstorming Groups in Context”, *Administrative Science Quarterly*, December 1, 1996.
- R van Oech, *A Whack on the Side of the Head: How You Can be More Creative* (1998).
- RL Ackoff and S Rovin, *Beating the System: Using Creativity to Outsmart Bureaucracies* (2005).
- D Dauten, *The Laughing Warriors:  How to Enjoy Killing the Status Quo* (2003).
- M Michalko, *Cracking Creativity: The Secrets of Creative Genius* (2001).

**Recommended References**

- Recommended References:  Brainstorming & Creativity

- more
-  info

## General Attributes of Effective AVAs

- No conflicts of interest or wishful thinking.
- No “Shoot the Messenger” Syndrome.  No retaliation or punishment against assessors, security personnel, or managers when vulnerabilities are found.
- Use of independent, imaginative assessors who are psychologically predisposed to finding problems and suggesting solutions, and who (ideally) have a history of doing so.

- To show resentment at a reproach is to acknowledge that one may have deserved it.        	      -- Tacitus  (55-117 AD)

- good
- AVA

## 
 	No binary view of security.

 	Rejection of a finding of zero vulnerabilities.

 	Rejection of the idea of “passing” the VA,                    	or of VAs as “certification”.

 	Discovering vulnerabilities is viewed as good            	(not bad) news.

-  	No binary view of security.
-  	Rejection of a finding of zero vulnerabilities.
-  	Rejection of the idea of “passing” the VA,                    	or of VAs as “certification”.
-  	Discovering vulnerabilities is viewed as good            	(not bad) news.

- When we were children, we used to think that    when we were grown-up we would no longer be vulnerable.  But to grow up is to accept vulnerability...  To be alive is to be vulnerable.
- 		-- Madeleine L'Engle

- good
- AVA

- General Attributes of Effective AVAs

## 
Done early, iteratively, and periodically.

Done holistically, not by component, sub-system, function, or layer.  (Attacks often occur at interfaces.)

 No unrealistic time or budget constraints on the  AVA, or on what attacks or adversaries can be  considered.

 Done in context.

- Done early, iteratively, and periodically.
- Done holistically, not by component, sub-system, function, or layer.  (Attacks often occur at interfaces.)
-  No unrealistic time or budget constraints on the  AVA, or on what attacks or adversaries can be  considered.
-  Done in context.

- He that will not apply new remedies must expect new evils;
- for time is the greatest innovator.
- 	              -- Francis Bacon (1561-1626)

- good
- AVA

- General Attributes of Effective AVAs

## 
No underestimation of the cleverness, knowledge,        skills, dedication, or resources of adversaries.

The good guys don’t get to define the problem, the bad guys do.

Simple, low-tech attacks are examined first.

- No underestimation of the cleverness, knowledge,        skills, dedication, or resources of adversaries.
- The good guys don’t get to define the problem, the bad guys do.
- Simple, low-tech attacks are examined first.

- good
- AVA

- General Attributes of Effective AVAs

- Do not touch anything unnecessarily.  Beware of pretty girls in dance halls and parks who may be spies, as well as bicycles, revolvers, uniforms, arms, dead horses, and men lying on roads--they are not there accidentally.
-                                -- Soviet infantry manual from the 1930's

## 
Rohrbach’s Maxim must be considered:  No security system will ever be used properly (the way it was designed) all the time.

Shannon’s (Kerckhoffs’) Maxim must be considered:  The adversaries know and understand the security systems, strategies, and hardware being used.

- Rohrbach’s Maxim must be considered:  No security system will ever be used properly (the way it was designed) all the time.
- Shannon’s (Kerckhoffs’) Maxim must be considered:  The adversaries know and understand the security systems, strategies, and hardware being used.

- Inanimate objects can be classified scientifically into three major categories;
- those that don't work, those that break down, and those that get lost.
- 			  -- Russell Baker

- Everything secret degenerates ... nothing is safe that does not show how it can bear discussion and publicity.
- 		        -- attributed to Lord Acton  (1834-1902)

- good
- AVA

- General Attributes of Effective AVAs

## 
 The following attacks are all considered:

		•  terrorism
		•  sabotage
		•  espionage 
		•  false alarming
		•  poke the system
		•  wait & pounce
		•  backdoor attacks
		•  impersonation
		•  social engineering
		•  tampering with security training
		•  insiders, outsiders, insiders + outsiders

-  The following attacks are all considered:
- 		•  terrorism
- 		•  sabotage
- 		•  espionage
- 		•  false alarming
- 		•  poke the system
- 		•  wait & pounce
- 		•  backdoor attacks
- 		•  impersonation
- 		•  social engineering
- 		•  tampering with security training
- 		•  insiders, outsiders, insiders + outsiders

- good
- AVA

- General Attributes of Effective AVAs

- When choosing between two evils, I
- always pick the one I never tried before.
- 	        -- Mae West (1893-1980)

## Keep in mind:  A security device, system, or strategy will tend to be most vulnerable near the end of its life. 

Don’t forget about the physical security of computers, peripherals, and computer media!

- Don’t forget about the _physical_ security of computers, peripherals, and computer media!

- good
- AVA

- General Attributes of Effective AVAs

- It does little good to have great computer security if wiring closets are easily accessible or individuals can readily walk into an office and sit down at a computer and gain access to systems and applications.  Even though the skill level required to hack systems and write viruses is becoming widespread, the skill required to wield an ax, hammer, or fire hose and do thousands of dollars in damage is even more widely held.            -- Michael Erbschloe, *Physical Security for IT* (2005)

##            20.   Avoid common fallacies, such as:

All the threats & vulnerabilities will be discovered.
When we are done, they will all be neutralized.
We won’t have to think about them until the next VA.
Future VAs can be cursory once we do a good one. 
Some software package can do most of the work for us.
Only security experts with many years of experience should participate.
Whatever is commonly done in our industry is enough. 
Multiple layers of mediocre security = good security.
Compliance = Good Security. 
High-Tech = High Security.
Inventory & Security are the same thing.

- All the threats & vulnerabilities will be discovered.
- When we are done, they will all be neutralized.
- We won’t have to think about them until the next VA.
- Future VAs can be cursory once we do a good one.
- Some software package can do most of the work for us.
- Only security experts with many years of experience should participate.
- Whatever is commonly done in our industry is enough.
- Multiple layers of mediocre security = good security.
- Compliance = Good Security.
- High-Tech = High Security.
- Inventory & Security are the same thing.

- Attributes of Effective AVAs (con’t)

- good
- AVA

##     Confusing High-Tech with High-Security, & Inventory with Security

**    ****Confusing High-Tech with High-Security, & Inventory with Security**

-    GPS
-    RFIDs
-    tamper-indicating seals
-    contact memory buttons
-    tamper-evident packaging
-    data encryption/verification
-    access control devices (including biometrics)
-   “Don’t Swallow the Snake Oil:
-   Understanding the Vulnerabilities &
-   Limitations of High-Technology
-    Wed 4:30-5.30 PM

- more
-  info

- If you think technology can solve your security problems, then you don't understand the problems and you don't understand the technology.
-                       -- Bruce Schneier

## 

Thinking about vulnerabilities & countermeasures does not end when the AVA is officially over!

 Don’t overlook or under-estimate the  		insider threat, especially from 				disgruntled employees.   

- Thinking about vulnerabilities & countermeasures does not end when the AVA is officially over!
-  Don’t overlook or under-estimate the  		insider threat, especially from 				disgruntled employees.

- good
- AVA

- General Attributes of Effective AVAs

- Honesty may be the best policy, but it's important to
- remember that apparently, by elimination, dishonesty
- is the second-best policy.
**	****	****  **-- George Carlin

##   Disgruntled Workers

- 	Research shows that employee disgruntlement
- 	is associated with perceptions of unfairness &
- 	inequity, not necessarily objective conditions.
-  	Disgruntled employees are known to be a risk for
- 	workplace violence, espionage, theft, & sabotage.

- We have met the enemy and he is us.
-             -- Walt Kelly, the words of Pogo 	in Earth Day 1971 cartoon strip

- not
- grunt
- l
- ed

- Actual courtroom testimony:
- Q:  James shot Tommy Lee?
- A:  Yes.
- Q:  Then Tommy Lee pulled out his gun and shot James in the fracas?
- A:  No, sir, just above it.

##      Workplace Violence (USA)

-   ~ 1.7 million victims of workplace violence
-  	each year
- 	>800 workers killed each year due to
- 	workplace homicide
- 	Homicide is the number one cause of
- 	on-the-job deaths for females
-  Source:  NIOSH

- Always go to other people’s funerals.  Otherwise
- they might not come to yours.    -- Yogi Berra

- not
- grunt
- l
- ed

## Disgruntlement Countermeasures

-  	Listen, acknowledge, validate, & empathize with employees.
-  	Allow employees to freely offer suggestions & concerns.
-  	Have _legitimate_ complaint resolution processes.  Too often these 	are non-existent, ineffective, adversarial, or fraudulent, especially 	in large or bureaucratic organizations.  This is very dangerous 	(and bad for productivity).
-  	Be aware that employee _perceptions_ about fairness
- 	are the only reality.
- 	Treat departing employees & retirees well.

- Sincerity is everything.  If you can fake that,
- you've got it made.
-                        -- George Burns (1896-1996)

- not
- grunt
- l
- ed

## 
It must be clear that the AVA will produce more suggestions & countermeasures than are likely to be implemented.  Security mangers (not the assessors) should ultimately decide which (if any) make sense to employ.

Findings are reported to the highest appropriate level without editing, inter-pretation, or censorship by middle managers.

- It must be clear that the AVA will produce more suggestions & countermeasures than are likely to be implemented.  Security mangers (not the assessors) should ultimately decide which (if any) make sense to employ.
- Findings are reported to the highest appropriate level without editing, inter-pretation, or censorship by middle managers.

- report

- The AVA Report

- The problem is not that there are problems.
- The problem is expecting otherwise and thinking that having problems is a problem.
-                             -- Theodore Rubin

## 
No confusion about the difference between AVAs and other kinds of testing (materials, environmental, ergonomic, field readiness, personnel, compliance).

The vulnerability assessors need to praise the good things because:
+  We want the good things to be recognized and to continue.
+  Security managers need to be willing to arrange for future VAs.
+  Discussing the good things will make security managers more willing to hear about potential problems.

- No confusion about the difference between AVAs and other kinds of testing (materials, environmental, ergonomic, field readiness, personnel, compliance).
- The vulnerability assessors need to praise the good things because:
- +  We want the good things to be recognized and to continue.
- +  Security managers need to be willing to arrange for future VAs.
- +  Discussing the good things will make security managers more willing to hear about potential problems.

- The AVA Report

- report

- Honest criticism is hard to take, particularly from
- a relative, a friend, an acquaintance, or a stranger.
- 		-- Franklin B. Jones

## 
 5.   The report needs to include:

-  5.   The report needs to include:

- The AVA Report

- report

-   identity & experience of the assessors
-   any conflict of interest
-   any *a priori* constraints
-   time & resources used
-   samples, details, and/or demonstration of attacks
-   time, expertise, & resources required by an
-        adversary to execute the attacks
-   possible countermeasures
-   a sanitized, statistical summary of the findings &
-        identity of the assessors if the sponsor wishes to
-        take public credit for the AVA.

- I don’t care what is written
- about me as long as it isn’t true.
-     -- Katherine Hepburn (1907-2003)

## 35 Attributes of Flawed Security Programs

-  	Widespread arrogance & overconfidence.
-  	Security is viewed as binary.  (This inhibits   	improvement.)
-  	Insiders are not viewed as a threat.
-  	Overly focused on paperwork, auditors,  		regulations, & formality.
-  	Security & security managers are micro- 	 	managed by unqualified business 			executives.

## Attributes of Flawed Security Programs (con’t)

- Security problems, vulnerabilities, & incidents are covered-up.
- Vulnerability assessment are rare;  security is rarely tested.
- “What if?” mental or walk-through exercises are rare, instead of being done daily or weekly.

- Attributes of Flawed Security Programs (con’t)

## Attributes of Flawed Security Programs (con’t)

-  10. 	Security personnel receive little training or
- 		practice, and are given few opportunities for
- 	 	professional advancement.
-  11.	Security supervisors & managers are not 	well respected by subordinates.
-  12.	Security managers rarely chat informally 		with regular (non-security) employees.
-  13.	Security personnel are not well respected 	by non-security personnel.

- Attributes of Flawed Security Programs (con’t)

## Attributes of Flawed Security Programs (con’t)

- 14.	 The morale and self-esteem of security 			personnel is low.  Appearance is poor.
- 15. 	Low-level security personnel are treated poorly.
- 16. 	Low-level security personnel are rarely 		recognized for good work.
- 17. 	Security training exercises are unrealistic & 	tedious.
- 18.	Security personnel have few opportunities to
- 	demonstrate their prowess in contests/exercises.

- Attributes of Flawed Security Programs (con’t)

## Attributes of Flawed Security Programs (con’t)

- 19. 	Security personnel feel no loyalty or  		connection to their employer, or to the  		employees and the organization they are  	protecting.
- 20. 	The organization lacks a fair and effective
- 	grievance or complaint resolution process
- 	for disgruntled employees (whether security 	or non-security personnel).

- Attributes of Flawed Security Programs (con’t)

## Attributes of Flawed Security Programs (con’t)

- 21. 	Security personnel are not briefed at the start
- 	of a shift, nor checked for fitness of duty.
- 22. 	Security personnel are not debriefed after  	their shift.
- 23. 	No pre-employment screening of  			employees;  no periodic, thorough 	  		background and reliability checks performed  	on security and other critical personnel.

- Attributes of Flawed Security Programs (con’t)

## Attributes of Flawed Security Programs (con’t)

- 24. 	Unexplained or unexpected absences of
- 			security personnel are not investigated, nor are 	sudden outbreaks of widespread illness.
-  	Sources of food and drink are not secure.  		Critical security personnel accept food & drink
- 			from colleagues, co-workers, and even the public.
-   26. 	Rosters, duty assignments, & schedules of
- 			authorized work are not well protected from 			tampering.  Paper documents and verbal orders		for security personnel are taken at face value.

- Attributes of Flawed Security Programs (con’t)

## Attributes of Flawed Security Programs (con’t)

- 27. 	Security personnel do not know exactly how  			& when to summon help or sound an alarm.
- 28. 	There are no clear policies on the use of 			physical force (including lethal force and  			force 	against coworkers), or else those  				policies are largely unknown to security  				personnel and rarely discussed in a “what  			if?” format.
- 29. 	Security personnel are vague on exactly  			what is expected of them.

- Attributes of Flawed Security Programs (con’t)

## Attributes of Flawed Security Programs (con’t)

- 30. 	The health and safety of security personnel is
-  		a low priority. Insurance and medical  			coverage is absent or poor.
- 31. 	VIPs are allowed to bypass standard security
-  		procedures.
- 32. 	Security managers are automatically fired  			when there is a major security incident.    			Low-level security personnel are 				automatically disciplined or fired when
- 		there is a minor security incident.

- Attributes of Flawed Security Programs (con’t)

## Attributes of Flawed Security Programs (con’t)

-  	Security awareness training for non-		security personnel is boring, insipid, 		insulting, & threatening.  It doesn’t 		emphasize why security is important to 		them.
- 35. 	Security rules are put in place with little  	thought, few sanity checks, and little input  	from the people affected.

- Attributes of Flawed Security Programs (con’t)

## The LANL

- You can request a copy at _[rogerj@lanl.gov](mailto:rogerj@lanl.gov)_

**The LANL**
**Vulnerability Assessment Team**

- http://pearl1.lanl.gov/seals/default.htm

- Roger Johnston, Ph.D., CPP,  Ron Martinez,  Leon Lopez,      Sonia Trujillo,  Adam Pacheco,  Anthony Garcia,
- Jon Warner, Ph.D.,  Alicia Herrera,  Eddie Bitzer, M.A.

- Ring the bells that still can ring.
- Forget your perfect offering.
- There is a crack in everything.
- That's how the light gets in.
-                          -- Anonymous

- more
-  info

## Other Issues Covered on the CD

- Q: 	Can AVA techniques be used to improve safety, not just security?
- A:	 	“Adversarial” Safety Analysis
- Q: 	How, to whom, and in what detail do you disclose security 			vulnerabilities that affect others?
- A:		Vulnerability Disclosure Index (0-100%)
- Q:  	How can we reduce security guard turnover?
- A: 	Tools from Industrial/Organizational Psychology
- Q:	 	How can cargo security be improved?
- A:		Better tamper-indicating seals, use protocols, and new cargo
- 		monitoring techniques
- Q:		How can we counter pharmaceutical counterfeiting w/o RFIDs
- 		(which don’t provide security or involve consumers)?
- A:		Numeric tokens

- more
-  info