# Presentation: 037076

**Motivation & Intent**

- Antonio Sanfilippo
*Pacific Northwest National Laboratory*

## Motivation and Intent (M&I)

- M&I is the most recent thrust area within the* Threat Awareness Portfolio* at the Science and Technology Directorate, Dept. Homeland Security
- It includes 5 DOE Labs
- LLNL, LANL, ORNL, PNL, SNL
- Program started in Aug 2005
- Set up as an enduring capability with current funding plans to through FY11

## Mission and Vision

- Create a framework that supports analysts in assessing the threat of violence posed by intelligence targets
- Use theories from social and behavioral science to model ideological, organizational, and contextual factors associated with a group’s likelihood of engaging in violence

- Establish a social and behavioral science roadmap that adds enduring value to Terrorism Analytics
- Improve the predictive power of Intelligence & Security Informatics systems

## M&I Analytical Framework

*** ******All WMD***
*** ******ADVISE***
*** ******TVIS***
*** ******BKC***
*** ******RTAS***
*** ...***

**Transition to**
**user community**

**Content Analysis  and Knowledge**
**Extraction **

**Modeling & **
**Simulation**
**Integrated**
**Evidentiary, **
**Hypothetical and**
**Evolutionary Reasoning**

**Integration of**
**M&I **
**components**
** ****and **
**development of **
**a graphical **
**user interface **

**Intelligence & Security**
**Information systems**

**M&I Analytical Framework**

**Consultations with Subject Matter Experts**

**M&I Ontology**

**M&I Data Portal**

**Data Source Comparison**

**Global Terrorism Database**

## Gather and Formalize Knowledge

**Help Analysts identify social and behavioral theories of interests and retrieve related knowledge**
- Work with experts to formalize relevant social science theories
- Develop ontologies based on definitions of terms from the social and behavioral sciences
- Populate knowledge base using comprehensive and evolving social and behavioral ontologies

## Candidate Theories 

**Social Movement Theory**
- Social Identity Theory
- Collective Action Theory
- Deviant Legitimization Theory

- Attribution Theory
- Social Cognition Theory
- Group Dynamics Theory
- Prospect Theory
- ...

- Currently focusing on ***Social Movement Theory***

## Social Movement Theory

**mobilizing structures and resources**
- formal
- religious
- mosque
- Christian identity church
- non Government Organizations
- professional societies
- student associations
- political parties
- military organizations
- informal
- Social networks
**frames and framing**
- frame types
- diagnostic
- prognostic
- motivational
- frame processes
- resonance
- credibility
- reinforcement

**opportunities and constraints**
- political
- level of formal and informal access to political institutions
- level of access to decision-makers
- degree of system receptivity to change and challengers
- prevalence of allies
- prevalence of opponents
- stability of ruling elite coalitions
- nature of state repression
- state institutional capacity
- cultural
- social
- economic

## Social Movement Theory is ideal for studying group behavior

**LARGE**
- International
- System
- State
- _Group_
- Individual
**SMALL**

**  ****LOW**

**HIGH **

- LEVELS OF AGENCY

- U
- N
- I
- T
- OF
- A
- N
- A
- L
- Y
- S
- I
- S

*Structural *
*Theories*

_*Social Movement *_
_*Theory*_

*Rational Choice*
*Theory*

## M&I Ontology: Social Science Theories in OWL-DL

**M&I Ontology: Social**** ****Science Theories in OWL-DL**

## The M&I Ontology Provides Links to Relevant Social Science References

## Content Analysis

- Seek and validate evidence to support
- Population of the Ontology KB
- Modeling effort
- Develop an analysis of messaging
*Who is saying what to whom, how and why*
- Strategy
- Use Content Analysis tools such as Qualitative Data Analysis Software in conjunction with Information Extraction tools

## Explain the Clustering of Social Movements during Cycles of Protest

**Frames & Framing**
- Contentious ideas are socially processed through grammatical constructs and interpretive lenses
- Create inter-subjective meaning and facilitate movement goals
- Examples
- Diagnostic Frame: *Corruption is the problem!*
- Prognostic Frame: *Islam is the solution!*
- Motivational Frame: *Establishing an Islamic State is the responsibility of every Muslim!*

## Content Analysis Methodology

- Formulate research question
*Which frames are associated with which groups?*
- Harvesting and classify documents
- Identify and code relevant text segments
- Create seed patterns manually
- Expand seed patterns using semi-supervised methods
- Construct rules of inference to answer the question and validate the evidence established

## Creation of Seed Patterns

| Diagnostic
anti-government
frame | ORG, denounce, despotic, regime, terrorize, innocent, citizen, ... |
| --- | --- |
|  | <despotic regime>, <innocent citizen>, <honest citizen>... |
|  | SV(ORG,denounce), SV(regime, terrorize), OV(citizen,terrorize), ... |

## Expansion of Seed Patterns Using Semi-Supervised Learning 

| Seed Patterns (e.g. diagnostic anti-government frame) |  |
| --- | --- |
| Seed patterns + acquired patterns |  |
| Seed patterns+ acquired patterns |  |
| Seed patterns + acquired patterns |  |

- Data Set

- Data set – acquired patterns

- Data set – acquired
- patterns

- Data set –
-  acquired
- patterns

**Expansion of Seed Patterns Using Semi-Supervised Learning **

## Inference and Validation

- Estimate strength and confidence of target analytical constructs
- How much support is there for the evidence collected in the documents analyzed?

## Inference and Validation

## Modeling & Simulation

**M&I**
**Inferences**

**Hypotheses**

**Evidentiary, Hypothetical and Evolutionary**
**Reasoning**

**Content **
**Analysis**

**New **
**Observables**

- Will the recent XXX terrorist attack in XXX increase support for radical Islamic activism?
- What would happen if XXX operatives seized some strongholds in the XXX movement?
- What will the XXX movement look like in 20 years?

**Prior **
**Knowledge**

**New Data **
**Sources**

**Data **
**Sources**

**M&I Ontology**

## M&I Single Modeling Framework

**Bayesian networks & Prospective Analysis **(PNNL)
- Utilize social science evidence to test hypotheses based on both objective and subjective estimates of likelihood and impact
**Agent-Based Social Simulation **(LANL)
- Generate ensembles of potential future scenarios with drill down analysis into the causal factors of agent behaviors in the simulation
- Used by the evidentiary reasoning component as candidate likelihood distributions
**Markov Chain Monte Carlo Inverse methods **(LLNL)
- Integrated with the simulation and evidentiary networks to perform non-linear sensitivity analysis and validation studies

## Prospective Analysis*

- Indications-based estimation of outcomes from observed evidence
- Based on Heuer’s Analysis of Competing Hypotheses (ACH)

**Relate Indicators to Outcomes**

**Relate Evidence to indicators**

- *A. Turner & B. Hetzler

**Relate observations and indicators to outcomes**

## Bayesian Networks (BN)

- Adapted from Sticha et al. 2005

**Notes:**
NB: all from the PEP for 2006. Slight editing. This slide is contextual.

More from the PEP:

ABMS: simulations are particularly useful where relevant historical data is lacking or non-existent, and for examining new situations and scenarios. Simulations are also the only safe, possible, or non-expensive way to perform experiments on a complex system. Simulations are also well suited to understand the dynamics of why a group may choose violence over political action (R. Axtell, “Why Agents? On the Varied Motivations for Agent Computing in the Social Sciences”, November 2000, Center on Social and Economic Dynamics, Brookings Institute)

ACH: prospective analysis provides the analyst with an intuitive and widely accepted interface to the framework (based on Analysis of Competing Hypothesis theory).

The MCMC inverse methodology---by its ability to visit all states of the parameter space of a given model and to measure their effects on the model outcomes---constitutes a powerful sensitivity analysis tool.

## Comparing ACH and BN

| ACH disadvantages | Bayesian Network advantages |
| --- | --- |
| Does not address interactions and does not handle complex relational structures
Limited uncertainty management

Limited sensitivity analysis capabilities
Limited 1-dimensional “what-if” analysis capabilities
Limited support for uncovering user input inconsistencies
Ad hoc mathematical model | Natural handling of interactions and relational structures 
Focuses on uncertainty and its propagation
Advanced diagnostics/sensitivity criteria
Advanced multi-dimensional “what-if” analysis
Consistent assessments through probability theory 
Well founded mathematical model |
| Bayesian Network disadvantages | ACH advantages |
| Steep learning curve: Complex mathematics
Complex graphical layout 
Graph representation does not scale well
Unnecessary precise quantitative statements | No learning curve: does not require mathematical expertise
Simple layout
Scale very well with folding/expanding lists
Use efficient 5-, 7- Lickert scale  |

## Linking Bayesian Networks and Prospective Analysis

- Expand Prospective Analysis with more complex chains of inferential reasoning
- Put a well understood, formal calculation system at the heart of Prospective Analysis
- Facilitate Inference Network parameter input

**Notes:**
PEP Reference: Task 1.2.2.11.4.4.1.3

## Evidence Based Reasoning

**Content Analysis**

**Agent-Based Social Simulation Output**

**HUMINT**

_**Some Evidence Sources**_

- Use structural argumentation approaches to combine pieces of evidence with credentials (confidence and inferential strengths)