# Presentation: 250908

## Distributed Cognition in the Pacific Rim

**Prof. Tom Furness**
**Human Interface Technology Lab**
**University of Washington**

- PARVAC

## before....

## after...

## The power of vision...

**The power of vision...**

## <footer>

- <date/time>

## adaptive system element...

## Distribution of receptors in eye

**Distribution of receptors in eye**

- Rod and cone densities as a function of eccentricity along the horizontal meridian.

## Super Cockpit revisited

## <footer>

- <date/time>

## Slide 11

## egocentric vs. exocentric views

## finger in god’s eye display

## Slide 14

## What we learned

**We have more power than we think**
**But the interface makes all the difference!**** **
**Virtual interfaces really work**** **
**Accelerated learning when immersed**
**>> 60 degrees**
**Situation awareness**

## But are we making progress?

## Distribution of receptors in eye

**Distribution of receptors in eye**

- Rod and cone densities as a function of eccentricity along the horizontal meridian.

## ...the most important interface

**= the last few inches**

*How to get bandwidth *_***to***_* and *_***from***_* the brain** *
*How to get bandwidth *_***between***_* brains*

## Pacific Rim Visualization and Analysis Center

## PACIFIC RIM

**PACIFIC RIM**

## WATER!!

**WATER!!**

## FIRE!!

**FIRE!!**

## http://earthquake.usgs.gov/eqcenter/recenteqsww

## Pacific Rim Threats

**Natural**
**Earth**
**Earthquakes**
**Volcanoes**
**Water**
**Tsunami**
**Waterways**
**Birds & fish**
**Migration patterns**
**Man-made**
**Ports of entry**
**Shipping**
**people**
**Lots of borders**
**Agri-terrorism**

***Ring of Fire!***

## PARVAC Mission 

**To advance distributed visual analytics for public safety and security around the Pacific Rim.**

## Threads

**Unlock intelligence**
**Bandwidth to brain**** **
**Link minds**
**Bandwidth between brains**** **
**Enlighten minds**
**Courses and simulations**** **
**Focus on threats to the Pacific Rim**** **
**Provide tools to other RVACs**

## PARVAC Collaborators

## Partners

**US**
**UW (Human Interface Technology Laboratory)**
**Tom Furness (IE, EE, ME, AA, TC)**
**Suzanne Weghorst (HIT Lab)**
**Bruce Campbell (HIT Lab-Oceanography)**
**Don Parker (HIT Lab-Medical School)**
**Bill Winn (Education)**
**Zelda Zabinsky (IE)**
**Univ. of Hawaii - Maui High Performance Computing Center & Pacific Disaster Center**
**Tak Sugimura**** **
**Univ. of Alaska - Arctic Supercomputing Center**
**TBD**

## Partners

**Canada**
**Simon Fraser University (John Dill, Nick Hedley)**
**Univ. of British Columbia (Brian Fisher)**
**Australia**
**Univ. of Western Australia (Karen Haines)**
**Royal Melbourne Inst. Of Technology (Mark Burry)**
**University of South Australia (Bruce Thomas)**
**HIT Lab Australia - University of Tasmania (Young Choi)**

## Partners

**New Zealand**
**HIT Lab NZ- University of Canterbury (Mark Billinghurst)**
**Industrial Members**
**The Boeing Company (David Kasik)**
**Change Tools Inc. (Fred Ordway)**
**ARToolworks (Alden Jones)**

## PARVAC TEAM

- Human Photonics

- Adv. Interfaces

- Interface FLOW

- Education

- Human Factors

-  Fear & Pain

- Human Interface Technology

- Telecollaboration

- Augmented Reality

- Seibel

- Billinghurst

- Weghorst

- Furness

- Parker

- Campbell

- Winn

## PARVAC PARTNERS

**PARVAC PARTNERS**

- = existing partners

- = planned partners

## Our expertise

**Super cockpit**
**Super computing**
**Cognitive modeling**
**Visualization (Center for Environmental Viz)**
**Representational constructs**
**Virtual reality, augmented reality, haptics**
**Control/display technology**
**Distance collaboration (e.g. GreenSpace)**
**Decision making & optimization**
**Vision**
**Education**

## Distributed Cognition thread

**DC proposes that human knowledge and cognition are not confined to the individual but distributed by placing memories, facts, or knowledge on the objects, individuals, and tools in our environment.**** **
**DC views a system as a set of representations, and models the interchange of information between these representations. These representations can be either in the mental space of the participants or external representations available in the environment.**

- from Wikipedia

## Further definitions of DC

**Distributed Cognition -Type “T” (thematic)**
**deconstruct VA tasks into parts and distribute for analysis**
**Trends**
**Anomalies**
**Reintegrate parts for correlation and hypothesis formulation **
**Team handoff**** **
**Distributed Cognition - Type “S” (spatial)**
**Location-specific perspectives**
**Common threats**
**Collaborative analysis of outcomes**

## PARVAC Project elements

**MELD = **_**M**_**ind **_**E**_**xpansion over **_**L**_**ong **_**D**__**_x0015_**_**istances**
**RimSim = Pacific Rim simulation engine**
**RimSem = emulation engine**
**JITC****3**** = just-in-time mobile command and control environment for first response**
**Bold Stroke = blue sky journeys into threat space**

## MELD

_**M**_**ind **_**E**_**xpansion across **
_**L**_**ong **_**D**_**istances**

## Vulcan mind meld

## MELD

**Representational Constructs**
**Type T= Knowledge representation & mental scaffolding**
**For hypothesis formulation and testing**
**Type S=situational awareness (spatial and state) **
**For decision making, preparedness and emergency response**
**Interactive constructs (as related to the representational constructs)**
**Individually within the VA process for individual themes**
**Between individuals for reintegrating thematic space **
**Between individuals to construct SA from different perspectives**
**Mind portals**
**Transporting contents of constructs from one mind to the another**
**Asynchronously**
**Synchronously**

## MELD approach

**Driven by cognitive science models**
**Build upon super cockpit experiences**
**Build upon Center for Environmental Visualization (CEV) ocean viz work**
**Network PARVAC participants to ‘walk the talk’**
**Develop metrics to assess goodness (e.g. flow, resource expenditure, etc.)**

## Center for Environmental Visualization - Oceanography

## = just-in-time mobile command and control environment for first response

**JITC****3**

## Emergency Response Network

- MCP-Police

- MCP-Fire

- MCP-Medical

- MCP-Transportation

- MCP-other

- C3

- MCP=mobile command post

- FR=field responder

- FR-P1

- FR-P2

- FR-P4

- FR-P3

- FR-F1

## NZ VisionSpace

**NZ VisionSpace**

## Mixed Reality

## Slide 46

## NSF sponsored project at Boston Museum of Science
ARToolworks Inc.
HIT LAB US
HIT LAB NZ
Technical Director - Dr. Nick Hedley
Opening Oct 27, 2005
www.starwars.mos.org

**ARToolworks Inc.**
**HIT LAB US**
**HIT LAB NZ**
**Technical Director - Dr. Nick Hedley**
**Opening Oct 27, 2005**
**www.starwars.mos.org**

## JITC3

**Develop a flexible, portable command post for deployment with emergency response personnel**
**Build upon augmented reality, MagicBook technology and virtual retinal display (use ARToolwork Inc. ARTK and ChangeTools ZENU technology**
**Build network with field PDA units that communicate to the command post and in turn to the central command post.**
**Integrate with C****3  ****simulations in US, Canada, Australia and NZ (e.g. VisionSpace at HIT Lab NZ**

## RimSim

**Simulation Engine for the Pacific Rim**

## RimSim

**Adapt existing gaming engine to generate dynamics of:**
**Tsunami**
**Dirty bomb in a west coast port SimCity-like**
**Use to test MELD & JITC****3**
**Use to test preparedness and first response dynamics**
**Grow RimSim --------> RimSEM (combination of simulation and emulation of the actual code and algorithms that can be deployed**

## BOLD STROKE!!

**Blue sky thinking about VA solutions**

## Bold Stroke Team

**Greg Bear**

**Nick Hedley**

**Doug Trumbull**

**David Kasik**

**?**

## PARVAC Products

**Tools for modeling, simulating and visualizing dynamic effects of geophysical events (e.g., earthquakes) and other geospatially distributed threats (e.g., pandemic) on local/regional critical infrastructure.**
**Tools for optimizing supply cache distribution in preparation for most likely threats.**
**Tools for parsing disaster response strategies into logistic functions (i.e., distributed cognition).**
**Tools for optimizing analyst flow in detecting and monitoring emergent events.**
**Tools for optimizing situation awareness and responder flow during an emergent event.**
**TOOLS FOR LINKING MINDS ACROSS THE PACIFIC RIM.**
**TOOLS FOR TEACHING THROUGH SIMULATION.**

## Current PARVAC Research

|   | Catastrophic Civic Infrastructure Threat | Catastrophic Public Health Threat | Border and   Port Security Threat |
| --- | --- | --- | --- |
|      Awareness | Critical Geospatial Infrastructure Mapping  (Campbell) | CDC/WHO Incidence Mapping (Campbell/Sugimura) | Collaborative TSA Image Screening   (Parker) |
| 
Response | ” | Med Incident Command VA (Weghorst/Fisher/Dill) |  |
| 
Recovery | ” | Real-Time Geospatial Casualty Mapping          (Campbell/Sugimura) |  |
| 
Prevention/ Preparation | ” | Regional SNS Supply Cache Distribution   (Zabinsky) |  |

| Cross-Cutting        Research |
| --- |
| PARVAC Collaborative Environment        
 (Campbell) |
| Mobile VA (Furness/Billinghurst) |
| Gaze-Contingent VA (Parker/Schowengerdt) |
| First Responder Interface Simulation/Evaluation (Furness/Winn) |
| Collaborative Flow in Time-Critical Decision-Making  (Weghorst/Winn) |
| 
Co-located v. Distributed Collaborative VA   (several)  |

- BOLD STROKE

## PARVAC Near-Term Capabilities

**Software pipeline for geospatially distributed data, based on CEV ocean sciences visualization platform.**
**Incorporation of Seattle/KingCo critical infrastructure data into CEV platform, for simulation of earthquake/tsunami effects.**
**Distributed cognition task/role analysis for Seattle/KingCo medical disaster response.**
**Simulation and optimization methods for regional distribution of medical response supplies.**
**Human interface requirements analysis and laboratory testbed configuration for primary classes of incident response roles (C3, crew chief, field operatives, local public).**
**RimSim game architecture for collaborative/competitive resource partitioning for simulated Pacific Rim events.**
**PARVAC team distributed collaboration platform, with support for both synchronouns and asynchronous collaboration.**

## “Where there is no vision...the people perish.”

*Proverbs 29:18*