# Presentation: 422159

## Mixed-Initiative Human & Autonomous Agent Systems

- Martin Howard & Kevin Corker
- Human Automation Integration Laboratory
- San José State University

## False start 1

- Agent definitions

## Autonomous Agents

## A few agent definitions

- AIMA (Russel & Norvig, 1995)
- Maes (1995)
- KidSim (Smith, Cypher, & Spohrer, 1994)
- Hayes-Roth (Hayes-Roth, 1995)
- Wooldrige-Jennings (Wooldridge & Jennings, 1995)
- Brustoloni (1991)
- ...
- ...

## Perhaps the most appropriate...?

- The Agent FAQ:
- 	“This FAQ will not attempt to provide an authoritative definition...”
- 			(as quoted by Franklin & Graesser, 1996)

## False start 2

- Agent history

## Current challenges — Current  technologyHuman Supervision of Agents

- Operational complexities:
- Vast distances
- Communication time lags
- Bandwidth and power constraints
- Unknown operational environments

## Slide 8

## Side effects

- Communicating agent behavior
- Automation surprise (Sarter & Woods, 1994)
- Communicating agent capability
- Underreliance (Sorkin, 1988); Overreliance (Riley, 1994)
- Understanding the impact on work
- Workload (Weiner, 1988); Situational awareness (Endsley, 1999); Nature of work (Rochlin, 1997)

## Slide 10

## Current  technology — Current challengesAgents’ Support of Humans 

- Workload and information load reduction (Maes, 1997)
- Agent control for interaction & information Mediation Services  (Genesereth, 1997)
- Agent support in coordinative and cooperation human-agent-human interaction (Jeong & Leon, 2002, Allan, 2002)

## New technology—New challenges

- New agents exhibiting:
- A degree of self-and-other awareness
- Intentionality (goal-oriented behavior)
- Data-driven behavior (dynamic goal selection)
- Adaptive exploration strategies
- Complex, dynamic, adaptive behavior
- Barriers in complexity and cause

## Agents’ Support of Humans 

- Agents’ Behavior Constrained
- Limited Extent of Autonomy
- Limited Range of Application Expertise
- Agents’ Span of Interaction Constrained
- 1: few agents to human
- 1:1 agents to agents

## Future technology—Future challenges

- Likely characteristics of future agents:
- Agents outnumber operators by 1:10 or more
- High degree of autonomy
- Increased functional capability
- Expanded agent mission complexity
- Reduced human control coupling
- Reduced explicit delegation
- More abstract communication

## What works for one, or a few...

- © Disney

## ...when agents multiply...

- © Disney

## ...existing strategies won’t work!

- © Disney

## False start 3

- Agent use

## From the practitioners’ perspective

- Stable, but dynamic, task environment.
- Goal, not tool/system/agent, focus.
- Manageable command set.
- Agent behavior understandable in domain terms.
- Environment constrains and shapes behavior.
- Adaptation to changes in environment

## A need for new research

- A complementary (non technology driven) direction:
- Understanding the notion of agency
- Designing for multi-agent teams
- Control of multiple, autonomous agents
- Collaboration with multiple, autonomous agents
- Impact on human work & cognition

## Research questions

- The notion of agency from a use-centered point of view;
- The impact of scale in human–multi-agent systems;
- Processes of adaptation, control, responsibility, and authority in mixed-initiative human & autonomous agent systems.

## Research direction

- Starting points:
- Perception of causality (Michotte, 1946/1963)
- Perception of animacy (Heider & Simmel, 1944; Scholl & Tremoulet, 2000)
- Distributed cognition & control (Siefert & Hutchins, 1992; Hutchins, 1995)
- Human motor control (Bernstein, 1967; Turvey, 1990; Kelso, 1995)
- Ecological control (Flach, 2000; Jagacinski & Flach, 2002)

## Research method: Overview

- 	Three research themes, coordinated around a space and planetary exploration scenarios.
- Limited windows of action/communication
- Semi-autonomous operation of remote systems
- Multiple mission objectives
- Multiple, distributed, simultaneously active rover platforms

## Research approach: Agency

- Analytical component: Review of past studies on phenomenology of agency
- Empirical component: Quick-and-dirty ethnographic study of the perception of agency (Hughes et. al., 1995; Graves & Nyce, 1992)

## Agency

- 	‘Agents “live” in the real world [...] and real world concepts yield fuzzy categories.’
- 					(Franklin & Graesser, 1996)

## Agency

- Expected results:
- Concept map of agency
- Key attributes dimensions contributing to the perception of agency

## Representation and interaction

- Analytical component: Domain functional analysis (Vicente, 1999)
- Design component: Prototype-driven specification process
- Empirical component: Simulation of future planetary exploration mission (WoZ-driven design/prototype/evaluate cycles)

## Representation and interaction

- Expected results:
- Theoretically and empirically validated strategy for multi-agent system control and representation.
- Abstracted, general Human Factors recommendation for mixed-initiative, human and autonomous agent systems design.

## Joint Cognitive Systems coordination

- Test bed development:
- Effectiveness measures
- Error paths, recovery paths, usability, user satisfaction
- Mission-specific evaluation
- Task support, feasibility, adaptability
- Leverage extant software (Corker, 2000)

## Expected results

- Joint Cognitive Systems approach to multi-agent systems
- Theoretical and empirical validation for proposed control, interaction and representation strategies
- Proposed architecture
- Demonstration system

## Significance

- Ecological control
- Multi-agent system representation
- Mixed initiative interaction & control strategies
- Methodology for Human Factors in multi-agent systems