# Presentation: 682114

## A Presentation to the Computation Institute:Dynamical Modeling of the Perception and Production of Human Movements 

- 				Bennett I. Bertenthal
- 		     		University of Chicago
- 		     		June 5, 2003

## Goals of Modeling Human Behavior

- Describe
- Predict
- Explain

## Statical vs Dynamical Models

- Statical models – relevant state remains constant; focus on relations between state variables
- Examples:  Memory – LTM, STM, episodic, etc.     			     Learning – Expert, novice, etc.
- Dynamical models – succession of states; unfolding of behavior over time
- Self-organization or dynamic pattern formation
- Different behaviors operate at different time scales

## Dynamical Modeling

- Multiple measurements over time
- Intra- and inter-individual variability
- Recover structure in random Gaussian, Poisson, or other error distributions
- New tools for assessing intrinsic and extrinsic noise
- Visual control of posture
- Perception of biological motion in dynamic noise

## Structured vs Unstructured Variability

- Developmental changes in visual control of posture (Bertenthal et al, 1997)
- 5- to 13-month-old infants tested while sitting in moving room
- Walls oscillated forward and backward and different frequencies
- Center of pressure sampled at 50 Hz
- Postural stability assessed
- RMS
- Covariation between visual motion and postural sway

## Slide 6

## Calculation of Center of Pressure

## Slide 8

## Slide 9

## Slide 10

## Slide 11

## Slide 12

## 	Surrogate Data Test 

- 	Surrogate Data Test
-              (Kennel & Isabelle, 1992)

## Slide 14

## Slide 15

## Slide 16

## Slide 17

## Change Mechanism

- Why does RMS show U-shaped function but sway coherence increases linearly with age?
- Infants learn to sit without support around 7 months of age – increased muscle stiffness
- More movement variability (unstructured) at 5 months
- instability
- More movement variability (structured) at 9 months –
- flexibility
- Following development of new coordinative structures, responses are modulated as a function of local conditions

## Slide 19

## Slide 20

## Perception of Biological Motions

- Hierarchical nesting of pendular motions
- 11 moving point-lights corresponding to head & major joints
- Static displays are indeterminate
- Dynamic displays are recognized in 200 ms as specifying human form performing an action

## Time

## Trajectories of individual point-lights corresponding to 

- Trajectories of individual point-lights corresponding to
- major joints on right side of person walking a path parallel
- to image plane

**Notes:**
Absolute motion trajectories

## Left panel:  Point-light walker (PLW)

- Right panel:  PLW embedded in dynamic noise

**Notes:**
Point-light walker with and without masking elements

## Stimuli and Procedure

**Stimulus Dimensions**:
- Target:  6.6 x 2.7 deg
- Mask:  9.1 x 10.5 deg
- Gait cycle:  40 frames (20 frames/sec)
- Display duration:  1.0 sec (Exp. 1); 2.0 sec (Exp. 2-4)
**Target** = 11 moving point-lights
**Mask** = 66 moving point-lights;  77 point-lights if no target
**Task:**  Judge presence/absence or directionality of target
**Conditions**:  Upright/inverted; canonical/off-joint/phase-shifted
**Design:**  100 randomized trials/condition
**Dependent Measure:**  d’ (bias  free index of sensitivity)

## Slide 26

**Notes:**
Table of Results

## Slide 27

**Notes:**
3 point-light stimulus displays

## Slide 28

**Notes:**
Table of results

## Conclusions

- Local processing constraints are insufficient for detecting target
- Detection is guided by global representation of gait pattern
- Temporal relations of limb movements specify global organization of moving point-lights
- Vertical axis of symmetry
- Orientation specificity suggests that previous experience biases the system to detect task-specific relations
- Actions vs movements

## Real-time Coding of Behavior

- Automated tracking of human movements (Tom Huang)
- Body/torso tracking
- Head/face tracking
- Hand motion tracking and analysis
- Challenges:  (1) high-dimensionality; (2) non-stationarity; (3) intrinsic variability; (4) non-linearity
- Extended Kalman filters (EKF)solve non-linear system dynamics in visual tracking
- 3D model used to exploit kinematic constraints

## Simulation of Human Behaviors

- Representing & parameterizing agent behaviors (Norm Badler)
- _Real-time_ portrayal of virtual humans
- Operational Challenge:  Infer emotional or mental state from behavior
- Multiple channels (speech, gestures, etc.) contribute and must not conflict
- Parameterized Action Representation allows agent to act, plan,and reason
- Portrays personality, mood, and affect in embodied agent

## Slide 32

## Coding and Representation of Multimodal Communication

- Visualization for Situated Temporal Analysis system (Francis Quek)
- Visualize, code and analyze multistream communicative content
- Multi-level speech & discourse
- Gesture, gaze, and posture
- Semantic operations-level
- Interface components include:
- Strip-chart presentation of time-varying signals
- Avataric representation of tracked objects
- Searchable metadata database

## Approach

- The Anvil Coding System

## The End