# Presentation: 221812

## The Surprising Versatility of Statistical Pattern Recognition

- Michael Turmon (JPL)
- Contributions from:  Robert Granat (JPL), Han Park (Boeing)
- AISR Program Meeting
- University of Maryland Conference Center
- 5 October 2006

## Outline

- Object recognition, tracking, track analysis
- Mixture modeling/unsupervised clustering
- Time series analysis
- Supported by an earlier AISR award
- Today, a high-speed recap only
- Time series search in ISS sensor archives
- ...using time series analysis techniques
- Behavior classification in ISS sensor streams
- ...using discriminants built on mixture models

## Object Identification and Analysis

- _Identification_:  Find objects in multispectral science images
- Featuring Gaussian mixture models
- _Tracking_:  Link identified objects in series of images
- _Trajectory Analysis_:  Model and classify object tracks
- Featuring Hidden Markov Models

- Identification

- Allow scientists in domains like solar physics and atmosphere & ocean circulation to *understand great volumes of temporal data* in directly informative terms

- Tracking

- Trajectory analysis

- Move scientists beyond looking at pixels to understanding phenomena

## Identification: Integrating Multimode Imagery

**S**

**N**

**Q**

- _Key_:
- S(pot)
- F(acula)
- Q(uiet sun)

- Flexible, general methods using statistical
- models to identify objects in images

**S**

**F**

**Q**

**1**: Experts identify classes
- in sample images

**2**: Learned mixture model performs classification automatically

**Q**

**S**

**F**

**Identification: Integrating Multimode Imagery**

- Can not distinguish classes from just one observable
- Select mixture model using sample images labeled by scientists
- One mixture model per class
- To classify, compute each class’s probability under its mixture and take the largest
- Move beyond ad hoc threshold rules to allow arbitrary class separators

## Identification: Results

- Turmon et al., “Statistical Pattern Recognition for Labeling Solar Active Regions: Application to SoHO/MDI Imagery,” *Astrophysical Journal*, March 20 2002, 396-407.

## Feature Identification:  Infusion

- This software will be used in the _HMI data pipeline_ at Stanford
- HMI imager will fly in 2008 on board SDO, the first LWS mission
- _[http://sdo.gsfc.nasa.gov](http://sdo.gsfc.nasa.gov/)_  and  _http://hmi.stanford.edu_
- HMI’s data volume is unprecedented in Solar Physics
- 4096x4096 pixel images every 90 seconds
- These data volumes make it more important to focus attention
- HMI/SDO is the successor to MDI/SoHO, on which these results are based
- This software is being evaluated to classify the _Kitt Peak imagery_
- Funded by NASA Sun Earth Connection, with Harry Jones, Kitt Peak
- Five observables (field, intensity + equivalent width, line depth, velocity)
- Taken 1992-2003, replaced by an upgraded vector spectromagnetograph
- Perhaps the best earth-based synoptic magnetic imagery
- Significant training data already gathered by Karen Harvey
- This software has been baselined by _CNES Picard_
- Picard has been given the go-ahead by CNES for 2009 launch
- _[http://smsc.cnes.fr/PICARD/](http://smsc.cnes.fr/PICARD/)_
- Picard will measure tiny variations in solar diameter and shape
- Active region recognition and rejection is important to its delicate results
- Software must clear ITAR hurdles

## Object Tracking Methods

- Associate objects in beforeand after images
- Correlation-based tracker
- Motion model: deterministic drift plus stochastic uncertainty
- For sunspots or cyclones, have motion and correlation on the sphere
- Correlation measure between *a* in *A* and *b* in *B* is *D(a,b)*
- Solve *assignment problem* to match *A* up to *B*:
- with *P* a permutation matrix
- Solution by linear programming
- For our applications, key is to get deterministic drift correct

*A*

*B*

**Before**

**After**

## Object Tracking:  Sunspots

**Magnetogram**

**Labeling**

## Object Tracking:  Ocean Eddies

- Eddies in shallow-water ocean simulation (Toshio M. Chin, JPL)
- State (position, size) of two labeled eddies through time, lower left
- Two subclasses of eddies are apparent, lower right
- We developed Kalman/HMM tools to *analyze* these track motions

## ...putting on the black hat...

## Time Series Search:  Objective and Overview

- A flight controller sees an interesting behavior, and wants to find similar examples in past data
- Exploratory data analysis
- Identifying precursors of conditions to be avoided
- Prelude to training special-purpose detectors for the behavior
- How search works
- The user selects a time series *snippet* that contains the desired behavior
- A statistical model is made for the snippet
- This model is used to search in archived data for similar behaviors
- Search, meet Beta Gimbal Array (BGA)
- BGA is prone to motor stalls and consequent circuit breaker trips
- EVA required to reset circuit breaker
- Flight controllers want to identify motor stalls in advance but have no automatic way to find them in historic data for better modeling
- BGA motor current shows potential precursors to motor stalls

## Search:  Operational Flow

**Search:  Operational Flow**

- Query Snippet
- (from user)

- ComputeFeatures

- ComputeFeatures

- Fit Model

- Target Time Series
- (from database)

- Sweep ModelAcross Target

- Compare Classifications
- Find Top Matches

- Return Matches
- to User

- model

- queryclassification

- targetclassification

## Search:  Modeling a Snippet

- In principle, any state-space statistical model could provide a basis for comparison of the snippet to the historical record
- In this work we used hidden Markov models (HMMs)
- These models separate the time series into discrete activity types
- Provides some robustness to noise and drift
- Our model fitting method (RDAHMM) ensures robust fitting
- Regularized Deterministic-Annealing HMM parameter estimation
- Produces good solutions on the first try
- Easy extension to multivariate sensor search
*PhD work of Robert Granat, 2001*
- The internals of the fitting procedure are invisible to the user

## Search: Match Results

- Snippet

- Top Match

- Match 1

- Match 4

- Match 2

- Match 3

- Match 5

## Time Series Behavior Classification

- Model-based time series classification is an automatic way to focus attention on the most interesting parts of the signal
- Analysis can learn HMM models for specific signatures without training or significant expert involvement (e.g., on right, or LF signature on left)
- Alternatively, identify outliers or novel behaviors (e.g., transient on right)
*These results are due to Robert Granat, JPL*

- •  Space Station motor current (~minutes)•  Snippet selected and modeled•  Sample matches at right

- •  Seismograph (3D velocity, two days)•  Hidden Markov classification (*color-coded*)

## Identifying Behaviors in Sensor Time Series

- ISS Control Moment Gyro = CMG
- Maintains attitude of International Space Station (ISS)
- Four on ISS; two failed and were replaced
- Two Gimbals and Motors (outer/inner) and Gyro Motor

- CMG Sensors
- Position and rate of gimbals
- Vibration
- Temperature
- Motor Current
- Signatures in the CMG Data
- Crew sleep-wake cycle
- Propulsive control
- Mechanical activity, e.g. the SSRMS (manipulator arm)
- As we will see, CMG can be used to spot ISS anomalies

## SSRMS – CMG Correlation

- CMG Outer Gimbal Rate
*Plots and analysis:  **Han Park*

- x 10 Sec

- x 10 Sec

- Showing 2005, day 042, about 90 minutes

- SSRMS Y Position

- SSRMS activity typically causes CMG disturbances
- False alarms could result from such “anomalous” CMG disturbances
- Motivates need for behavior classification

## Dynamical Invariant Anomaly Detector

**Sensor****Time****Series**

**Any****threshold****crossings?**

**Dynamical Invariant Anomaly Detector**

- Fits autoregressive (AR) model to sensor stream
- Recover *a*’s by fitting this model:
- Changes in AR coefficients from nominal model indicate underlying system has changed
- Quantify deviations from nominal
- Anomaly is declared when threshold is passed
*Detector idea due to Zak, Fijany, Park, et al.*

**Dynamical Invariant**
**Anomaly Detector**

**Sensor stream**

**Deviation**
**from nominal**

## ISS Activity Detection in Action

- CMG3 – Outer Gimbal Rate

- x 105 Sec

**ISS Activity Detection in Action**

- Detector Threshold Function “Delta”

- CrewAwake

- }

- Example data from CMG3, about two weeks shown
*Question:  Can this classification be done automatically by clustering?*

**T**

## ISS Behavior Classification

- Fit a Gaussian mixture to the DIAD autoregression parameters
- Four behaviors separate well (left), corresponding to objective conditions (right)
- Improvements
- Classification, not just 0/1 anomaly detection
- Eliminates hand selection of models (finding a nominal model)
- Objective determination of class boundaries versus intuitive detector criterion “Delta”
- Quantitative outlier detection is possible using the mixture

- CrewAwake

- CrewAsleep

- ArmMoving

- Re-boost_x0010_

**Classification**

**Mixture Model**(k = 4 Gaussians)