# Presentation: 016405

## Emulating Human Recognition ofDriving Context 

**Dr. Kevin R. Dixon**
***Sandia National Laboratories***
**(505) 284-5615**
**krdixon@sandia.gov**

## Automotive Cognition Project

**Humans multitask while performing many jobs**
**Increasing workload may result in performance decline on all tasks**
**Talking on phone while driving increases accident risk similar to intoxicated driving**
**Present***** information *****and *****tasks***** in a manner appropriate for operator’s current capacity**

## Human Recognition ofVehicle-Driving Situations

**Create a cognitive model of a human “back-seat driver”**
**Identify potentially dangerous driving situations:**
**“****Approaching Slow-Moving Vehicle”**
**“****Entering On-Ramp”**
**“****Preparing to Change Lanes”**
**...**
**Use this information to minimize impact of untimely distractions during high-difficulty situations**
**Delay unimportant messages**
**Present material visually or auditorily**

## Human Recognition ofVehicle-Driving Situations

**Collected ~24 hours of real-world driving data**
**Vehicle-based sensors**
**Driver-posture sensors**
**Asked humans to label video according to their perception of current situations:**

## Emulating Human Recognition of Driving Situations

**Trained cognitive model to emulate and predict the human recognition of driving situations**
**The goal is to automatically derive a cognitive model that explicitly identifies implicit relationships that humans used when labeling the videos**

## Supervised Learning

**Sandia Cognitive Framework’s pattern recognizer is a type of nonlinear dynamical system**
**Tune system parameters to minimize error between the human’s labels and the estimated labels from the cognitive model**

## Optimizing the Model

**The vast majority of the time (~75%), nothing interesting happens, according to human labelers**
**they’re watching German drivers, not Italian drivers**
**This low ratio of interesting to boring situations does not force the system to learn what causes those rare, but important, events**

**Notes:**
The implication on this slide is that by guess “nothing,” a brain-dead algorithm can “predict” human labels about 95% of the time

## Optimizing the Model

**Penalize the system for incorrectly predicting events *****inversely proportional *****to how frequently the event occurs**
**Forces the system to “pay attention” to the interesting situations as much as the boring ones**
**Iteratively tune the parameters of the approximated model to minimize the “weighted percentage wrong”**
**Typically converges in less than one minute**

## Learning in Action

**each column is output**

**each row is an input**

## Learning in Action

**Learning in Action**

**Algorithm changes the weights to minimize the error between its labels and the human’s labels**

**Notes:**
This is shows the how the weights in the cognitive model change from the weighted-regression algorithm.  The initial weights are from the closed-form solution that I mentioned earlier

## Human Recognition ofVehicle-Driving Situations

**Results for predicting human recognition of driving situations**

## Human Recognition ofVehicle-Driving Situations

## Automated Context Extraction

**How do we know that we identified appropriate contexts?**
**Better to let data tell *****us***** what contexts are important**

## Automated Context Extraction

**Identify for statistically “meaningful” regularities in the data**
**These clusters may or may not correspond to an obvious real-world analogue**
**Meaningless: An artifact from sensor instrumentation**
**Meaningful: Underlying physical process induces clusters**
**We transform *****temporal***** signals into *****vectors***** using regression over a pre-defined time window**

## Unsupervised Learning

## Adapting Models to New Users

**Suppose we obtain a data set from a group as**

**But we want to *****adapt***** the model to a new user**

## Adapting Models to New Users

## Difficulty Scoring

**Instead of identifying specific contexts, human merely indicates perceived difficulty using an analog device, where “zero” is easy, and 100 is sudden death**
**Train cognitive model to emulate and predict human perception of driving difficulty**

## Difficulty Scoring

**Test-Set Results:**
**Average error (L1): ****	****	****	****4.99**
**Mitigation Correct: ****	****	****	****93.17%**

## What’s Next?