# Presentation: 612193

## 

Wendy S. K. Doyle, Ann C. Gentile &
 W. Philip Kegelmeyer
Sandia National Laboratories, USA

NECDC|06
October 24, 2006
Los Alamos, NM

** **
**Wendy S. K. Doyle, Ann C. Gentile &**
** ****W. Philip Kegelmeyer**
**Sandia National Laboratories, USA**
**NECDC|06**
**October 24, 2006**
**Los Alamos, NM**

- Sandia is a multiprogram laboratory operated by Sandia Corporation, a Lockheed Martin Company,for the United States Department of Energy under contract DE-AC04-94AL85000.

**Extracting Information from Data:****Ease Data Analysis Development with FCLib**

**Notes:**
My colleagues and I are at SNL CA.
I’m going to talk about FCLib. It’s a data analysis tool that makes it easier to extrac information from Data.

## Data is NOT Information ... yet

**Information is actionable = answers questions and informs decisions**
**Data (just the facts, ma’am) needs to be interpreted to become information**
**The distinction can be fuzzy **
_**Data analysis**_** helps turn data into information**

- / 31

- 8/24/2005

**Notes:**
Fuzzy = some data is more informative that others
What I’m talking about today is the process of data  analysis and making it easier.

## Data Analysis Example

**Data Analysis Example**

**Overall Process: create a model, stress it, evaluate (repeat)**
**Model: can with a hatch attached by screws**
**Stress: crush at an angle**
**Evaluate: Is the damage within acceptable limits?**

- / 31

- 8/24/2005

**Notes:**
The overall process, of which data analysis is a part ...
Do the run, you have the data. How do you answer the question? 

## Data vs. Information

**Data vs. Information**

**Damage *****Information*****:**
**How many screws break?**
**How many screws are damaged?**
**How many tears (dead element regions)?**
**Do any tears penetrate?**
**How much gapping?**

- / 31

- 8/24/2005

**Notes:**
Explain the data
Unless you are really clever setting up your problem, you can’t read your answers straight from the data

## Data vs. Information

**Data vs. Information**

**Use FCLib to convert data to information**

**The analyst can now act on the information.**

- / 31

- 8/24/2005

**Notes:**
Start interpreting the data using our  library
FCLib quickly answers initial question and human can get to work on harder questions.

## Data Analysis Development

**Issues:**
**Analysis development is iterative**
**Data size is growing**
**Humans are better at creating information than computers**
**Problems with current methods:**
**Specialized and one-off tools can’t be extended**
**“****By hand” methods are slow, hard to quantify & have poor repeatability**
**Our solution is to help the human create information:**
**Automate**** as much as possible**
**But also allow ****flexibility**

- / 31

- 8/24/2005

**Notes:**
Note: data analysis is not just post-processing

## Feature Characterization Library (FCLib)

**FCLib = Toolkit of data analysis building blocks**
**C Library**
**Open Source (BSD) **_**[http://fclib.ca.sandia.gov](http://fclib.ca.sandia.gov/)**_
**FCLib Features: **
**Provide variety of simple building blocks that can be composed into complex analyses**
**Support feature-based analysis (feature = region of interest)**
**Minimize low-level processing (automation)**
**Generality - applicable to a variety of science domains**
**Current File Formats: ExodusII, SAF & LSDyna**
**Provide an ‘elegant’ interface - simple but not constrained**

- / 31

- 8/24/2005

**Notes:**
Our primary goals were 1 and 2. 
The last three goals are more general.
Want 3. But at same time 4.
5. Have a variety of level of users so wanted an interface that was simple and easy to use w/o crippling it.
All though our toolkit is specific to spatio-temporal data, most of these goals align well with the goals of the IDA community and it is our hope that our discussion of reaching these goals is useful to the wider community.

## Presentation Outline

**Introduction**
**FCLib Overview**
**Example 1: Damage Assessment in Weapon Drop Simulation**
**Example 2: Support for Machine Learning**
**Example 3: Feature Tracking**
**Summary & Conclusions**
**Future Work**

- / 31

- 8/24/2005

**Notes:**
Because all data analyses must interact with the data, the data model and its interface strongly influence the usability goals of the toolkit and so we will spend a lot of time discussing them.

## Key Features of the Data Model

**Key Features of the Data Model**

**Spatio-temporal data stored on unstructured meshes**

**Minimal set of abstract data objects **
**Types are deeper in the interface**
**Subsets are full-fledged data objects**
**Mesh ownership of data**
**Time-varying data available per variable instead of per step**

- / 31

- 8/24/2005

**Notes:**
Our focus has been on structural mechanics code, but could be applied to anything that can be  represetented with unstructured meshes.
Here are our 5 primary data objects again and this graph show how they are related. At the top, dataset owns everyone and each mesh owns its own subsets and variables.
Key features: Where we deviate from other data models:
Variable & subsets hide details & characterizations work on all kinds - works in hand with minimal # of data objects.
Data access methods. Data models optimized for simulation & viz are not optimal for data analysis.
Didn’t want to write our own data model, no existing ones fit our needs.
Data models optimized for simulation & viz are not optimal for data analysis.

## Analysis Building Blocks I

**Analysis Building Blocks I**

**Mesh Topology**
**Get mesh entity children/parents/neighbors**
**                          ****,                                         ,**
**Segment into separate connected components**
**Get skin**
**Mesh Geometry (Spatial)**
**Edge lengths/surface areas/region volumes**
**Get mesh entities within bounding box or sphere**
**                                               ****,**

- / 31

- 8/24/2005

**Notes:**
Don’t have time to go into all of the building blocks - this slide and the next some representative examples.
Building blocks range from very simple to more complex ...
Explain the pictures

## Analysis Building Blocks II

**Analysis Building Blocks II**

**Variable**
**Variable Math**
**Statistics: Min/Max/Mean/StandardDeviation/Sum**
**Decompose vectors into normal & tangent components**
**Threshold: find subset**
**Subset**
**Set operations: AND, OR or XOR to get new subset**
**Time**
**Feature tracking - track regions of interest (ROIs) over time**
**Build your own!**

- / 31

- 8/24/2005

**Notes:**
Levels of usage

## How to Build an Analysis

**Explore stress in compressed regions**
**01  /* get handles to objects of interest */**
**02  fc_loadDataset(“some_data.ex2”, &dataset);**
**03  fc_getMesh(dataset, “some_mesh”, &mesh);**
**04  fc_getVariable(mesh, “pressure”, &pressureVar);**
**05  fc_getVariable(mesh, “stress”, &stressVar);**
**06**
**07  /* analysis = string together building blocks */**
**08  fc_threshold(pressureVar, “>”, 0., &subset); **
**09  fc_getSubsetVariableMeanSdev(subset, stressVar,**
**10                               &mean, &sdev);**
**11  printf(“Stress = %g +/- %g\n”, mean, sdev****_x0018__x0013_****);**

- / 31

- 8/24/2005

**Notes:**
Wrap this up with a main and some argument handling and you have command  line tool.
Library comes with some basic comand line tools, fcdump, extents, varstats, etc.

## Example 1: Damage Assessmentin Weapon Drop Simulation

**Three tools were used by W80 analysts on 150+ datasets**
***screwBreaks***** **
***tears***
***gaplines***
**Usage modes**
**Generate a human readable report.**
**Use single measure for sensitivity plots.**
**Use single measure to drive optimization of drop angle to produce maximum damage. **
**Tools were developed by FCLib developers in close communication with analysts.**

- / 31

- 8/24/2005

**Notes:**
Tools ranked by development difficulty. Gaplines tackles a conceptually harder problem and has taken the longest.

## About The Data

**Example Data:**
**2.5 Gb**
**5 parts, ~640,000 elements**
**Largest: ~400,000 elems**
**18 screws**
**13 time steps**

**Real Data:****_x0018__x0013_**
**~50 Gb**
**~100 parts, ~1.8M elements**
**Largest: ~350,000 elems**
**~80 screws**
**4-30 time steps**

- / 31

- 8/24/2005

## How Many Damaged or Broken Screws?

**Visual inspection ...**
**fclib/bin/screwBreaks d3plot**

- / 31

- 8/24/2005

## screwBreaks Output

**...**
**Mesh Screws               Screw 0   Step 12  BR =  0.00 (   0.00/   6.00)**
**Mesh Screws               Screw 1   Step 12  BR =  0.00 (   0.00/   6.00)**
**Mesh Screws               Screw 2   Step 12  BR =  0.00 (   0.00/   6.00)**
**Mesh Screws               Screw 3   Step 12  BR =  0.00 (   0.00/   6.00)**
**Mesh Screws               Screw 4   Step 12  BR =  0.00 (   0.00/   6.00)**
**Mesh Screws               Screw 5   Step 12  BR =  0.31 (   1.89/   6.00)**
**Mesh Screws               Screw 6   Step 12  *** still broken *****
**Mesh Screws               Screw 7   Step 12  BR =  0.27 (   1.64/   6.00)**
**Mesh Screws               Screw 8   Step 12  BR =  0.00 (   0.00/   6.00)**
**Mesh Screws               Screw 9   Step 12  BR =  0.00 (   0.00/   6.00)**
**Mesh Screws               Screw 10  Step 12  BR =  0.00 (   0.00/   6.00)**
**Mesh Screws               Screw 11  Step 12  BR =  0.00 (   0.00/   6.00)**
**Mesh Screws               Screw 12  Step 12  BR =  0.00 (   0.00/   6.00)**
**Mesh Screws               Screw 13  Step 12  BR =  0.00 (   0.00/   6.00)**
**Mesh Screws               Screw 14  Step 12  BR =  0.15 (   0.93/   6.00)**
**Mesh Screws               Screw 15  Step 12  *** still broken *****
**Mesh Screws               Screw 16  Step 12  BR =  0.27 (   1.64/   6.00)**
**Mesh Screws               Screw 17  Step 12  BR =  0.00 (   0.00/   6.00)**
**Mesh Screws               Broken/Total screws: 2/18**
**  ****1 screw(s) broke at Step 4: 15**
**  ****1 screw(s) broke at Step 5: 6**
**All 1 Mesh(es)            Broken/Total screws: 2/18**

- Status of each screw at each time step (only last step shown)

- Screw is damaged (BR = breakage ratio)

- Summary for mesh

- Summary for dataset

- Screw is broken.

- / 31

- 8/24/2005

**Notes:**
Only examines meshes with  “screw” in their name.

## How Bad are the Tears?

**Visual inspection ...**
**fclib/bin/tears -d displacement d3plot**

- / 31

- 8/24/2005

## tears Output

**Tear characterizations for dataset 'd3plot'**
**Tears criteria: 'elem_death' <= 0**
**Time step index: 12**
**5 mesh(es)**
**Mesh 0: 'Shell' has 18 dead element region(s)**
**Mesh 1: 'Plate' has 26 dead element region(s)**
**Mesh 2: 'Cover Plate' has 0 dead element region(s)**
**Mesh 3: 'Horseshoe Plate' has 0 dead element region(s)**
**Mesh 4: 'Screws' has 6 dead element region(s)**
**Combining of dead elem regions not requested**
**Found 50 tears**
**Sorting tears by region diameter (largest first) ...**
**Tear 0:**
**  ****numDeadElementRegions = 1**
**  ****meshIDs = 0**
**  ****meshNames = 'Shell'**
**  ****numCell = 280**
**  ****region volume = 35.8475**
**  ****region diameter  = 19.8277**
**  ****displ exposed diameter = 22.2711**
**  ****region bb  = [ -64.1624, 19.009, -8.56745 ] - [ -54.5411, 30.8475, 5.49459 ]**
**  ****displ exposed bb = [ -66.4801, 1.4563e-07, -13.6168 ] - [ -55.0189, 2.31026, 5.96411 ]**
**  ****intersections with Shape (0:0) (TUNNEL,NONADJSIDES,MAJOR): ( 17) ( 5) ( 5) ( 5) ( 5) **

- Input details

- Tear details

- Per mesh summary

- / 31

- 8/24/2005

## More tears Output

**File: tear-regions.bb => bounding boxes of the tears**

- / 31

- 8/24/2005

## How Much Gapping?

**Visual inspection ...**
**fclib/bin/gaplines -d displ d3plot Shell “Cover Plate” 0.1**

- / 31

- 8/24/2005

## gaplines Output

**Dataset: 'd3plot'**
**Meshes: 'Shell' and 'Cover Plate'**
**Displ: 'displacement'**
**Min Dist: 0.1**
**Number of gap lines found = 12482**
**Number of sets of sides involved = 2**
**Stats for set 1 ('Shell_shape0_side18-Cover Plate_shape0_side2'):**
**numGapline = 10628**
_**    **__**Step      |      |    Gap Length                          .**_
_**ID     Value  |  num |    min        max        mean      stdev**_
**0    0.000000  10628   0.000000   0.028636   0.003368   0.005055**
**...**
**12   0.003000  10628   0.002291   6.851685   1.527860   1.449041**
_**    **__**Step      |      |    Normal Component of Gap Length      .**_
_**ID     Value  |  num |    min        max        mean      stdev**_
**...**
**12   0.003000  10628   0.002291   6.851685   1.527860   1.449041**
_**    **__**Step      |      |    Tangent Component of Gap Length     .**_
_**ID     Value  |  num |    min        max        mean      stdev**_
**...**
**12   0.003000  10628   0.000764   2.306140   0.911504   0.52459**

- Input details

- Stats reported for each side and overall

- Result Summary

- Resolved with respect to face normals

- Gap length stats

- / 31

- 8/24/2005

## More gaplines Output

**File: gaplines.ex2 => the gaplines and length variables**
**2 adj. surfaces found => Cover Plate sits in recess in Shell**
**Location and amount of gapping easily visualized **

- / 31

- 8/24/2005

## Example 2: Support for Machine Learning (ML)

**FCLib used to manipulate data to and from ML algorithms**
**At least 6 of the tools written by ML researchers**
***varSmooth***** - smooth a variable by averaging values within given radius**
**Reduces noise and other artifacts of ML algorithms**
***connect_comp_regions***** & *****gen_overlap_matrix***** - extract and match up regions from different datasets**
**Generate accuracy measures for different ML algorithms**
***Crossval***** - breaks the dataset into multiple partitions**
**Generate random samples for cross validation training**

- / 31

- 8/24/2005

**Notes:**
Initially wasn’t used in ML algorithms, but they are using now
Generate a saliency variable treating each node individually. Smoothing adds back in local information.

## Example 3: Feature TrackingWhat’s Going on in the Ridges?

**Example 3: Feature Tracking****What’s Going on in the Ridges?**

- t0

- t5

- t10

- t15

- t20

- t25

- t30

- t35

- / 31

- 8/24/2005

**Notes:**
We return to the can crush dataset and we notice that as the can crushes, these ridges form, and it looks like these ridges correspond with where the stress variable is getting very high.
The library enables you to take a closer look at the ridges ...

## Find & Track Features

**01  /* get handles to objects of interest */**
**02  fc_loadDataset(“cancrush.saf”, &dataset);**
**03  fc_getMeshByName(datatset, “can”, &mesh);**
**04  fc_getSeqVariable(mesh, “eqps”, &numStep, &stressVar);**
**06**
**07  /* create a feature group and populate with features */**
**08  fc_createFeatureGroup(&featureGroup);**
**09  for (i = 0; i < numStep; i++) {**
**10    fc_threshold(stressVar[i], “>=”, 1.0, &subset);**
**11    fc_segment(subset, 0, &numROI, &ROIs);**
**12    fc_trackStep(i, numROI, ROIs, featureGroup);**
**13  }**
**14  fc_writeFeatureGraph(featureGroup, “graph.dot”);**

- / 31

- 8/24/2005

**Notes:**
.. Where features are regions of high stress.
The feature group decides how to turn ROIs into features

## Reading the Feature Graph

**Reading the Feature Graph**

- t0

- t35

- Feature #4

- t25

- t25

- F6

- F9

- F8

- / 31

- 8/24/2005

**Notes:**
The feature graph shows how the features exist in time, and how they interact. 
15 features. 
Read across graph, get how many ROIs for that step. 
A feature spans time --collection of ROIs from a sequences of steps that we have decided are the same thing.
On this graph a colored circle represents the start of a feature and continuation in time is represented by a thick, colored line. Interactions between features (merge & split) are represented by thin black arrows.
I have colored them by their eventually fate (one of 6 existing features), for clarity.

## Analyze the Features

**01  /* loop over each feature */**
**02  fc_featureGroupGetNumFeature(featureGroup, &numFeature);**
**03  for (i = 0; i < numFeature; i++) {**
**04    fc_getFeatureROIs(featureGroup, i, **
**05                      &numROI, &stepIDs, &ROIs);**
**06    /* loop over each time step the feature exists */**
**07    for (j = 0; j < numROI; j++) {**
**08      /* do analysis */**
**09      fc_getVariableSubsetMinMax(seqVar[stepIDs[j]],**
**10                      ROIs[i], &mins[i][j], &maxs[i][j]);**
**11      /* print stats (not shown) */**
**12    }**
**13  }**

- / 31

- 8/24/2005

**Notes:**
Now we have the features, we can start asking questions about them. For this simple example, we’re just going to get the maximum value of each feature over time.
Each feature is a series of ROIs, 
And ROIS are just subsets.

## Interpret Results & Repeat

**Feature Graph**

- / 31

- 8/24/2005

**Notes:**
Use feature tracking - now know the behavior of high stress points. 
See things that are starting points for further analysis - crumple stress plateaus and seems to do it about the same time a new crumple shows up. And successive crumples have lower and lower plateaus.
Now you could go back and look at other variable values, or perhaps calculate size of crumples and see how that correlates, etc.

## Summary & Conclusions

**FCLib is an open source toolkit of analysis building blocks.**
**FCLib has been used by weapons analysts and machine learning researchers to create data analysis tools**
**FCLib makes development of data analysis easier:**
**Automates low level processing**
**Can assemble analyses from building blocks**
**Developed building blocks can be reused**
**Analyses are tweakable and extensible**
**Analyses are quantitative and repeatable**

- / 31

- 8/24/2005

**Notes:**
FCLib has improved analyst efficiency

## Future Directions

**Make FCLib more usable, looking into**
**Scriptable (Python?)**
**GUI interface (like AVS Express or LabView)**
**Output database (xml) to support varied query modes**
**Grow FCLib community**
**Create space for users to share code**
**More building blocks! More data analyses!**
**Dataset comparisons (very hard)**

- / 31

- 8/24/2005

## Acknowledgements

**Acknowledgements**

**Jay Dike, Tim Shelton & Tim Kostka - SNL, Analysts**
**Ken Buch - SNL, Machine Learning (ML)**
**Robert Banfield, Larry Hall, et. al. - Univ. of S. Florida, ML**
**This research was supported by ASC’s Pre and Post Processing Environments (PPPE) Data Discovery (DD) Program.**
**Contact: Wendy Doyle <wkoegle@sandia.gov>**
**FCLib available at **_**[http://fclib.ca.sandia.gov](http://fclib.ca.sandia.gov/8900/sdm.php)**_

- / 31

- 8/24/2005

**Notes:**
You can read the manual on the website