# Presentation: 525250

## Onboard HSI Compression Algorithms

**Sid Cook**
**303-971-4609**
**sidney.cook@lmco.com**

**Hyperspectral Compression Workshop**
**May 23, 2003**

## Agenda

**Describe Algorithms**
**Cuprite Compression Evaluation Results**
**Coleamberally Compression Evaluation Results**
**Greater Victoria Watershed (GVWD) Compression Evaluation Results**
** ****Onboard Processing Requirements**
**Onboard Processor FPGA Breadboard System Results**

## Directed Data Compression Processing Flow

**Threshold **
**Results**

**Spectral **
**Uniqueness**
**Monitor**

**Automatic**
**Subspace**
**Discrimination**

**Tunable Anomaly **
**Detector**

**Radiance**
**Image**
**Cube**

**Anomalous **
**Pixel Mask**

**Spectral **
**Compression**

**Automatic **
**Target **
**Detection**

**Compressed**
**Image**
**Cube**

**Spatial **
**Compression**

**Dominant **
**Subspace **

**Step # 1**

**Step # 2**

**Step 1: **
**Identifies anomaly locations **
**Determines pixel model order **
**Produces Dominant Subspace Model**
**Anomalous Subspace Endmembers & Classes**
**Produces Full Endmember Description**

**1, 2**

**3**

**5**

**A**

**B, C**

**Step 2:**
**Compressed Hypercube**
**Endmember Fractions **
**Dominant Subspace Fractions**

**4**

**Notes:**
The compression process begins with what we call a dominant/anomalous image segmentation.  This is performed by the tunable anomaly detector (TAD) algorithm.  The TAD algorithm uses second order statistics to determine the eigenvectors, and then projects out the dominant eigenvector subspace.  This result is automatically thresholded to identify pixels that have high information content – anomalous pixels.  The anomalous pixels are passed along to the Spectral Uniqueness Monitor (SUM) where the basis signatures required to describe all of the anomalous pixels are determined.  The automatic subspace discriminator function is then used to assign all of the anomalous pixels to one of the SUM derived classes.  

The spectral basis signatures along with the dominant subspace eigenvectors are then used together to provide a compressing transformation in the spectral dimension.

## HSI Compression Evaluation

**Analytical Analysis**
**EO-1 Science Team Evaluations **
**Dr. Steve Unger / Dr. Lawrence Ong – NASA GSFC**
**Dr. Jay Pearlman – TRW/Boeing**
**Dr. David Goodenough – Canadian Forest Service**
**Dr. Seth Orloff – MIT Lincoln Labs**

## Compression Evaluation Algorithms

**Spectral Angle (SA) Error**
** ****Pixel Vector (V) = v****1**** + v****2**** + ... v****#bands**
** ****SA = arccos(Vo·Vc/(mag(Vo) x mag(Vc)))**
**Root Mean Squared Error (RMSE)**
** ****RMSE = **
** ****Signal-to-Noise Ratio (SNR)**
** ****SNR =**
**Spearman’s Correlation Coefficient (CC)**
**CC = **

**Subscript “O” Denotes Original Data Cube**
**Subscript “C” Denotes Compressed Data Cube**

**“****P” Denotes Pixel Within a Band**
******Denotes Sum Over Pixels within Band**

## 105:1 Compression

**Cuprite Scene - Hyperion**

** ****Spectral Angle Error Histogram**

**Average = 1.37 Degrees**
**Sigma = 0.29 Degrees**

**Spectral Angle Error Map**
**Large Image Compression = 105:1**

**Notes:**
This Cuprite scene is a mineral outcropping area in Nevada and was taken from the EO-1 Satellite using the TRW Hyperion HSI sensor
Our compression algorithm produces two output files: a compressed data cube, and an an auxiliary file containing instructions for uncompressing the compressed data cube
Since the auxiliary data file does not scale with the size of the data cube and it is on the order of a few hundred Kbytes,
And since operationally we expect data cubes to be strip maps of one or more gigabytes,
We have neglected the size of the auxiliary data file in computing the compression ratio.
Visually we cannot see any difference between the compressed and uncompressed data cubes.
We use the spectral angle as a measure of merit for comparing compression degradation
Spectral angle is computed by taking the arccosine of the spectral dot product of each pixel and dividing by the magnitude of the product of the two pixel vectors.
Small angle implies small compression error
In the Spectral Angle Error Map the dark areas are small errors and lighter areas are the larger errors.
Note the lower right hand corner is a little brighter- and in the visible image this is a dark area – These low reflectance areas are noisy and difficult to compress.

## Cuprite Error Analysis

## 99:1 Compression

**Spectral Angle Error Map**
**Large Image Compression = 99:1**

**0**

**Degrees**

**7.9**

** ****Spectral Angle Error Histogram**

**Average = 1.37 Degrees**
**Sigma = 0.403 Degrees**

**Coleambally Scene – Hyperion**

**Notes:**
Coleambally is an agricultural scene in Australia.

## Coleambally Error Analysis

## Coleamberally Scientist Evaluation Results

| Comp Ratio | EO-1 Scientist Evaluation Observations |
| --- | --- |
| 99:1 | Some Classification Error (<1%) |
| 60:1 | No Classification Error
Some Minimum Noise Fraction (MNF) Analysis Error |
| 28:1 | No Observed Error |

**Analyzed by Dr. Jay Pearlman – TRW – EO-1 Science Team**

**Notes:**
Evaluation Performed by Jay Pearlman - TRW

## 10:1 Greater Victoria Watershed (GVWD) Compression

**GVWD - Hyperion**

**0**

**Degrees**

**2.37**

** ****Spectral Angle Error Histogram**

**Average = 0.53 Degrees**
**Sigma = 0.17 Degrees**

**Spectral Angle Error Map**

## 22:1 Greater Victoria Watershed (GVWD) Compression

**GVWD - Hyperion**

**0**

**Degrees**

**4.61**

** ****Spectral Angle Error Histogram**

**Average = 0.84 Degrees**
**Sigma = 0.30 Degrees**

**Spectral Angle Error Map**

## GVWD Error analysis

## OBP Data Throughput Requirements

**Area Coverage Per Day (KM****2****/day)**
**OBP Rate Requirement (GFLOPS)**
**Data Transmission Req. (Gigabits/day)**

**Current Hyperion**

**OBP Enhanced Hyperion**

**Future***

**2**

**0.02**

**160**

**Sensor Characteristics:**
** ****Pixel Spatial Resolution – 30m**
** ****# Spectral Bands – 220**
** ****Sensor Data Generation Rate – 150 mbps**

**14.7 K**

**1.5 M**

**117 M**

***  “Earth Science Enterprise Technology Planning Workshop on High Performance Spectrometry”**
**     ****May 12, 2000 (Hyperion Sensor with 250 KM Swath Width and SNR = 1000:1 VNIR and 500:1 NIR)**

**Continuous / Real-time Hyperion**

**6**

**4.5 M**

**Current Hyperion Operational Characteristics**
** ****6 Downlinks/day**
** ****2 Image Swaths per Downlink**
** ****Image Swath Width – 7.65 KM**
** ****Image Swath Length – 160 KM**

**67**

**67**

**201**

**5360**

## Area Coverage Per Day

**Current Hyperion**

**Compression- Enhanced Hyperion**

**Notes:**
Here is a visualization showing an area coverage of 14.7K km2/day as compared with 100X that amount or 1.5M km2/day.

## Special Purpose Hardware

**Custom FPGA Hyperspectral Compression PCI Card**

## System Performance

|  | Max | Min | Mean | Sigma | Time* |
| --- | --- | --- | --- | --- | --- |
| MATLAB | 3.692 | 0.052 | 1.273 | 0.368 | 5.653s |
| MAX w/o accum | 3.693 | 0.052 | 1.273 | 0.368 | 9.134s |
| MAX with accum | 3.696 | 0.052 | 1.273 | 0.368 | 7.045s |

**Spectral Angle Error (Deg)**

***Time = total time for calculating the covariance matrix using a Vertex 1000 chip**
** ****Most lengthy process in the spectral compression algorithm.**
** ****Vertex II chips would be faster and are now advertised as being rad- tolerant**
** ****MATLAB time was measured on a 1.8GHz Intel Pentium 4 computer with 1GB of DDR SDRAM.**
** ****More compute power than would be available in a rad-tolerant environment. **