/ 

./ 

7 

HIRIS Performance Study 

J. P. Kerekes 

D. A. Landgrebe 

TR-EE 89-23 

April, 1989 

School of Electrical Engineering 

Purdue University 

West Lafayette, Indiana 47907 

'3/43 

O- L637J 

unclas 

02d9.1.29 

J 
TR-EE 89-23

April 1989

HIRIS PERFORMANCE STUDY

J. P. Kerekes

D. A. Landgrebe

School

West

of Electrical Engineering

Purdue University

Lafayette, Indiana 47907

This research was supported in part by the National Science

Foundation under Grant ECS 8507405 and National Aeronautics and

Space Administration Grarlt NAGW-925.
HIRIS Performance Study

John P. Kerekes and David A. Landgrebe

Laboratory for Applications of Remote Sensing

and School of Electrical Engineering,

Purdue University,

West Lafayette, IN

1. Introduction

In this report, the remote sensing system simulation is used to study a

proposed sensor concept. An overview of the instrument and its parameters is

presented, along with the model of the instrument as implemented in the

simulation. Signal-to-noise levels of the instrument under a variety of system

configurations are presented and discussed. Classification performance under

these varying configurations is also shown, along with relationships between

signal-to-noise ratios, feature selection, and classification performance.

2. Instrument Description

Driven by the recent advances in optical detector array technology and

the opportunities for a permanent polar orbiting platform provided by the

upcoming space station program, research and development has been

progressing on a High Resolution Imaging Spectrometer (HIRIS) [1]. This

instrument is planned to be a part of a cluster of scientific instruments forming

the Earth Observing System (EOS) to be launched in the mid 1990's. This

international group of instruments will provide many new forms of scientific data

of the Earth's surface & atmosphere.

The HIRIS instrument will provide high spectral resolution samples

across the visible and near infrared portions of the optical spectrum by the use

of large CCD infrared detector arrays. The incoming radiance will be spectrally

dispersed onto the arrays to provide one line of spatial data in each of the

spectral bands simultaneously, as shown in Figure 1. The two dimensional

image is then formed by the forward motion of the instrument platform.

In the proposed instrument there will be two separate detector arrays to

cover the entire spectral range. The Very Near InfraRed (VNIR) array will

provide 64 bands between 0.4 and 1.0 pm, while the Short Wave InfraRed

(SWIR) array will contain 128 bands from 1.0 to 2.5 pm.

Functional parameters of the instrument are summarized in Table 1. One

particular item of note is that the maximum output data rate is smaller than the

internal data rate, thereby necessitating data editing or compression be

performed on board the instrument.

I 4_0_9
Dispersing

Element

Lens

Collimator

Slit

Detector

Arrays

Objective

Lena

Figure 1.

Imaged

Scene Pixels Platform

Direction

Imaging Spectrometer Configuration

Design Altitude

Ground IFOV

Swath Width

Spectral Coverage

Average Spectral Sample Interval

0.4 - 1.0 I_m

1.0 - 2.5 l_m

Pointing

Down-track

Cross-track

Data Encoding

Maximum Internal Data Rate

Maximum Output Data Rate

Image Motion Compensation Gain

705 Km

30 m

20 Km

0.4 - 2.5 l_m

192 Bands

9.4 nm

11.7 nm

+600/-30 °

+200/-20 °

12 bits/pixel

512 MBPS

300 MBPS

1,2, 4, or8

Table 1. HIRIS Functional Parameters

2 4/20/89
This instrument will be operated in an on-demand mode by gathering

data only at the request of a scientific investigator. As part of that request, the

investigator will need to specify several mission parameters. A typical list is

given in Table 2. The goal of this report is to investigate sensor performance

under a variety of mission parameter settings.

Location of Observation Site on Earth

Time of Day/Year or

Zenith Angle of Sun

Relative Azimuth between Sun and HIRIS

Look Angle of HIRIS

Acceptable Atmospheric Quality

IMC Gain State

Radiometric Resolution

Spatial Data Editing/Compression

Spectral Data Editing/Compression

Table 2. Example HIRIS Mission Parameters

3. Instrument Model

For this investigation the model shown in Figure 2 is used for the HIRIS

instrument. This model version has 201 equally spaced (10 nm intervals)

bands from 0.4 to 2.4 I_m and includes most major spectral, spatial, and

radiometric effects of the instrument. The model was implemented in the system

simulation program described in reference [2].

Instrument parameters have been obtained from a progress report by

JPL (reference [3]). These parameter levels are based upon preliminary

specifications and prototype testing.

The following paragraphs and figures detail the blocks in the overall

diagram.

The spectral transmissivity of the instrument optics is shown

in Figure 3. Note the low response at the spectral gap between the VNIR and

the SWIR arrays at 1.0 I_m.

The normalized spatial response of the optics and field stop is assumed

to be similar to the that of the Landsat Thematic Mapper instrument, as they both

have a GIFOV of 30 meters. Figure 4 shows the measured down scene and

across scene normalized responses as a function of angular distance, taken

from reference [4]. The data points shown are the discrete values used in the

simulation. At the nominal altitude of the instrument, the distance on the ground

between these data points is approximately 7 meters.

3 4_o_9
OPTICS

Incoming

Spectral

Radiance

DETECTOR

Detector Shot Read

Response Noise Noise

Dark

Current

SIGNAL CONDITIONING

Output

Digital

Number

Figure 2. HIRIS Model Block Diagram

4 4_0_9
0,5, ,

C

O

m

N

.m

E

lm

.=

tj

4)

eL

_n

0,4 "

0.3"

0.2"

0°1 =

0.0

0.4

r------

0 6 08 1 0 1.2 1.4 1.6 1.8 2 0 2.2 2.4

Wavelength (micrometers)

Figure 3. Spectral Transmissivity of Optics

1,2'

o

C

O

O.

m

O

nO

N

E

O

Z

1.0

0.4 "

0.2 °

0.0 °

-0.2

-80 -,o -;o Jo ; =o ,;, ,'o

Angular Distance (mlcroradlans)

Figure 4. Normalized Spatial Response

8O

Across Scene

Down Scene

5 4/20/89
The radiometric conversion from the incoming spectral radiance

(mW/cm2-sr-i_m) is accomplished by dividing by 1000 mW/W, and multiplying by

the A.Q, the product of the detector area and the solid angle of view, of the

optics. The output of the optics model Px, the incident spectral power, is then in

units of watts/_m. The A.Q used in the model is

A.Q = 1.44 x 10.6 cm2-sr.

;L?,..D_P,/.(_t_E The spectral quantum efficiency of the detectors is shown

in Figure 5. The incident spectral power Px at wavelength ;L is converted to a

number of electrons S at the detector by the integration of the incident photon

level over the pixel integration time, as in equation (1).

where,

S = P_" _." h--_'ot

,_. = 10 nm, wavelength interval of spectral samples

_. = wavelength of interest (p.m)

h = 6.62 x 10.34 Joule-sec, Planck's constant

c = 3 x 108 meters/sec, the speed of light

t = 4.5 millisec, pixel integration time

(1)

Since the noise level data and full scale specifications were obtained in

terms of number of electrons, the signal level is stated in these same terms and

is unitless.

The Image Motion Compensation (IMC) is implemented through

movement of the down-track pointing mirror to offset the platform speed and

effectively multiply the pixel integration time by the gain state selected: 2, 4, or 8.

6 4/2o/8g
O

¢=

e

..o

==

1.1,1

E

Q

o

1.0

0"91 0.8

0.7-

0.6"

0,5-

0.4-

0.3"

0,2-

0.1-

0.0 • • • | • | • I • • ' • •

Wavelength (micrometers)

Figure 5. Spectral Quantum Efficiency

The noise in this model consists of a deterministic dark signal

level, and random shot and read noise. Thermal noise has been found to be

insignificant. All noise is considered to be stochastically independent between

noise types and spectral bands. While calibration errors are expected to be

significant, data have not been obtained to develop an appropriate model.

Thus, this type of error has not been included as of yet.

The dark current level is given in Table 3. This level is added to the total

received signal.

VNIR 0 e- I SWIR 2.7 x 10 4 eTable 3. Dark current levels in terms of electrons

The shot noise in the model consists of zero mean Gaussian random

numbers with a standard deviation equal to a function of the total signal level in

the detectors. This total signal is comprised of the incoming radiance, and the

dark current level mentioned above. Figure 6 shows several points relating

total signal and shot noise levels, along with a best fit curve and its equation.

7 4/20/89
6OO

A

I

¢ 500,

=0

E!

400,

m

|

.,I

Q 300,

_=

O

Z

J=

r,/)

IO0

y - - 1061.1 + 278.05"LOG(x) RA2 - 0.931

100000 200000 300000 400000

Slgnal Level (electrons)

Figure 6. Shot Noise vs. Signal Level

Thus, the standard deviation of the shot noise process is given by

equation (2).

Gshot = -1060 + 278 Ioglo S' (2)

S' is the total detector signal level in electrons. Note, this relationship is

assumed to be independent of wavelength.

Read noise is added in as a zero mean Gaussian random number with a

standard deviation as in Table 4. Within each detector array, the read noise

level is assumed to be constant over wavelength.

VNIR 300 e- ] SWlR , 1000 eTable 4. Read Noise Levels

5.4 Signal Conditioning. Table 5 gives the full scale e- values F for each

detector army. These are values obtained from the JPL report, and as with all of

the data values presented, should be considered to be subject to change.

8 4_o_9
VNIR SWIR 577,395 e- I 1,441,440 e"

Table 5. Full Scale Electron Levels F

The conversion from the e" levels S" (received signal plus noise) to a

digital number (DN) occurs as in equation (3).

where,

DN = nin t{IMC_'',,F * I20 ,1} I_l

IMC = IMC Gain State

F = Full Scale Electron Level

Q = Number of radiometric bits (nominally 12)

The division by the IMC gain state is included to preserve the dynamic

range of the detectors over the various gain states.

4. Signal to Noise Variations

This section presents the results of the effect of varying system

parameters on the signal-to-noise ratio (SNR) of the HIRIS model.

In many Earth resource analysis remote sensing applications, the output

product is some form of classification map of the observed area. The

classification is usually obtained by a computer algorithm that uses the mean

and covariances of the multispectral image data to distinguish between the

classes. Thus, in this application, not only are signal levels important, but so

are signal power variations.

To gain a more realistic sense of how the system parameters affect HIRIS

performance, two versions of SNR are defined: Voltage SNR, and Power SNR.

These are defined for each spectral wavelength band m as in equations (4) and

(5).

Voltage SNRm= 20 IOgl0'_#O_shot,m+O2.read,m+_quan J (4)

where,

f _G_m

Power S""m=,0,o0, o"[sho,,m+o quantf15,

Prn = Mean surface reflectance at wavelength band m

9 4/20/89
O2m= Variance of surface reflectance at wavelength band m

Gm = Conversion factor with units of number of electrons due to the

solar irradiance at the surface, atmospheric transmittance, and

the sensor response for wavelength band m

°_shot,m = Variance of shot noise in wavelength band m

°_read,m = Variance of read noise in wavelength band m

O2

quant = Variance of quantization noise

The quantization noise is assumed to be uniform with a standard

deviation of

where A is the number of electrons per digitization interval.

For 12 bit radiometric resolution and the full scale values in Table 5, the

standard deviation of the quantization error is given in Table 6.

VNIR SWIR 41 e- J 102 eTable 6. Standard Deviations of Quantization Error

For the SNR results included in this report, the system configuration

shown in Table 7 was used as a baseline. The solar illumination and

atmospheric effects were obtained using the LOWTRAN 7 [5] computer code.

The 1976 U.S. Standard Atmosphere with rural extinction was used.

Meteorological Range 16 Km

Solar Zenith Angle 30°

Solar Azimuth Angle 180 °

View Zenith Angle 0°

View Azimuth Angle 0o

IMC Gain State 1

Sensor Noise Levels Nominal

Radiometric Resolution 12 bits

Table 7. Baseline System Configuration for SNR Study

Figure 7 shows the voltage SNR for three surface albedos and the

baseline system configuration of Table 7.

10 _o,89
LOWTRAN 7

60

50

A

m

13 40

n,.

Z 30

r,/)

o

oD

lO

Figure 7.

0

0.4

• I ! I • I • I ! l ! I •

0.6 0.8 1.0 1.2 1.4 1.6 1.8

! , | •

2.0 2.2 2.4

m

_Dm_

Nbedo ,, 1.00

Nbedo = 0.10

Nbedo - 0.01

Wavelength

SNR of HIRIS for various albedos using LOWTRAN 7 and the

baseline system configuration given in Table 7.

For the study of parameter effects on SNR, it was decided to use typical

surface reflectance statistics rather than a deterministic albedo. The surface

reflectance statistics used were obtained from a site in Finney County, KS on

May 3, 1977 [6]. 1551 observations were combined from three classes: winter

wheat, summer fallow, and an unknown class. The mean reflectance and

variation of this data ensemble are plotted in Figure 8.

To obtain an idea of how this reflectance is modified by the atmosphere

and sensor response, a simulated image was created using the baseline

system configuration and the reflectance of Figure 8. The resulting mean digital

counts and their variation are shown in Figure 9. Several effects are

immediately noticeable. The absorption bands of the atmosphere are present,

as well as a reversal in the relative values of the visible and infrared responses

due to the solar illumination and gain settings of the sensor. Also, an apparent

reduction in relative variation is seen due to the additive constant path radiance

and dark current in the instrument. No correction for these effects was made.

11 4/20/89
A

v

0

C

Q

0

Figure 8.

data set.

50

4O

10

-_- Mean + Sld Dev

Mea_ Reflectance

........ Mean - Std Dev

.5--/

0 • I • | • I • I

0,4 0.6 0.8 1,0 1.2

I • • I • '

1, 11618 21o2122,

Wavelength

Mean and variation of the surface reflectance of the combined

:3

0

0

OD

1,,oo]

800 "|

0.4 0.6 0.8 1.0 1.2 1.4 1.6 1.8 2.0 2.2 2.4

Wavelength

Figure 9. Mean and variation of image vector as received by HIRIS.

12 4/20/89
The following Figures 10 through 21 show the Voltage and Power SNR

variations as a function of the parameters shown in Table 8.

Meteorological Range

Solar Zenith Angle

View Zenith Angle

IMC Gain State

Sensor Noise Levels

Radiometric Resolution

2,4,8,16,32 Km

0°, 15°, 30 °, 45°, 60 °

0°, 15°, 30°, 45°, 60°

1,2,4,8

1/4,112, 1,2, 4

8, 10, 12, 14, 16 bits

Table 8. Parameters Studied and Their Variations

13 4/20/89
A

m

"o

rr

z

(I)

o

al

m

0

>

5O

i_/,I _.,J _'|

20-li: ; _._ ,3; t_

P # J_i E

li!/f if.

Itl f ; , I:

10_/ / ill'l_ !

llJ i

i ;

oV/........

0.4 0.6 0.8 1.0 1.2 4 1.6

I

!

1.8 2.0 2.2 2.4

32Km

16 Km

........ 8 Km

....... 4Km

........ 2 Km

Wavelength

Figure 10. Voltage SNR for varying Meteorological ranges.

A

m

qo

v

Z

O)

lira

o

0

a.

5O

32 Km

40 16 Km

........ 8 Km

....... 4Km

30 ........ 2 Km

Rj._'

f,!i '

10

I/i ' i.Jl . '.

0 I • I • I - I • r , • •

0.4 0.6 0.8 1.0 1.2 1,4 1 6 1 8 2 0 2 2 2 4

Wavelength

Figure 11. Power SNR for varying Meteorological ranges.

14 4/20/89
A

m

"o

z

¢n

o

¢G4=*

O

5o

4o

3O

2o

lO

0 I • • " "

0.4 0.6 0.8 1.0 1.2 1.4 8 2.4

Wavelength

Figure 12. Voltage SNR for varying solar zenith angles.

0 Degrees

15 Degrees

30Degrees

45 Degrees

60 Degrees

50

40

, i

2O !

1o !

I

o | ! i , i | i | i

0.4 0.6 0.8 1.0 1.2 1,4 1.6 1.8 2.0 2.2 2.4

om...D.

Wavelength

Figure 13. Power SNR for varying solar zenith angles.

0 Degrees

15 Degrees

3O Degrees

45 Degrees

60 Degrees

15 4/20/89
A

¢D

"o

v

rlZ

O)

o

m4--*

o

5o

°

0 ,

0.4 0.6 0.8 1.0 1.2 1.4 1.6 1.8 2 0 2 2 2.4

0 Degrees

15 Degrees

........ 30 Degrees

....... 45 Degrees

........ 60 Degrees

Wavelength

Figure 14. Voltage SNR for varying view zenith angles.

A

m

"o

v

E

Z

O)

i--

o

o

a.

0 Degrees

15 Degrees

........ 3O Degrees

....... 45 Degrees

........ GODegrees

Wavelength

Figure 15. Power SNR for varying view zenith angles.

16 4_0_9
m

,o

{Z:

Z

(/)

o

o)

o

70

6O

0 1 • , • • • , • , •

0.4 0.6 0.8 1.0 1.2

i,

| ! •

.4 1.6

i/ I'V_

"/_,.__'_.

• | • i

.8 2.0 2.2 2.4

IMC = 8

.......... IMC = 4

........ IMC = 2

....... IMC = 1

Wavelength

Figure 16. Voltage SNR for various IMC gain states.

A

m

'o

v

E

Z

U)

im

o

o

n

40"

50-

°1 i

• ! • • • • | •

0.4 0.6 018 1_0 112 1.4 116 .8 2.0 2 2 2.4

IMC = 8

_, IMC = 4

........ IMC =2

....... IMC = 1

Wavelength

Figure 17. Power SNR for various IMC gain states.

17 4/20/89
A

m

'o

v

Z

U)

o

m

m

o

>

7O

6O

40 _ v ,_,

i\:i"

.fi,(i ",w

10Ii;_ , , , " I , "

0 I • I • •

| ' • i " I

0.4 0.6 0.8 1.0 1.2 1.4 1.6 1.8 2 0 2.2 2.4

I/4

I/2

Wavelength

Figure 18. Voltage SNR for varying factors of shot and read noise.

"o

nZ

U)

a.

o

o

o.

60

40"

30,

20

10

0t / _4A , ,

0

• I • I • I

0.4 0.6 0.8 1.0

.. r

! ' ! _1

• t!

t

2'

!

u• 4 , 1.6I , 1.8 • 2.0l • 2 .I2 • 2.4

1/4

........ 1/2

------" 2

-'---'-" 4

Wavelength

Figure 19. Power SNR for varying factors of shot and read noise.

18 4/20189
A

m

,o

v

E

Z

(/)

o

o

5o

4o

10

a

r_

i q

r

I

i I

0

0.4 " 0.6' " 0 i8 " 1.0' " 1 i2 " 1_4 ....1.6 1.8 2 T0 " 2.2' 2.4

16 bits

......... 14 bits

........ 12 bits

....... 10 bits

........ 8 bits

Wavelength

Figure 20. Voltage SNR for various numbers of bits.

A

G3

'o

v

E

Z

t.t)

!._

o

}

o

o.

50

,°t

30-

2O

10

ii

!

#

0 • i • i - i I i

0.4 0.6 0.8 1.0 1.2

wi,

4

I

I

$

!

4 1.6 1.8 2.0 2.2 2.4

16 bits

14 bits

........ 12 bits

....... 10 bits

........ 8 bits

Wavelength

Figure 21. Power SNR for various numbers of bits.

19 4/20/89
Several common features of all the graphs are immediately noticeable.

The wide water absorption bands are present at 1.4 and 1.9 l_m, while several

narrow absorption bands due to other atmospheric constituents are also

present. The relatively low SNR's around 1.0 I_m are due to the instrument

optics and detector response fall off at the spectral gap between the VNIR and

the SWIR arrays.

Looking at the effect of the meteorological range, one observes a more

pronounced effect on the SNR in the visible and near infrared portions of the

spectrum, than those at longer wavelengths.

Considering the goniometric configurations, the effects on SNR of

varying the solar or view angle seem to be similar, at least for the system

configuration chosen. Note that these variations here are only due to the

atmospheric effect, as angle dependent reflectance variations of the surface

were not considered in this model.

The effect of the IMC and noise level parameters seem to be reasonable

and consistent. The radiometric resolution has little effect on SNR at resolutions

of 12 bits or more. This seems to indicate that the quantization error at 12 or

more bits is insignificant compared to other noise sources, while at 8 or 10 bits

quantization error becomes the dominant noise source.

20 4_0_9
5. Parameter Effects on Classification Accuracy

The effects of the parameters in Table 8 were also studied on the

classification performance of the HIRIS instrument. Simulated scenes were

generated using the three classes of the Finney County data set of May 3, 1977

and the system simulation program [2]. Since the model HIRIS sensor

generates images with 201 bands, a feature selection algorithm was applied

that combines bands together to form each feature. For the following

experiments, a set of 16 features were used with a maximum likelihood

multivariate classifier (Gaussian assumption) to assess accuracy. Table 9

shows the wavelength bands combined for each of the features. These features

are based upon the algorithm described in reference [7]. Table 10 contains

relevant system parameters used in the simulation.

Featu re Wavelength (i_m)

1 0.70 -0.92

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

1.98 - 2.20

2.20 - 2.40

0.66 - 0.84

1.48- 1.64

0.52 - 0.66

1.64 -1.78

1.16 - 1.28

0.96 - 1.06

1.04- 1.12

0.94 - 1.00

0.44 - 0.50

1.12 - 1.16

0.92 - 0.96

0.40 - 0.44

1.00 - 1.04

Table 9. Features used in classification experiments.

I

Scene Size 90 by 90 Pixels |

Scene Pixel Size 15 meters I

Image Size 45 by 45 Pixels

Number of Pixels per class 675

Number of trainincj samples per class 300

Table 10. System Simulation Parameters.

The following Figures 22 through 27 show the results of the various

parameter changes on classification accuracy. The accuracies shown are the

average of the three class accuracies, and represent the mean of 10 repetitions

of the simulator.

21 4_o_g
95

94

,2

m

qm

m 91

9O I

o 1'o 20 3'0 ,o

Meteorological Range (Km)

Figure 22. Classification accuracy vs. meteorological range.

95

94

i

i

m

92

m

ca

(.1

91 o " 2'0 ,'0 e'o eo

Solar Zenith Angle

Figure 23. Classification accuracy vs. solar zenith angle.

22 4_0_9
97

O

ca

L_

:1

8

<

¢:

O

m

m

m

m

¢=

n

O

96

95

94

93

92 I

o 20 ,'o 60 80

View Zenith Angle

Figure 24. Classification accuracy vs. view zenith angle.

88

1 2 4 8

IMC Gain State

Figure 25. Classification Accuracy vs. IMC gain state.

23 4/20/89
96

< 92

88 ! |

; 2 3 ;

Noise Level Factor

Figure 26. Classification Accuracy vs. shot and read noise level.

96

A

v 94

O

¢=

L.

o

o

'_ 92

C

O

m

m

_0

m

¢=

m

88

8 10 12 14 16

Number of Bits

Figure 27. Classification Accuracy vs. number of bits.

24 4/20/89
Most of these results are as one would expect. In hazier atmospheres, or

with high noise levels the accuracy decreases. Figure 24 showing the effect of

view angle is a little less intuitively satisfying. The sudden rise in classification

accuracy at a view angle of 60° is surprising. This seemingly abnormal result is

due to the complex interaction of the scene pixel size and the sensor spatial

response. At this view angle the distance on the ground between samples of

the discrete spatial response becomes approximately 15 meters, the same as

the size of the scene pixels. At angles less than 60 o, about 25 scene pixels

were appropriately weighted to form one image pixel. Now, at 60 o, about 45

scene pixels are combined. This has the effect of reducing the within class

variation and increasing the class separability and classification accuracy.

This result points out how complex interactions between components of

the system can affect classification accuracy. It also points out some of the

pitfalls in the use of a discrete simulation for the study of system effects. One

must exercise caution in specifying system parameters and in interpretation of

results.

One significant part of the real instrument not included in the model of

Figure 2 is error in radiometric calibration. This omission is mainly due to the

fact that adequate data does not exist to develop a model for this error. To test

the significance of this omission, a model of the error based on percentage of

value, and distributed uniformly with a mean of zero, was inserted before the

signal conditioning. This error is distributed randomly across the image and

represents a relative error. Figure 28 shows the effect of this error as it varies

over the range of 0.0 to 4.0 percent.

2 5 4120189
ae

v

u

2

u

u

<

e-

_o

_u

_=

w

eo

o

D

L

95

93

91

89

87

85

Relative Calibration Error (%)

Figure 28. Accuracy vs. Relative Calibration Error

This model forcalibrationerrorshows a significanteffectforerrorson the

order of one percent ofvalue or greater,and shows the importance of accurate

relativecalibration.

Next, an experiment was run to evaluate the interrelationshipsof two

parameters. The effectof the IMC gain state on classificationaccuracy vs.

meteorologicalrange was studied. Figure29. shows the result.Itappears that

the IMC gain statehas a greatereffecton improving classificationaccuracy in

hazy, or low meteorological range, atmospheres. But, even in clear

atmospheres the highergain stateshows an improvement.

26 4/20/89
96

A

>,

O 95

m

L.

:l

¢J

¢J

_4

¢:

O

i

¢J

m

¢@

¢=

5

@ 92

¢=

Im

O

>

,<

91 !

o lo 20 3"0 ,o

Meteorological Range (Kin)

Figure 29. Classification accuracy vs. meteorological range for several

IMC gain states.

2 7 4/20189
6. Relationships between SNR, Classification Accuracy,

Separability, and Feature Selection

In this section we discuss computing the SNR based on the combined

features, and present results showing the relationships between this SNR, class

separability and classification accuracy for several feature sets.

In calculating the SNR of a feature, the signal and noise levels of the L

component wavelength bands are combined. In the case of the voltage SNR,

the signal levels are simply added together, as are the variances of the noise

levels. This is shown in equation 6.

Feature Voltage SNR = 20 Ioglo ,_ i_1 02 +02 +0 2

t ..,, ua ,,j

(6)

For the power SNR, the signal levels cannot be simply added because of

the band-to-band correlation present in the reflectance data. Here, the signal

variance is the sum of the individual variances, plus terms due to the covariance

between each pair of bands I and m, combined in the feature as in equation (7).

Feature Power SNR = 10 Ioglo<_

t

L i-1 m-1 t

+,<=,,,+<u,n,,,J

(7)

An experiment was then carried out to determine if the SNR of a

particular feature had any effect on its importance in classification. The

combined voltage and power SNR's of the feature set shown in Table 9 were

computed and the results shown in Figure 30.

28 4_0_9
6O

5O

m

O 4O m

¢=

n-

¢D 3O mu

O

Z

6 =o E

¢B

I¢

(n IO

• Voltage SNR

[] Power SNR

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

Feature Number

Figure 30. SNR's for features of Table 9.

These features were then ranked according to the measures of Table 11.

Then the effect of these rankings were measured by computing accuracy vs.

number of features ordered by each ranking measure. The results are shown in

Figure 31.

Measure

Bandwidth

Voltage SNR

Power SN R

V-P SNR

Ranking order

Decreasing bandwidth of feature

Decreasing Voltage SNR of feature

Decreasing Power SNR of feature

Decreasing difference between Voltage

and Power SNR of the feature

Table 11. Various rankings of the feature set. The ranking order is for

increasing feature number.

29 4/20189
A

0<

>,

O

I,=

o

o

C

Om

¢=

o

m

m

¢0

am= (J

100

90

80

7O

6O

5O

4O

-- Bandwidth

•---,P'-- Voltage SNR

.... -B---- Power SNR

----e--- V - P SNR

0 2 4 6 8 10 12 14 16

Number of Features Used in Classification

Figure 31. Classification accuracy vs. number of features for several

different orderings of the feature set.

The ranking by bandwidth, as in Table 9, appears to have the best

performance for small numbers of features. For one feature, the feature with the

highest power SNR performed the best, and above 9 or 10 features, all rankings

performed similarly.

An observation of these results is that the SNR of any individual feature is

a poor predictor of that feature's importance when included in a multi-feature

classification algorithm.

To obtain an overall measure of the SNR for a set of features, the method

behind equations (6) and (7) was extended to combining the signals and noise

from all wavelength bands contributing to the feature set.

To relate these combined SNR's to classification accuracy, a scatter plot

was made of the SNR's for the various configurations used to produce Figures

30 4/20/89
22 through 27, and the resultant classification accuracies. This is shown in

Figure 32 for the voltage and power feature SNR's.

A

m

,o

e.

IT.

Z

U)

80

70

6O

50

O VoltageSNR

!, PowerSNR

O

O

O

O0 z

O

e o o u,,

o x_

O ""xill_=

!

x z X

o

X

N

40 = I =

88 90 92 g4 96

Average Classification Accuracy (%)

Figure 32. Classification accuracy vs. combined feature SNR.

Attempts to fit a linear or polynomial equation to these plots yielded a

correlation coefficient of 0.71 for the best fit, that being a 3rd order polynomial.

While there is a general relationship here between SNR and classification

accuracy, they appear to be loosely correlated at best.

Also in this experiment, a multiclass implementation of the transformed

Bhattacharyya separability measure was used to compute the effect of the

parameters on separability. Equation (8) shows the two class transformed

Bhattacharyya distance.

where,

C_-_

B..=I -e "_

ij

_-(M i - Mj)T_.1;i2 (Mi - IVIj) + 1⁄2 log,) (l iI j

(8)

Here, Mi is the feature mean vector of class i, while _i is the feature

covariance matrix of class i. The multiclass separability is computed using the

31 4_o_g
apriori probability weighted sum over all classes i, j of the two class distances,

as in equation (9).

Separability = '_'_ p(o)i)P(O_j)Bij

i-1 j,-1

(9)

The plot of the classification accuracy vs. separability is shown in Figure

33. The p2 value of the linear best fit line is 0.93. Thus, over this range of

classification accuracy and separability there is a high correlation between the

two.

96

A

v 94

:=,

O

i=

:1

O

tJ

'( 92

¢:

2

u

m go

m

m

0

Im

m

[]

mm

mUm

Separability

Figure 33. Classification accuracy vs. transformed Bhattacharyya distance.

A plot of the separability vs. SNR is shown in Figure 34. The best fit line

has an p2 = 0.62 for the voltage SNR and p2 = 0.63 for the power SNR.

A tentative conclusion from these results is that while separability and

classification accuracy are highly correlated, much less correlation exists

between either the voltage or the power SNR and classification

accuracy/separability.

32 4_0_9
A

m

n,.

z

o)

+1• Voltage SNR • •1

7oj • Power SNR • /

] ..'.1

l . r- "/

60] uUU • °O'u /

1 " '-' /

411

0.84 0.86 0.88 0.90 0.92 0.94 0.96

Separability

Figure 34. Transformed Bhattacharyya separability vs. combined feature SNR.

Several sets of six features (shown in Table 12) were used to evaluate

their classification performance.

Featu re

1

2

3

4

5

6

SFD

0.42-0.66

0.66-0.84

0.70-0.92

1.48-1.64

1.98-2.20

2.20-2.40

TM

0.45-0.52

0.52-0.60

0.63-0.69

0.76-0.90

1.45-1.75

2.08-2.35

WSNR

0.40-0.70

0.77-0.90

1.00-1.10

1.15-1.30

1.50-1.74

1.97-2.40

NSNR

0.51-0.56

0.81-0.86

1.02-1.07

1.20-1.25

1.59-1.64

2.16-2.21

SSFD

0.59

O.75

0.81

1.56

2.10

2.30

SSNR

0.54

0.84

1.04

1.11

1.61

2.19

Table 12. Wavelength bands combined for the various feature

sets. The various feature sets are defined as SFD = Spectral

Feature Design algorithm, TM = Landsat Thematic Mapper, WSNR

= Wide Signal-to-Noise Ratio, NSNR = Narrow Signal-to-Noise

Ratio, SSFD - Single band Spectral Feature, SSNR = Single

band Signal-to-Noise Ratio.

The SNR features were chosen based upon regions of high SNR. These

various sets were chosen to see how classification accuracy and combined

signal-to-noise ratios compared. Figure 35 shows the combined SNR for the

various feature sets, while Figure 36 shows the resultant classification accuracy

for the baseline image used in this report.

33 4/20/89
A

m

I=

¢:

m

v

tr

;[

u)

8o

60,

4O

2O

• Volage SNR

• PowerSNR

SFD 1M WSNR NSNR SSFD SSNR

Feature Set

Figure 35. SNR's for the various feature sets.

85

SFD TM _ NSNR _ SSNR

Feature Set

Figure 36. Classification accuracy for the various feature sets.

Clearly, the SFD features performed the best for this data set.

34 4/20189
The robustness of the spectral feature design algorithm was then studied

by comparing the accuracy of the various feature sets in classifying scenes

created from other data sets than those from which the features were derived.

Two other scenes were tested. One was created from reflectance data of three

varieties of spring wheat. The second was from an artificial data set created

from the covariance matrices of barley, pasture, and grain sorghum, with the

same mean vector (that of barley). This was done to decrease the class

separability and increase any difference in the classification accuracy due to the

differing feature sets. When the original means were used all feature sets gave

99.9% accuracy. Tables 13 and 14 gives the specific fields from the LARS field

data base combined for the two data sets.

Classes

Spring Wheat

Field

118

154

199

291

292

Number of Observations

13

29

28

28

16

Total = 114

SW 1809

SW Mix

296

303

75

281

28

58

Total - 86

13

55

Total = 68

Table 13. Classes and fields used to compute statistics for the Spring Wheat

test scene. The data is from Hand County, South Dakota, on July 26, 1978.

Classes

Barley

Pasture

Grain Sorghum

Field

153

231

294

109

194

198

110

256

275

Number of Observations

26

21

55

Total = 102

43

45

24

Total = 112

15

51

36

Total = 102

Table 14. Classes and fields used to compute statistics for the Crops 3

test scene. The data is from Hand County, South Dakota, on July 26, 1978. In

simulating the scene, the mean vector of the Barley class was used for all three

classes.

35 4/20_g
Figure 37 shows the resulting accuracy for the various features and for

the two test scenes. The crops 3 scene was tested for IMC = 1 and 8 to see the

effect of increasing the SNR of the instrument on classification performance.

100

A

eo

v

O

¢1

L_

:= 80

O

O

<

¢=

0 70

1o

u

m

m

m

m 60

m

0

[] Crol_ 3 IMC=I

• cro_3=MC-a

50

SFD TM _h_NR NSNR _ SSNR

Feature Set

Figure 37. Classification accuracy for the various feature sets over two test

scenes.

In all cases the features formed from the wavelengths used in the

Landsat TM performed the best. The features derived from the high SNR

regions also performed well. Compared to these two feature sets, the SFD

feature set performed poorly.

This leads to a tentative conclusion that over varying scenes, the

features derived from the reflectance of a different crop type, or scene, perform

less well at classification than features derived from signal-to-noise regions of

the instrument, or even the wavelength bands used in the Thematic Mapper.

This is not surprising since the SFD procedure is intended to be case-specific; it

is intended to provide features optimal for its design case, as compared to being

optimal in the general case.

36 4_o_g
7. Summary and Conclusions

This report summarizes the current status of research into predicting the

performance of the HIRIS instrument under varying system parameters, signalto-noise conditions, and feature extraction methods.

Several tentative conclusions can be made.

• The current noise levels of the HIRIS instrument are adequate for use

in classification studies.

The radiometric resolution of 12 bits is well matched to the other

sources of noise. At 8 bits, the quantization error becomes the

dominant source of noise.

The SNR of a feature obtained by combining bands is a poor

predictor of the effectiveness of that feature alone in multivariate

classification algorithms.

• The combined SNR of a feature set is a "loose" predictor of

classification performance.

• Classification and separability are highly correlated.

The features derived from the spectral feature design algorithm work

well for a scene created from the statistics used in their design.

However, they are not broadly optimum, and can be outperformed by

more simply chosen feature sets.

Further work is necessary before etching these conclusions in stone.

37 4_o_9
e

[I]

[2]

[3]

[4]

[5]

[6]

[7]

References

Goetz, A.F.H., and M. Herring, "The High Resolution Imaging Spectrometer

(HIRIS) for Eos," IEEE Transactions on Geoscience and Remote Sensing,

Vol. GE-27, No. 2, pp. 136-144, March 1989.

Kerekes, J.P., and D.A. Landgrebe, "Simulation of Optical Remote Sensing

Systems," submitted to the IEEE Transactions on Geoscience and Remote

Sensing, for possible inclusion in the Special Issue on IGARSS '88 to be

published in November 1989.

JPL, "High-Resolution Imaging Spectrometer (HIRIS): Phase A Final

Report," JPL D-4782, Jet Propulsion Laboratory, California Institute of

Technology, Pasadena, CA, November 1987. Also, data was obtained from

viewgraphs used in a presentation by V. Wright of JPL on October 6, 1987 to

the Imaging Spectrometer Science Advisory Group (ISSAG).

Markham, B.L., "The Landsat Sensors' Spatial Responses,* /EEE

Transactions on Geoscience and Remote Sensing, vol. GE-23, No. 6, pp.

864-875, November 1985.

Kneizys, F.X. et. al., "Users' Guide to LOWTRAN 7," AFGL-TR-88-0177, Air

Force Geophysical Lab, Bedford, MA, August 1988.

Biehl, L.L., M.E. Bauer, B.F. Robinson, C.S.T. Daughtry, L.F. Silva, and D.E.

Pitts, "A Crops and Soils Data Base for Scene Radiation Research,"

Proceedings of the 8th International Symposium on Machine Processing of

Remotely Sensed Data, pp. 169-177, Purdue University, West Lafayette, IN,

1982.

Chen, C.-C., and D.A. Landgrebe, " A Spectral Feature Design System for

High Dimensional Multispectral Data," Proceedings of IGARSS '88

Symposium, pp. 891-894, Edinburgh, Scotland, 13-16 September 1988.

Also see Ph.D. thesis by C.-C. Thomas Chen, "Spectral Feature Design In

High Dimensional Multispectral Data," School of Electrical Engineering,

Purdue University, West Lafayette, IN, August 1988.

38 4/20/89