**The 'BandSystem' Spectral Analysis System**

P. R. Gazis

San Jose State University Foundation, NASA Ames Research Center, Moffett
Field, CA

**Abstract:** The 'BandSystem' spectral analysis system is a rule-based
system for the identification of constituents of vis/near-IR reflectance
spectra and mid-IR reflectance and thermal emission spectra. This system
consists of a preprocessor and a set of feature extractors that identify
possible spectral bands and deliver them to a rule-based system where
they are used for purposes of identification. With appropriate
modifications, this system could be used for a broad range of spectral
classification purposes.

**Introduction**

The spectral analysis system described below (hereafter referred to as
\'the system\') accepts a spectrum and returns a classification. This
system, in association with the appropriate set of identification rules,
identified the presence of carbonates in near-IR reflectance spectra
with performance comparable to a human expert during the 1999 Marsokhod
Field Experiment \[*Gazis and Roush*, 2000\]. The operation of this
system is controlled by a set of control parameters. These control
parameters are summarized in Appendix A. In general, the values of the
control parameters for a particular spectrum and application will be
specific to the instrument and mineral targets associated with that
application. These modules and control parameters are described below.

The input spectrum is stored in two arrays: one contains wavelength
channels while the other contains the corresponding spectral values. In
its current form, the system reads this information from one or two
ASCII files, each of which consists of a header followed by a data block
of wavelength and/or spectral values in single or multiple-column
format, one column for each parameter. The wavelength channels *must* be
sequential. In principle, they can be in either ascending or descending
order, but the system has only been tested with wavelengths in

ascending order.

Wavelengths and spectral values can be in a variety of units. In its
current form, the system scales wavelengths to um and reflectance values
to percentages. For internal purposes, it flags bad reflectance values
with a BAD DATA flag of -999.99.

The system is designed to fail gracefully if the spectrum is not
delivered in the proper format. Under such circumstances, it will appear
to function properly (e.g. it will not \'crash\') but it will deliver a
nonsensical result. *It is the responsibility of the calling program to
ensure that the*

*input spectrum is meaningful*.

The output of this system consists of a string that represents the
classification. The actual semantics of this string is determined by the
rule set used by the Spectrum Analyzer.

**I/O and overall organization**

This system is written in object-oriented fashion in C++, and consists
of a driver routine which invokes a succession of modules preprocess the
spectrum, extract features, and use these features to identify possible
constituents of the sample spectrum. Each module is embodied as a
separate class. Processing is performed by the constructors and member
functions of these classes. Information is passed between classes and to
i/o operations using access functions.

The system reads spectra from a set of input files and writes its
results (which may be spectra) to a set of output files. Reading of
input spectra is controlled by 6 parameters: \'waveFileSpec:\',
\'speFileSpec:\', 'nWaveHeader:\', \'nSpeHeader:\', iWaveColumn:\', and
\'iSpeColumn:\'.

\'waveFileSpec:\' is an ASCII string that contains the complete pathname
and filespec of the wavelength file. \'speFileSpec:\' is an ASCII string
that contains the complete pathname and filespec of the spectral file.
The wavelength and spectral files are each assumed to consist of a
header followed by a data block in a single or multi-column format, with
one record for each wavelength channel. Note that the wavelength and
spectral files can be obtained from the same file, in which

case \'waveFileSpec:\' and \'speFileSpec:\' will be identical.

\'nWaveHeader\' is an integer variable that specifies the number of
lines in the header of the wavelength file. If this parameter is not
specified, the system uses a default value of 2. \'nSpeHeader\' is an
integer variable that specifies the number of lines in the header of the
spectral file. If this parameter is not specified, the system uses a
default value of 2. Obviously, if \'waveFileSpec:\' and \'speFileSpec:\'
are identical, \'nWaveHeader:\' and \'nSpeheader:\' must also be

identical.

\'iWaveCol:\' is an integer variable that specifies the column of the
data block of the wavelength file in which the wavelength channel
information is stored. if this parameter is not specified, the system
uses a default value of 1 to refer to the first column of the data
block. \'iSpeCol:\' is an integer variable that specifies the column of
the data block of the spectral file in which the spectral channel
information is stored. If this parameter is not specified, the system
uses a default value of 2 to refer to the second column of the data
block.

**Preprocessor**

The Preprocessor prepares the spectrum for use by subsequent modules of
the system. In principle, it could also be modified to check each
spectrum for format errors and analogous problems. Input to the
Preprocessor consists of two arrays: one containing wavelength channels
and the other spectral values, both in some predefined units. Output
consists of two corresponding arrays: one containing wavelength channels
in μm and the other reflectance values in

percentages. *WARNING:* The current version of the Preprocessor does not
contain any provision to recognize and re-label any bad data it might
receive from the calling application!

The Preprocessor performs the following steps:

1\) Multiply wavelength channels by a scaling constant to convert them
to the appropriate units. In principle, the Preprocessor could convert
wavelength channels to any units, but subsequent modules of this system
expect wavelengths to be expressed in μm. This step is controlled by a
single control parameter, \'waveUnitScale:\'. This a double precision
floating point variable that contains the scaling constant described
above. This parameter is instrument specific.

2\) Multiply spectral values by a scaling constant to scale them to the
appropriate units and flag any values that lie above some upper limit
with the a BAD DATA flag (-999.999). For reasons of efficiency, these
two operations are both performed within the same iteration loop. In
principle, the Preprocessor could scale the spectral values to any
units, but subsequent modules of the system expect to receive
reflectance values in the form of percentages. This step is controlled
by two control parameters: \'valueUnitScale:\' and
\'absoluteUpperLimit:\'. The first parameter, \'valueUnitScale\', is a
double precision floating point variable that contains the scaling
constant described above. This parameter is instrument specific. The
second parameter, \'absoluteUpperLimit\', is a double precision floating
point variable that contains the upper limit described above.

3\) Calculate the average albedo of the spectrum (ignoring data flagged
as bad), then multiple reflectance values by a constant scaling factor
to rescale the spectrum to a user-defined average albedo. This step is
controlled by a single control parameter, \'normalizedAlbedo:\'. This a
double precision floating point variable that contains the user-defined
average albedo described above. If this parameter is not specified, this
step is not performed. This parameter is instrument- and
application-specific.

4\) Store the preprocessed spectrum as two arrays, wavelengths in μm and
percentage reflectance values as described above, for use by other
modules in the system.

**Band Feature Extractor**

This module applies a procedure based on the method of *Grove et al.*
\[1992\] to identify and characterize possible spectral bands. This
module performs a sequence of operations to process an input spectrum.
It stores the input spectrum and the intermediate results from each
operation and in a series of arrays, each of which contains a spectrum.
One of these arrays (specified the input parameter, \'spectrumType:\'
described below) is selected for use as the final processed spectrum. In
principle, any of the arrays could be chosen for this purpose, even the
one that contains the raw input spectrum, but in practice, for purposes
of carbonate detection, the smoothed spectrum produced by Step 2)
described below is used. The feature extraction algorithm is applied to
this spectrum to produce a list of spectral bands and associated
characteristics. Details of the feature extraction algorithm are
summarized in Appendix C.

This module performs the following operations:

1\) Clip the raw input spectrum and store the clipped raw spectrum in an
array, \'rawSpec\'. This step is controlled by two control parameters,
\'minWave:\' and \'minWave:\'. These are double precision floating point
variables that specify the range, in microns, of the clipped raw
spectrum. If these values are not specified, the module uses default
values of 0.0 and 1.0E+06 respectively. These parameters are instrument
and application specific

2\) Smooth the spectrum using a boxcar average and store the result in
an array, \'smoothedSpec\'. This step is controlled by a single control
parameter, \'nSmoothed:\'. This is an integer variable that controls the
number of wavelength channels to be used for the boxcar average. This
parameter is instrument and application specific.

3\) Smooth the spectrum using a boxcar average to generate a background
and store the result in an array, \'backgroundSpec\'. This step is
controlled by a single control parameter, \'nBackground:\'. This is an
integer variable that controls the number of wavelength channels to be
used for the boxcar average to generate a background. Note that
'nBackground' can equal 'nSmoothed', in which case the background and
smoothed spectrum will be identical. This parameter is instrument and
application specific.

4\) If requested, calculate a hull fit to the background spectrum
calculated in Step 3) and store the result in the array,
\'backgroundSpec\'. This step is controlled by two parameters:
\'fitUpperHull:\' and \'hullChannelWindow:\'. The first parameter,
\'upperOrLowerHull:\' is an ASCII string, \'upper\' or \'lower\', which
specifies whether the hull fit should be applied to the top or the
bottom of the spectrum. If this parameter is not specified, the module
uses a default value of \'upper\'. This parameter was introduced for
development purposes, and we have yet to identify any circumstances for
which the default parameter is inappropriate. The second parameter,
\'hullChannelWindow:\', is an integer variable that controls the width
of the window within which the hull fit is applied. If this value is
less than the number of channels in the spectrum, the window is slid
along the spectrum to produce a modified hull fit that could be concave.
If this value is not specified, the module uses a default value of 5000.
This parameter was introduced for development purposes. It may be
instrument or application specific, but we have yet to identify any
circumstances for which this window should be narrower than the entire
spectrum.

5\) Subtract the background spectrum, 'backgroundSpec', from the
smoothed spectrum 'smoothedSpec', to generate a difference spectrum,
'differenceSpec'.

6\) Divide the smoothed spectrum, 'smoothedSpec', by the background
spectrum, 'backgroundSpec', to generate a difference spectrum,
'differenceSpec'.

7\) Select the spectrum from which features are to be extracted and
store it in an array, \'proSpec\'. This step is controlled by a single
parameter, \'spectrumType:\'. This is an ASCII string, \'raw\', \'box\',
\'hf\', \'hd\', or \'hq\' which selects as a source either the raw
spectrum, smoothed spectrum, hull fit, hull difference, or hull
quotient. If this parameter is unspecified or unrecognized, the module
uses a default value of \'hd\'. This parameter was introduced for
development purposes. It is instrument and application specific. In the
current usage of this system for purposes of carbonate detection, this
parameter is set to \'box\' and the system uses the smoothed spectrum
calculated in Step 2).

8\) Apply first step of the feature extractor to the final processed
spectrum to identify possible bands. This step identifies local minima
and inflection points that could represent the centers of bands. This
step is controlled by five input parameters: \'minDepth:\',
\'orderFeaturesBy:\', \'nSlopeWidth:\', \'deltaFirstIP:\', and
\'deltaSecondIP:\'. \'minDepth:\', is a double precision variable that
specifies a threshold. Bands with a depth below this threshold will be
discarded. This parameter is instrument and application specific.
\'orderFeaturesBy:\', is an ASCII string which can take the values
\'wavelength\' or \'depth\'. \'nSlopeWidth:\' is an integer variable
that controls the number of channels over which the slope of the
spectrum is averaged for the determination of extrema and inflection
points. \'deltaFirstIP:\' and \'deltaSecondIP:\', are double-precision
variables that specify the minimum values required for a pair of
inflection points to define a shoulder in the spectrum.

9\) Apply the third step of the feature extractor to discard possible
bands that may be associated with noise. This step is controlled by
three parameters: \'variationLimit:\' \'rejectNeighborWidth:\', and
\'noiseSpikeWidth:\'. These are double-precision floating point
variables. The use of these parameters will be described in a later
revision of this document.

10\) Store a list of \'band features\' in a format similar to that used
for the \'noise features\' described above. This list of features
consists of a list of successive lines of ASCII text, each of which is
enclosed by a pair of parentheses and describes a single band. The
entire list is enclosed by an outer pair of parentheses:

((LINEB000 1 2.490 33.963 2.393 17.162 2.490 33.963 2.357 0.000 0.097 T)

(LINEB001 2 2.322 22.804 2.089 0.000 2.393 17.162 3.203 1.266 0.304 T)

(LINEB002 3 2.000 4.280 2.000 4.280 2.089 0.000 0.178 0.000 0.089 T))

Each token in this string represents a \'parameter\' of the \'band
feature\'. Note that some of these parameters may be redundant, or
irrelevant for particular applications. The first token is an ASCII
string that contains the name for this feature, followed by a 3-digit
0-based index for this feature. The name is not determined by the
control parameter \'rootLineName:\', which was set to \'LINEB\' for the
example shown above. The indices will always begin with \'000\'. The
second token is an integer that contains the ordinal number of this
feature. These ordinals will always begin with \'1\'. The next three
tokens are double precision floating point variables that contain the
central, left, and right-hand values for the wavelength range of the
band. The next three tokens contain the integrated area, \'asymetry\',
and \'width\' for this band. The final token is a noise flag for which
\'T\' and \'F\' correspond to \'clean\' and \'noisy\'.

**Spectrum Analyzer**

This module is a simple forward-chaining expert system \[*Winston,*
1984\] that applies a set of rules to the list of noise and band
features described above to classify each spectrum. This module is
described in Appendix D. This module is not instrument-specific, and in
principle it could be used to perform any classification for any
instrument and application for which data could be represented as a
modest list of features and classification could be accomplished by a
set of deterministic rules.

This module performs the following operations:

1\) Apply a set of rules to the list of input features to classify the
spectrum. This step is controlled by three parameters:
\'ruleFileSpec:\', \'factFileSpec01:\', and \'factFileSpec02:\', which
specfiy the rule file and the two input files that contain the noise and
band features.

2\) Write the resulting classification to an output file specified by
the control parameter \'outFileSpec:\'. This output file will consist of
a list of facts in a LISP-style format, enclosed by nested pairs of
parentheses, such as:

((SAMPLE IS CARBONATE)

(CERTAINTY IS HIGH)

(MINERALCLASS: 1))

Note that if the module was unable to assert any results, this list will
be empty, and consist of a single pair of empty parentheses \'()\'!

**Output**

In the current version of the system, this output has three values: \'to
noisy to analyze\', \'non-carbonate\', and \'carbonate\'. *NOTE:* The
noise determination made by the system is delivered

here!

**Appendix A: Control Parameters for the 'BandSystem' Spectral Analysis
System**

This document describes the control parameters for the October99
spectral analysis system in alphabetical order.

absoluteUpperLimit: 1000.0 \-- Upper limit for scaled spectrum

archiveFileSpec: librarian.out

extractorFileSpec_0: extractorA.jbishop.in

extractorFileSpec_n: extractorA.jbishop.in

iSpeColumn: 2

iWaveColumn: 1

maxWave: 2.500 \-- Maximum wavelength

minWave: 0.0 -- Minimum wavelength

nSpeHeader: 10 \-- \# of lines in header of spectral file

normalizedAlbedo: 50.0 \-- Normalize albedo to this value

nWaveHeader: 10 \-- \# of lines in header of wavelength file

outPathName: temp\\

processAllFiles: Y

ruleFileSpec: rulelist.txt \-- Filespec for rule list

sourceFileSpec: \*.out

sourcePathName: g:\\mid_ir\\ASTER\\

valueUnitScale: 1.0 \-- Scale spectral values by this factor

waveUnitScale: 1.0 \-- Scale wavelengths by this factor

writeFacts: Y

writeProcessedSpectrum: Y

writeSpectrum: Y

writeVariableList: N

backgroundType: smoothed

deltaFirstIP: 0.0 \-- Minimum threshold for the first inflection point
(in scaled spectrum and

wavelength) for test to identify the edges of a band

deltaSecondIP: 5.0 \-- Minimum threshold for the 2nd inflection point
(in scaled spectrum and

wavelength) for test to identify the edges of a band

findEmissionFeatures: N

findInflectionPoints: N

fitUpperHull: Y

maxNLines: 45 -- Maximum number of bands to be identified

minDepth: 0.5 \-- Minimum depth (in scaled spectral values) below which
possible bands will not

be accepted

nBackground: 100 \-- \# of channels for boxcar average of background

noiseSpikeWidth: 0.010 \-- Width (in scaled wavelength) used for the
noise test described above

nSlopeWidth: 20 \-- \# channels over which spectrum is averaged for the
calculation of slope

nSmoothed: 10 \-- \# of channels for boxcar average of spectrum

orderFeaturesBy: wavelength \-- Order features by depth or wavelength

outPathName: temp\\

rejectNeighborWidth: 0.15 \-- Distance (in scaled wavelengths) below
which potential features will

be rejected if they lie close to a feature (but not a noise spike!) that
has already been identified

rootLineName: LINEB \-- Root name for features produced by this module
(e.g. \'LINEB000',

'LINEB001\', etc.)

spectrumType: hq \-- Type of spectrum from which features are to be
extracted: raw, boxcar

averages, hull fit to boxcar averages, or boxcar averages from which
hull fit has been

subtracted

%variationLimit: 100.0 \-- Range (in scaled spectral values) below which
potential features will be

rejected as noise if they are closer than \'noiseSpikeWidth:\' defined
below to a feature or noise

spike that has already been identified

vScaleValue: 100.0

writeBands: Y

**Appendix B: Spectral Band Feature Extraction and Noise Removal
Algorithms**

The feature extractor applies an algorithm based on the method of *Grove
et al.* \[1992\] to identify and characterize possible spectral bands.
This algorithm has been modified to incorporate two ad hoc procedures to
remove noise-like features associated with small fluctuations in the
spectrum. One of these procedures a discards a diminishing succession of
local minima as potential noise spikes as long as 1) the depth of each
successive local minimum is less than the depth of its predecessor and
2) each successive local minimum is within a wavelength range
'noiseSpikeWidth' of its predecessor. The other procedure simply
discards any local minimum within a avelength range
'rejectNeighborWidth' of the band center as a potential noise feature.

This algorithm proceeds as follows:

1\) Copy the input spectrum array to a test array. Elements of this test
array

will be used for purposes of calculation and as flags to indicate what

portions of the spectrum have been examined by the feature extraction

algorithm.

2\) Determine the maximum and minimum values, 'vmax' and 'vmin', of the
spectrum.

3\) Identify and characterize the next band.

3.1) Find the next deepest local minimum of the spectrum. If no local

minimum was found, or if the depth of this local minimum was less than a

threshold, 'threshold', then quit.

3.2) Set values of the index, 'iCenter', wavelength, 'vWavelength', and

spectrum, 'vCenter' to identify this array element as the center of this

band.

3.3) Initialize the test depth for diminishing noise spikes,
'vLocalMin', to

'vCenter'.

3.4) Examine successive elements to the left of the band center to
locate the

left edge of this band.

3.41) If this element is at the left edge of the spectrum or is equal to

the maximum value, 'vmax', of the spectrum, identify this element as

the left edge of this band and quit.

3.42) If this element is a local minimum with a depth less than the last

test depth, 'vLocalMin', of the preceeding local minimum, then reset

'vLocalMin' to the depth of this local minimum and continue.

3.43) If this element is within a wavelength range 'rejectNeighborWidth'
of

the band center then continue.

3.44) Calculate slopes to the left and right of this element.

3.45) If this is a local maximum, identify this as the left edge of the

band.

3.46) If this is the first inflection point that has been encountered
and

it involves a change in slope greater than a threshold 'deltaFirstIP',

record that an inflection point was encountered.

3.47) If this is the second inflection point that has been encountered
and

it involves a change in slope greater than a threshold

'deltaSecondIP', (e.g., this is a 'shoulder' in the spectrum),

identify this as the left edge of the band and quit.

3.47) Set the value of this element of the test array to 'vmax' to
indicate

that it has been evaluated.

3.5) Reset the value of the test copy corresponding to the band center
to

'vCenter' so that the search for the right edge of the band can be

conducted.

3.6) Examine successive elements to the right of the band center to
locate

the right edge of this band.

3.7) Evaluate parameters of this band: the central, left, and right-hand

values for the wavelength range; the integrated area, \'asymmetry\', and

\'width\', and a noise flag.

The feature extractor returns a list of \'band features\'. This list
consists of a list of successive lines of ASCII text, each of which is
enclosed by a pair of parentheses and describes a single band. The
entire list is enclosed by an outer pair of parentheses:

((LINEB000 1 2.490 33.963 2.393 17.162 2.490 33.963 2.357 0.000 0.097 T)

(LINEB001 2 2.322 22.804 2.089 0.000 2.393 17.162 3.203 1.266 0.304 T)

(LINEB002 3 2.000 4.280 2.000 4.280 2.089 0.000 0.178 0.000 0.089 T))

Each token in this string represents a \'parameter\' of the \'band
feature\'. Note that some of these parameters may be redundant, or
irrelevant for particular applications. The first token is an ASCII
string that contains the name for this feature, followed by a 3-digit
0-based index for this feature. The name is not determined by the
control parameter \'rootLineName:\', which was set to \'LINEB\' for the
example shown above. The indices will always begin with \'000\'. The
second token is an integer that contains the ordinal number of this
feature. These ordinals will always begin with \'1\'. The next three
tokens are double precision floating point variables that contain the
central, left, and right-hand values for the wavelength range of the
band. The next three tokens contain the integrated area, \'asymmetry\',
and \'width\' for this band. The final token is a noise flag for which
\'T\' and \'F\' correspond to \'clean\' and \'noisy\'.

**Appendix C: Rule Representation in the Classification Module**

The spectral classification module is a simple forward-chaining expert
system \[*Winston,* 1984\] that applies a set of rules to the list of
noise and band features described above to classify each spectrum. These
noise features, band features, and rules are all represented as strings
of ASCII characters in a LISP-style format. The formats of the noise and
band features are described in previous sections of this report. The
format used to represent rules is described below.

Each rule consists of three blocks: a rule name, a conditions block
('IF' block), and a results block ('THEN' block):

(RULE006 (IF (LINEB\* (\<= 2) (2.33 +/- 0.03) \* \* \* \* \* \* \* \* T)

(LINEB\* (\<= 2) (2.50 +/- 0.02) \* \* \* \* \* \* \* \* T))

(THEN (SAMPLE IS CARBONATE)

(CERTAINTY IS HIGH)))

The rule name is included as a convenience for developers, and is not
used by the classification module. The conditions block contains a list
of conditions that must be met for the rule to be satisfied. The results
block contains a list of facts that will be added to the fact list if
the rule is satisfied.

Conditions are compared with known facts using a simple pattern matcher.
This pattern matcher compares each successive element of the condition
to be tested with the corresponding element of the fact with which is to
be compared. This comparison is successful if one of the following four
conditions is satisfied: 1) exact match (e.g. condition element
'LINEB001' with fact element 'LINEB001'), 2) partial match with a
trailing wildcard in the condition (e.g., condition element 'LINEB\*'
with fact element 'LINEB001'), 3) arithmetic relation \<, \<=, ==, \>=,
\> (e.g. condition '(\<= 2)' with fact '1'), or 4) arithmetic range
(e.g. condition '(2.33 +/- 0.03)' with fact '2.37'). If all of the
comparisons are successful, the condition is satisfied.