Adaptive Target Detection in FLIR Imagery Using 

the Eigenspace Separation Transform and Principal 

Component Analysis 

by S. Susan Young, Heesung Kwon, and Nasser M. Nasrabadi 

ARL-TR-3544 June 2005 

Approved for public release; distribution unlimited. 
NOTICES 

Disclaimers 

The findings in this report are not to be construed as an official Department of the Army position 

unless so designated by other authorized documents. 

Citation of manufacturer’s or trade names does not constitute an official endorsement or 

approval of the use thereof. 

Destroy this report when it is no longer needed. Do not return it to the originator. 
Army Research Laboratory 

Adelphi, MD 20783-1197 

ARL-TR-3544 June 2005 

Adaptive Target Detection in FLIR Imagery Using 

the Eigenspace Separation Transform and Principal 

Component Analysis 

S. Susan Young, Heesung Kwon, and Nasser M. Nasrabadi 

Sensors and Electron Devices Directorate, ARL 

Approved for public release; distribution unlimited. 
ii 

REPORT DOCUMENTATION PAGE Form Approved 

OMB No. 0704-0188

Public reporting burden for this collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data 

needed, and completing and reviewing the collection information. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing the burden, 

to Department of Defense, Washington Headquarters Services, Directorate for Information Operations and Reports (0704-0188), 1215 Jefferson Davis Highway, Suite 1204, Arlington, VA 22202-4302. 

Respondents should be aware that notwithstanding any other provision of law, no person shall be subject to any penalty for failing to comply with a collection of information if it does not display a currently valid 

OMB control number. 

PLEASE DO NOT RETURN YOUR FORM TO THE ABOVE ADDRESS. 

1. REPORT DATE (DD-MM-YYYY)

June 2005 

2. REPORT TYPE 

Final 

3. DATES COVERED (From - To) 

5a. CONTRACT NUMBER 

5b. GRANT NUMBER 

4. TITLE AND SUBTITLE 

Adaptive Target Detection in FLIR Imagery Using the Eigenspace Separation 

Transform and Principal Component Analysis 

5c. PROGRAM ELEMENT NUMBER 

5d. PROJECT NUMBER 

5e. TASK NUMBER 

6. AUTHOR(S) 

S. Susan Young, Heesung Kwon, and Nasser M. Nasrabadi 

5f. WORK UNIT NUMBER 

7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES) 

U.S. Army Research Laboratory 

ATTN: AMSRD-ARL-SE-SE 

2800 Powder Mill Road 

Adelphi, MD 20783-1197 

8. PERFORMING ORGANIZATION 

 REPORT NUMBER 

ARL-TR-3544 

9. SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES) 10. SPONSOR/MONITOR'S ACRONYM(S) 

U.S. Army Research Laboratory 

2800 Powder Mill Road 

Adelphi, MD 20783-1197 

11. SPONSOR/MONITOR'S REPORT 

 NUMBER(S) 

12. DISTRIBUTION/AVAILABILITY STATEMENT

Approved for public release; distribution unlimited. 

13. SUPPLEMENTARY NOTES 

14. ABSTRACT 

In this paper, an adaptive target detection algorithm for forward-looking infrared (FLIR) imagery is proposed which is based on

measuring differences between structural information within a target and its surrounding background. At each pixel in the image 

a dual window is opened where the inner window (inner image vector) represents a possible target signature and the outer 

window (consisting of a number of outer image vectors) represents the surrounding scene. These image vectors are then 

preprocessed by two directional highpass filters to obtain the corresponding image gradient vectors. The target detection 

problem is formulated as a statistical hypotheses testing problem by mapping these image gradient vectors into two linear 

transformations, P1 and P2 and, via principal component analysis and eigenspace separation transform, respectively. The first 

transformation P1 is only a function of the inner image gradient vector. The second transformation P2 is a function of both the 

inner and outer image gradient vectors. For the hypothesis H1 (target): the difference of the two functions is small. For the 

hypothesis H0 (clutter): the difference of the two functions is large. Results of testing the proposed target detection algorithm on 

two large FLIR image databases are presented. 

15. SUBJECT TERMS

target detection, eigenvector analysis, eigenspace separation transform, principal component analysis, FLIR imagery, statistical 

hypotheses testing 

16. SECURITY CLASSIFICATION OF:

19a. NAME OF RESPONSIBLE PERSON

S. Susan Young 

a. REPORT 

Unlcassified 

b. ABSTRACT 

Unclassified 

c. THIS PAGE 

Unclassified 

17. LIMITATION 

OF ABSTRACT 

SAR 

18. NUMBER

OF PAGES 

26 19b. TELEPHONE NUMBER (Include area code) 

301-394-0230 

Standard Form 298 (Rev. 8/98) 

Prescribed by ANSI Std. Z39.18
iii 

Contents 

List of Figures iv

List of Tables iv

1. Introduction 1

2. FLIR Imagery Feature Set 3

2.1 Image Vectors..................................................................................................................3 

2.2 Gradient Image................................................................................................................4 

3. Adaptive Detection of Target in FLIR Imagery 5

3.1 EST and PCA ..................................................................................................................5 

3.2 Hypotheses ......................................................................................................................7 

3.3 Target Detection Procedure.............................................................................................9 

4. Results and Discussion 10

4.1 Image Database .............................................................................................................10 

4.2 ROC Results..................................................................................................................12 

4.3 Discussion .....................................................................................................................14 

5. Conclusions 17

6. References 18

Distribution List 20
iv 

List of Figures 

Figure 1. Image vectors construction............................................................................................. 4 

Figure 2. Examples of gradient images: (a) original image, (b) gradient image obtained by the 

highpass filtering in the x direction, and (c) gradient image obtained by the highpass 

filtering in the y direction........................................................................................................ 5 

Figure 3. Examples of EST and PCA. ........................................................................................... 6

Figure 4. Visualization of hypotheses............................................................................................ 7 

Figure 5. Procedure of the target detection algorithm. .................................................................. 9 

Figure 6. Examples of FLIR images: (a) an image from the database yuma9207_roi, where the 

background contains many high temperature spots and (b) an image from the database 

huli9204_roi, where the background is cool compared with the targets in the image.......... 11 

Figure 7. (a) Overlapping area ratio. The solid line window corresponds to an inner window 

that is centered at a target coordinates. The bold dashed line window corresponds to one of 

the inner windows that the algorithm selects. Overlapping area ratio is defined as the ratio 

of the overlapping area of these two windows to the area of a single window, (b) normalized 

histogram of the overlapping area ratios for short ranges, (c) normalized histogram of the 

overlapping area ratios for medium ranges, and (d) normalized histogram of the overlapping 

area ratios for long ranges..................................................................................................... 13 

Figure 8. ROC results from the proposed implementation: (a) database huli9204_roi and (b) 

database yuma9207_roi......................................................................................................... 14 

Figure 9. ROC results of the proposed implementation comparing with the ideal situation for the 

database yuma9207_roi: (a) ROC results of false alarm rates between 0 and 25; (b) ROC 

results of false alarm rates between 0 and 5. ........................................................................ 15 

Figure 10. ROC results of the proposed algorithm comparing with the SADA for the database 

yuma9207_roi: (a) ROC results of false alarm rates between 0 and 25 and (b) ROC results 

of false alarm rates between 0 and 5..................................................................................... 16 

Figure 11. Detection results: (a) gradient image obtained by the highpass filtering in the x 

direction, (b) residual energy Emin values, and (c) the points with Emin < 0.3 are 

superimposed into the gradient image in (a) as markers (+)................................................. 17 

List of Tables 

Table 1. Database......................................................................................................................... 11 
1 

1. Introduction 

The structural information of man-made targets (ground-based vehicles) in forward-looking 

infrared (FLIR) imagery appears different from that of their surrounding background. Automatic 

target detection techniques use the radiated energy from a man-made target in FLIR imagery to 

detect targets (1-2). Energy distribution patterns on the targets are determined by the operational 

conditions of the vehicles, weather conditions and solar loading, atmospheric conditions, and 

other factors. Similarly, the apparent temperature of background objects is determined by 

environmental conditions. 

The research work in infrared imaging system and automatic target detection/recognition 

(ATD/R) were started approximately at the same time nearly 40 years ago; see the review paper 

by Ratches et al., (3). The survey papers by Bhanu (4-5) summarize various algorithms for 

ATD/R in static FLIR images that were developed up to early 1990s; these algorithms 

predominately use traditional image processing approaches for (optical) picture processing. 

Since early 1990s, combinations (or mixtures) of classical/traditional image processing and 

emerging techniques have been proposed for the FLIR target detection problem. For instance, 

the traditional approach of segmenting the target from the background still draws attention of the 

FLIR researchers. Meanwhile, various emerging techniques are also being studied in 

segmentation algorithm. For example, Sang et al., (6) use Hopfield neural network with edge 

constraint, and Sun et al., (7) exploit fuzzy thresholding and edge detection in their segmentation 

algorithms. 

There have also been studies of “matched filtering” algorithms that assume a specific 

shape/model for the FLIR signature of a man-made target. For instance, Erinsse et al., (8) use 

DOG (difference of Gaussian) filter as a bandpass filter to enhance the signature of man-made 

targets in FLIR imagery, and use the resultant information to detect (“pull”) these targets. Park 

et al., (9) assume that a man-made target exhibits a peak response when a Mexican hat filter is 

applied to FLIR imagery; in this case, the Mexican hat filter is used as the mother wavelet of the 

multidimensional wavelets. The filtered image is thresholded to obtain the location of man-made 

targets. Zhou et al., (10) use five simple matching patterns (shapes of rectangle, square, oval, 

rounded-rectangle, and circle) for all targets. They applied the Gabor function (a sinosoidal 

function weighted by a Gaussian function) to both their matching patterns and the IR image to 

generate Gabor feature vectors. The similarity measure between the Gabor feature vector at each 

image point and that of the matching patterns is calculated. The similarity values are then 

thresholded to obtain the locations of the targets. Weber et al., (11) use six Gabor function 

filters; these are two directional RGFs (real Gabor functions) to detect object height and width 

and four directional IGFs (imaginary Gabor functions) to detect the left, right, top and bottom 

edges of the object. The input image is correlated with each of the six Gabor filters, and these 
2 

six correlation outputs are then quadratically combined; the final detection is performed using a 

threshold. 

Neural network technologies are usually used as a classifier for an automatic target recognition 

system; see Roth (12) and Rizvi et al., (13) for a survey. Neural networks have also been 

exploited to classify each pixel in the FLIR imagery as either man-made structures or 

background (clutter) in order to detect targets. For example, see the articles by Dwan et al., (14) 

and Ramanan et al., (15). 

Multiscale fractal method is another emerging technique that has been used for FLIR target 

detection. Xue et al., (16) compute fractal dimensions through several resolution levels of an 

image. The K-mean method is then used to classify the target and background based on the 

resultant fractal dimensions. Shekarforoush et al., (17) have also used a multi-fractal formalism 

for object detection and tracking in FLIR sequences. 

The apparent temperature contrast between a man-made target and its surrounding is a key factor 

in most FLIR target detection algorithms. A target with high contrast (temperature) is relatively 

easy to detect; this property is exploited by most of the above-mentioned algorithms. However, 

for low contrast targets, some form of structural information needs to be exploited. In most of 

the above-mentioned algorithms, the image characteristics are globally analyzed either in the 

filtering operation or in the feature extraction process. This is achieved by comparing the pixels 

representing the target with the other pixels in the entire image. This global-based approach 

neglects the local variations between the signatures of the target and its surrounding medium. 

Some researchers have used double-gated filters that exploit these local variations for target 

detection: see Gregoris et al., (18) and Der et al., (19). In these methods, the authors identify an 

inner target window surrounded by an outer window. The inner window carries information on 

the FLIR signature properties of the target zone, and the outer window is used to identify the 

signature properties of the nearby background. In reference (18), a filter is used to measure the 

difference in the mean pixel values of the inner target window and the outer background 

window, and the pixel standard deviation in the background window. The ratio of the mean 

difference to the standard deviation, that is, a “normalized” mean difference, is compared to a 

pre-assigned threshold. The inner windows with ratios exceeding the threshold are labeled as 

possible man-made targets. In reference (19), more comprehensive features (such as contrast 

difference, gradient strength, straight edge information, etc.) are computed from the inner 

window and the outer window, and exploited for target detection purposes. 

In this paper, the local similarity measure between the inner and outer window is used to explore 

the differences between structural information of a target and its surrounding scene in FLIR 

imagery. Due to the signature variability of FLIR images, the presence of individual structural 

patterns, such as edges, shapes, textures, etc., cannot be reliably predicted. We use an 

eigenspace analysis to represent the variations of the structural information of a target and its 

background. We first construct the inner and outer image vectors that represent the multi-
3 

dimensional signal properties of the FLIR signatures of the target and its background, 

respectively. While the inner image vector is solely constructed from the inner window (box), 

the outer image vectors are constructed from a more extended outer window that consists of 

many overlapping outside boxes that are of the same size as the inner box. These two image 

vectors are then processed by two directional gradient operators. The resulting image gradient 

vectors are mapped by two transformations, P1 and P2, via principle component analysis (PCA) 

and the eigenspace separation transform (EST) (20), respectively. The first transformation P1 is 

a function of the inner image vector. The second transformation P2 is a function of both the 

inner and outer image vectors. The target detection problem is formulated as a statistical 

hypothesis testing problem. That is, for the hypothesis H1 (inner image vector is a target): the 

difference between the components of P1 and P2 is small. For hypothesis H0 (inner image vector 

is clutter): the difference between these two functions is large. 

This paper is organized as follows. The image feature set is described in section 2 using image 

vector construction and gradient signals. The adaptive target detection algorithm is discussed in 

section 3, where EST and PCA are outlined, and the hypotheses and target detection procedure 

are presented in detail. In section 4, results of testing the proposed algorithm on two large FLIR 

image databases are presented using ROC (receiver operation characteristics) curves. The 

proposed algorithm is also compared with other detection algorithms. Conclusions are given in 

section 5. 

2. FLIR Imagery Feature Set 

In this section, we describe how the image vectors are constructed to represent both a target 

region and it surrounding scene. Directional gradient filters are used to preprocess the image 

vectors. The resulting gradient image vectors are used as the feature set in our target detection 

algorithm. 

2.1 Image Vectors 

For a given pixel in the image, a window (called inner box) centered at the pixel is constructed as 

shown in figure 1. The size of the inner box is determined by the largest target size in the target 

library, which is based on a known or estimated range. Similarly, an outer window, Xout, as 

shown in figure 1, is constructed by a bigger window that surrounds but does not include the 

inner window. The size of the outer window in our implementation is three times the width and 

height of the inner window. The outer window is then partitioned into N small overlapping outer 

boxes (shift by 1 pixel in x or y direction) which have the same size as the inner box. The inner 

image vector that is denoted by Xin is the vectorized image data of the inner box. It is written as 

 ] [ , , , in 1 2 m X = x x L x
4 

Where m = w × h, w and h are the width and height of the inner box, and xj is the image pixel 

value, j = 1,2,...,m. 

Figure 1. Image vectors construction. 

The outer window, Xout, which contains N smaller outer boxes represented by xout,i where i = 

1,2,...,N. Each of the N small outer boxes is the vectorized image data representing each of N

overlapping boxes that have the same dimension m as the inner box. It is written as 

[ , , , ] out out,1 out,2 out,N X = x x L x

xout,i = [ y1 y2 ,Lym ], i = 1,2,L, N

where yj is the image pixel value, j = 1,2,...,m. 

2.2 Gradient Image 

The performance of many image processing algorithms is improved by first applying gradient 

operators to enhance the edge information within the image. Many studies of the human visual 

system provide evidence that the brain extracts edge and motion information early in the visual 

processing (21-22). Our experiments likewise show that the proposed algorithm performs much 

better using the gradient images than the pixel gray levels of the original FLIR imagery. 

Two gradient images are formed by passing the input image through two directional high pass 

filters. Filter Fh is designed to high pass filter the image in the horizontal direction x. Similarly, 

Fv is the vertical high pass filter used to enhance the vertical edges in the y direction. The two 

directional high pass filters are defined as two separable differentiation operations, that is, 

 . 1

1

1/ 2[ 1 1]; 1/ 2 



 



− Fh = − Fv =

Examples of gradient images are shown in figure 2. Note that the gradient (edge) information 

about the target (truck) in all directions is preserved. However, the edge information about hot 

roads are suppressed in the x direction but emphasized in the y direction. Usually a target 

contains edge information in all directions. The background scene might only contain edge 

information in a particular direction. In this paper, the gradient images in both directions are 

passed to the detection technique where a detection decision is made by using the detection result 

from both of the input gradient images. 
5 

Figure 2. Examples of gradient images: (a) original image, (b) gradient image obtained 

by the highpass filtering in the x direction, and (c) gradient image obtained by 

the highpass filtering in the y direction. 

3. Adaptive Detection of Target in FLIR Imagery 

The two image vectors Xin and Xout that were defined above are mapped into two transformations 

P1(Xin) and P2(Xin, Xout) via EST and PCA, respectively. Note that P1(Xin) is a function of Xin

only, while P2(Xin, Xout) is a function of Xin and Xout. 

The target detection problem is formulated as a hypothesis test. For hypothesis H1: target being 

within the inner box, the difference between P1(Xin) and P2(Xin, Xout) should be small. For 

hypothesis H0: clutter being within the inner box, the difference between these two functions 

should be large. That is: 

= − > δ 2

0 1 2 ( ) ( , ) H P Xin P Xin Xout (1) 

= − < δ 2

1 1 2 ( ) ( , ) H P Xin P Xin Xout

where δ is pre-selected as a threshold. The two functions P1(Xin) and P2(Xin, Xout) that are 

obtained via EST and PCA are discussed in the following subsection. 

3.1 EST and PCA 

The EST has been proposed by Torrieri as a preprocessor to a neural binary classifier in (20). 

The EST calculates the difference covariance matrix C 

C = Cout −Cin
6 

where Cout and Cin are the covariance matrices of the outer and inner image vectors, respectively. 

That is, 

[ ][ ]T

in in Xin in Xin C = X − μ X − μ

[ ][ ]T

out i X

N

i

out out i Xout i out i X X

N

C , , ,

1

,

1 = ∑ − μ − μ =

where μx is the mean value of either x = Xin or x = Xout,i. N is the number of small outer boxes 

forming the outer window. The eigenvalues and eigenvectors of difference covariance matrix C

are calculated. Some of these eigenvectors are associated with the positive eigenvalues. They 

are referred to as EST positive eigenvectors Vest+. The eigenvectors associated with the negative 

eigenvalues are referred to as EST negative eigenvectors Vest–. Based on the eigenvector 

properties, all EST positive and negative eigenvectors are orthogonal to each other. The positive 

eigenvectors Vest+ mainly represent the outer image vector subspaces. Similarly, the negative 

eigenvectors Vest– represent the inner image vector subspaces. In the design of the covariance 

matrix in this paper, only one inner image vector forms the inside covariance matrix, so there is 

only one non-zero EST negative eigenvector. 

An example of showing EST property is illustrated in figure 3, where two target chips (truck and 

tank) are considered as the outer and inner image vectors, respectively. In this example, the 

truck image is considered as the outer image vector (however, in a real scenario, it would be a 

clutter chip), while the tank image is considered as the inner image vector. The original images 

of the truck and tank are shown in figures 3a-b. The positive eigenvector Vest+ and the negative 

eigenvector Vest– are shown in figures 3c-d. From figure 3, we can observe that Vest+ contains 

truck and the shadow of the tank. The negative eigenvector Vest– contains mainly tank but the 

shadow of the truck is still visible. 

Figure 3. Examples of EST and PCA. 
7 

In PCA analysis, the eigenvectors of the inside covariance matrix Cin and outside covariance 

matrix Cout are calculated separately. We call the eigenvectors obtained from Cout PCA outside 

eigenvectors Vpca,out. Similarly, the eigenvectors obtained from Cin are called PCA inside 

eigenvectors Vpca,in. PCA inside eigenvector Vpca,in is, of course, not expected to be orthogonal to 

PCA outside eigenvectors Vpca,out. Since there is only one inner image vector to form the inside 

covariance matrix, there is only one non-zero PCA inside eigenvector. Therefore, Vpca,in is the 

normalized version of Xin, that is, 

Vpca,in = βXin

where β is the square root of the energy (norm) of the inner image vector, which is the same as 

the corresponding eigen value. 

3.2 Hypotheses 

A visual motivation for the formulation in equation 1 is illustrated in figure 4. For simplicity, 

assume that the outer window consists of only two vectors, Xout,1 and Xout,2. The inner image 

vector is Xin, which is aligned in the same direction as the PCA inside eigenvector Vpca,in. After 

the EST procedure, the positive and negative eigenvectors Vest+1, Vest+2, and Vest– are aligned 

approximately with the inner and outer image vectors, respectively. In addition, all positive and 

negative eigenvectors are orthogonal to each other. 

Figure 4. Visualization of hypotheses. 

Consider the EST properties as demonstrated by the results in figure 3. The negative EST 

eigenvector predominately exhibits features that originally appear in the inner window (the tank) 

and the positive EST eigenvector contains the features that appear mainly in the outer window 

(the truck). Thus, the negative and positive EST eigenvectors approximately align with the inner 

and outer windows, respectively. We use these properties for our hypothesis testing problem in 

the following fashion. For hypothesis H0, where clutter is within the inner box, the re-alignment 

of EST eigenvectors with respect to the PCA eigenvectors causes the negative EST eigenvector 

Vest– to be distant from the PCA inside eigenvector Vpca,in. For hypothesis H1, where the target is 
8 

within the inner box, the EST negative eigenvector Vest– is more closely aligned to the PCA 

inside eigenvectors Vpca,in. 

When there is clutter within the inner box that is somewhat similar to its surroundings, the 

difference between the inner and outer image vectors is not large. The EST procedure mixes the 

inner and outer image vectors and generates the positive and negative eigenvectors. The EST 

positive eigenvectors present mainly the subspace corresponding to the outer image vectors, 

which are also similar to the inner clutter image vector. Because the EST negative eigenvector 

has to be orthogonal with the EST positive eigenvectors, the EST negative eigenvector is more 

distant from the inner image vector. This causes the residual energy between the EST negative 

eigenvector Vest– and the PCA inside eigenvector Vpca,in to be large. On the other hand, when 

there is a target in the inner box, there is a larger difference between the inner and outer image 

vectors. The EST negative eigenvector is generated to be orthogonal with the EST positive 

eigenvectors, which are already very different from the inner image vector. The EST negative 

eigenvector is then closer to the inner image vector that is proportional to the PCA inside 

eigenvector. Therefore, the residual energy between the EST negative eigenvector Vest– and the 

PCA inside eigenvector Vpca,in is small. 

A common method of measuring the difference between two sets of vectors is to project 

elements of one set upon the other. One way we could accomplish this is by projecting the inner 

image vector into the subspaces that are generated by the EST positive or PCA outside 

eigenvectors. The relative error energy is calculated as follows, 

1

ˆ

2

2

≈ − = +

in

in in

X

X X

E

where Xin+ ˆ is the reconstruction of Xin using the subspace generated by the EST positive 

eigenvectors. However, this relative error energy is close to 1 for both target and clutter inside 

the inner box. This is because that the inner image vector Xin is almost orthogonal to the positive 

eigenvectors. Therefore, this is not an effective measurement. We use the following hypothesis 

test instead: 

a. Under the null hypothesis (clutter inside), mixing of outer and inner image vectors, Cout and 

Cin, via EST results in a measurable change in “alignment” of EST positive and negative 

eigenvectors with respect to PCA inside and outside eigenvectors. That is, 

= − − > δ 2H0 Vpca,in Vest

b. Under the alternative hypothesis (target inside), mixing of outer and inner image vectors, 

Cout and Cin, via EST does not alter the “alignment” of EST positive and negative 

eigenvectors with respect to PCA inside and outside eigenvectors. That is, 
9 

= − − < δ 2H1 Vpca,in Vest

Comparing the PCA outside eigenvector and EST positive eigenvector could also be used for 

exploring the differences between inner and outer image vectors. However, this analysis is in a 

multi-dimensional space. In the following discussion of detection criterion, we use only the PCA 

inside eigenvector and EST negative eigenvector. 

3.3 Target Detection Procedure 

The overall procedure of the target detection algorithm is outlined in the following (as shown in 

figure 5.) 

Figure 5. Procedure of the target detection algorithm. 

1. The input FLIR image is passed through the gradient operators as mentioned above, to 

obtain two gradient images lx and ly, which represent the gradient images in the x and y

directions, respectively. 

2. For each pixel of the gradient images, the inner window and outer window are formed to 

obtain the inner and outer image vectors, Xin and Xout. 

3. Two transformations P1(Xin) and P2(Xin, Xout) are calculated via PCA and EST, where 

P1(Xin) represents the PCA positive eigenvector Vpca,in and P2(Xin, Xout) represents the EST 

negative eigenvector Vest–. That is, 

P1 Xin Vpca,in ( ) =

P Xin Xout = Vest− ( , ) 2

4. The residual energy Evi is obtained. That is, 

, , 0

2

Evi = Vpca,in −Vest− if Vpca in∗Vest− >

, , 0

2

Evi = Vpca,in +Vest− if Vpca in∗Vest− <
10 

 If Vpca,in and Vest– are the same sign, the residual energy is the mean square difference of 

these two vectors. If Vpca,in and Vest– are different signs, the residual energy is the mean 

square sum. 

5. For each pixel, two values of residual energy, Evi,x and Evi,y are calculated, corresponding to 

the x and y directions. The minimum of these two residual energies is retained. 

 ) min( , Emin = Evi,x Evi, y

6. If this minimum Emin is above a threshold δ, the pixel is declared clutter. If this minimum 

is below δ, the pixel is declared as target. 

4. Results and Discussion 

4.1 Image Database 

We used two image databases yuma9207_roi and huli9204_roi from Comanche database to test 

the proposed target detection algorithm. Yuma9207_roi is the more difficult database because 

the images were taken in the summer in the Arizona desert and the background contains many 

high temperature spots. Huli9204_roi is an easier database. The images were taken in the spring 

in central California and the background is cool compared with most of the targets in the 

database. Figure 6 shows image examples from the two databases. 
11 

 (a) (b)

Figure 6. Examples of FLIR images: (a) an image from the database yuma9207_roi, where the 

background contains many high temperature spots and (b) an image from the database 

huli9204_roi, where the background is cool compared with the targets in the image. 

The proposed algorithm is based on exploring the structural differences between a target and its 

surroundings. For targets in a close range, the structural features of the targets are prominent 

within the image. For targets in a longer range image, the target appears very small and the 

structural information is not obvious. In order to test the algorithm performance, the algorithm is 

run at the different ranges. We broke the ranges down into three categories, short, medium, and 

long. Table 1 shows the information about these two databases. In the database, some images 

contain targets and some do not. When ROC curves are calculated for these databases, a roughly 

equal number of images, with and without targets, are randomly selected. This makes the 

experiment less biased. 

Table 1. Database. 

Total 

number 

of image 

Number 

of image 

without 

targets 

Number 

of image 

with 

targets 

Range: number of images/ 

number of targets 

yuma9207_roi 2644 839 1805 Short range: 804/804 

Medium range: 430/640 

Long range: 571/1010 

huli9204_roi 1910 686 1224 Short range: 971/974 

Medium range: 253/403 
12 

Table 1 shows the details of the image databases. Images are stored in 10 bits. For the image 

database yuma9207_roi, the total number of images is 2644. Among these, the number of 

images without target is 839 and the number of images with target is 1805. For the images with 

targets, the numbers of images in short, medium, and long ranges are 803, 430, and 571, 

respectively. For these three ranges, numbers of targets are 804, 640, and 1010, respectively, 

since some images have more than one target. For the image database huli9204_roi, the total 

number of images is 1910, where 686 images have no targets and 1224 images have targets. For 

the images with targets, there are 971 images in short range, where 974 targets are found, and 

there are 253 images in medium range, where 403 targets are found. 

4.2 ROC Results 

Ideally, the proposed algorithm should calculate the residual energy at all pixels in the image. 

However, to reduce the computation time, the algorithm only calculates the residual energy for 

points that are separated in distance by a quarter of the size of the inner window. As we show in 

the following, this would not significantly affect the performance of the proposed algorithm. 

In figure 7a, the solid line window corresponds to an inner window that is centered at a target 

coordinates. The bold dashed line window corresponds to one of the inner windows that our 

algorithm selects. Consider the ratio of the overlapping area of these two windows to the area of 

a single window; we call this ratio the overlapping area ratio. When this overlapping area ratio 

is 1, the calculating point is at the center of the target. When this overlapping area ratio is less 

than 1, the calculating point is away from the center of the target. Figures 7b-d show the 

normalized histograms of the overlapping area ratios for the images at short, medium, and long 

ranges, respectively, of yuma9207_roi database. From figures 7b-d, about 90% of the images 

have the overlapping area ratios above 0.70 at all ranges. The huli9207_roi database illustrates 

the similar distribution of the overlapping area ratios. 
13 

(a) (b)

(c) (d)

Figure 7. (a) Overlapping area ratio. The solid line window corresponds to an inner window that is centered at a 

target coordinates. The bold dashed line window corresponds to one of the inner windows that the 

algorithm selects. Overlapping area ratio is defined as the ratio of the overlapping area of these two 

windows to the area of a single window, (b) normalized histogram of the overlapping area ratios for short 

ranges, (c) normalized histogram of the overlapping area ratios for medium ranges, and (d) normalized 

histogram of the overlapping area ratios for long ranges. 

ROC results are shown in figure 8. For the huli9204 database, as is shown in figure 8a, the two 

curves are for the short and medium range targets. The ROC results are very similar. For a 

threshold corresponding to 1 false alarm per frame (per image), the detection rate is about 81%. 

If the false alarm is increased to 20 per frame, the detection rate reaches 96%. 
14 

(a) (b)

Figure 8. ROC results from the proposed implementation: (a) database huli9204_roi and (b) database 

yuma9207_roi. 

For the yuma9207 database, as is shown in figure 8b, the ROC results are plotted for short, 

medium, and long ranges. The performances of these three range bins are quite different. As 

discussed above, for long ranges, the target appears very small in the image. In this case, there is 

not enough structural information available in the target chip. At 1 false alarm per frame, the 

detection rate is about 23%. When 20 false alarms are allowed per frame, the detection rate is 

about 46%. For medium ranges, the performance is improved to a detection rate of 40% for 1 

false alarm per frame and 71% for 20 false alarms per frame. The best performance is achieved 

for short ranges. The detection rate is 58 % at 1 false alarm per frame and 84% at 20 false 

alarms per frame. 

4.3 Discussion 

As we mentioned early, to reduce the computation time, the proposed implementation only 

calculates the residual energy for points that are separated in distance by a quarter of size of the 

inner window. In order to assess the performance of this implementation, we compare this result 

with an ideal situation. In this ideal situation, the algorithm calculates the residual energy for 

points that are separated in distance with the same size as the inner window. The algorithm starts 

by replacing the inner window at the center of the target, which is obtained from the ground truth 

information. Subsequent inner windows are placed so that the inner windows are adjacent, nonoverlapping, and cover as much of the image as is possible consistent with both inner and outer 

windows being completely contained within the image. 

Figure 9 shows the ROC results for the proposed implementation and the ideal situation for the 

database yuma9207_roi. Figure 9a shows the results for 0 to 25 false alarms per frame. Figure 
15 

9b shows the enlarged portion of the false alarm rates between 0 to 5. The solid lines are the 

ROC results for the ideal situation. The dash lines are the ROC results for the proposed 

implementation that is obtained in figure 8b. The upper lines in the figure are ROC results for 

the short ranges, the middle lines are for the medium ranges, and lower lines are for the long 

ranges. The overall results show that similar results are obtained from the proposed 

implementation and the ideal situation. For the long ranges, the detection rate of the proposed 

implementation is 23% versus 25% of the ideal situation at 1 false alarm per frame, and 46% 

versus 48% at 20 false alarms per frame. For the medium ranges, at 1 false alarm rate, the 

detection rates of the proposed implementation and the ideal situation are 40% and 49%, 

respectively; at 20 false alarms per frame, 71% and 72%, respectively. For the short ranges, at 1 

false alarm per frame, the detection rates of the proposed implementation and the ideal situation 

are 58% and 63%, respectively; at 20 false alarms per frame, 84% and 85%, respectively. 

(a) (b)

Figure 9. ROC results of the proposed implementation comparing with the ideal situation for the database 

yuma9207_roi: (a) ROC results of false alarm rates between 0 and 25; (b) ROC results of false 

alarm rates between 0 and 5. 

This result demonstrates that the algorithm performance is not significantly changed when the 

center of the inner window is not exactly located at the center of the target. However, the 

measure of closeness of the center of the inner window and the center of the target is required to 

be within a certain degree. For example, the overlapping area ratio that was defined above needs 

to be over 0.70 as shown here. 

The proposed algorithm is also compared with another target detection algorithm, called spatial 

anomaly detection algorithm (SADA) which was described in [19], for the database yuma9207. 

The ROC curves are shown in figures 10a-b. Figure 10a shows the results for 0 to 25 false 

alarms per frame. Figure 10b shows the enlarged portion of the false alarm rates between 0 and 
16 

5. The solid lines represent the results for the SADA algorithm and the dash lines are for 

proposed algorithm. The upper lines in the figure are ROC results for the short ranges, the 

middle lines are for the medium ranges, and the lower lines are for the long ranges. The overall 

results show that the proposed algorithm has a higher detection performance than the SADA at 

low false alarm rates. At higher false alarm rates, the SADA gives slightly higher detection 

rates. See figure 10b; at short ranges, the detection rates of the proposed algorithm and the 

SADA are similar. For medium ranges, at 1 false alarm per frame, the detection rate of the 

proposed algorithm is 40% from the 29% obtained by the SADA. For long ranges, at 1 false 

alarm per frame, the detection rate of the proposed algorithm is increased to 23% from 18% 

obtained by the SADA. 

(a) (b)

Figure 10. ROC results of the proposed algorithm comparing with the SADA for the database yuma9207_roi: 

(a) ROC results of false alarm rates between 0 and 25 and (b) ROC results of false alarm rates 

between 0 and 5. 

The results of testing on each point for two images (both containing one target) showed that the 

hit points are concentrated in the target area, suggesting that the performance of the algorithm 

does not deteriorate significantly by not testing on every image point. Detection results obtained 

by testing each point on one image are shown in figures 11a-c. Figure 11a shows the gradient 

image obtained by highpass filtering the FLIR image in the x direction. The values of the 

residual energy Emin are displayed as the image gray-level values in figure 11b. The image is 

displayed in an inverse mode such that the lower Emin values appear brighter. All brighter areas 

appear in the target area and the transition areas between vegetation and ground, where strong 

structural information exists. In figure 11c, the points with Emin values that are less than the 

threshold (0.3 in this case) are superimposed into the gradient image. All hit points are within 

the target area. 
17 

 (a) (b) (c) 

Figure 11. Detection results: (a) gradient image obtained by the highpass filtering in the x 

direction, (b) residual energy Emin values, and (c) the points with Emin < 0.3 are 

superimposed into the gradient image in (a) as markers (+). 

As discussed earlier, the range between a target and the sensor changes the target structural 

information that appears in FLIR images. The proposed algorithm is designed to detect targets 

that appear at short and medium ranges within the FLIR imagery. For targets at long ranges 

(where targets appear as “blob” if it is hot), a different algorithm (such as an energy detector), 

that does not rely on a detailed structural information, is needed. 

5. Conclusions 

An adaptive target detection algorithm based on exploiting the structural information within a 

target and its background is presented. The algorithm calculates the inner and outer image 

gradient vectors representing the target and its background, respectively. The EST and PCA 

transforms are locally generated from the inner and outer image gradient vectors. The 

differences between the EST and PCA eigenvectors are used to distinguish the target from its 

background. Results of testing the proposed algorithm on two large FLIR databases were 

presented. 
18 

6. References 

1. Lloyd, J. M. Thermal imaging systems, New York: Plenum, 1982. 

2. Wolfe, W. L.; Zissis, G. J. Eds. The Infrared Handbook, Erim International, 1985. 

3. Ratches, J. A.; Vollmerhausen, R. H.; Driggers, R. G. Target acquisition performance 

modeling of infrared imaging systems: past, present, and future. IEEE Sensors Journal 

2001, 1 (1), 31–40. 

4. Bhanu, B.; Jones, T. L. Image Understanding research for automatic target recognition. 

IEEE AES Systems Magazine October 15–22 1993. 

5. Bhanu, B. Automatic target recognition: state of art survey. IEEE Trans. on Aerospace and 

Electronic Systems 1986, AES-22 (4), 364–379. 

6. Sang, N.; Zhang, T. Segmentation of FLIR images by Hopfield neural network with edge 

constraint. Pattern Recognition 2001, 34, 811–821. 

7. Sun, S-G.; Park, H. Segmentation of forward-looking infrared image using fuzzy 

thresholding and edge detection. Optical Engineering 2001, 40 (11), 2638–2645. 

8. Ernisse, B.; Rogers, S. K.; DeSimio, M. P.; Raines, R. A. Complete automatic target 

cuer/recognition system for tactical forward-looking infrared images. Optical Engineering 

1997, 36 (9), 2593–2603. 

9. Park, S-I.; Murenzi, R.; Smith, M.J.T. Multidimensional wavelets for target detection and 

recognition. Wavelet Applications III (8-12 April 1996 : Orlando, FL, USA, Proceedings of 

the SPIE 2762, 177–187, 1996. 

10. Zhou Y. T.; Crawshaw, R. Contrast, size and orientation invariant target detection in 

infrared imagery. Automatic Object Recognition, Proceedings of SPIE 1471, 1991. 

11. Weber, D. M.; Casasent, D. P. Quadratic Gabor filters for object detection. IEEE Trans. on 

Image Processing 2001, 10 (2), 218–230. 

12. Roth, M. W. Survey of neural network technology for automatic target recognition. IEEE 

Trans. on Neural Networks 1990, 1 (1), 28–43. 

13. Rizvi, S. A.; Nasrabadi, N. M. Neural network algorithms for automatic target recognition 

using forward-looking infra-red imagery: A Survey. PINSA 2001, 67A (2), 243–276. 

14. Dwan, C. M.; Der, S. Z. A neural network based target detection system for FLIR imagery. 

Applications of Artificial Neural Networks in Image Processing III (26-27 Jan. 1998: San 

Jose, CA, USA). Proceedings of the SPIE 3307, 14–21, 1998. 
19 

15. Ramanan, S.; Peterson, R. S.; Clarkson T. G.; Taylor, J. G. pRAM nets for detection of 

small targets in sequences of infra-red images. Neural Networks 1995, 8 (7/8), 1227–1237. 

16. Xue, D-H.; Zhu, Y-T.; Zhu, G-X; Xiong, Y. Recognition of low-contrast FLIR tank object 

on multiscale fractal character vector. Automatic Object Recognition VI: (9–10 April 1996, 

Orlendo, FL, USA), Proceedings of SPIE 2756, 38–45, 1996. 

17. Shekarforoush, H.; Chellappa, R. A multi-fractal formalism for stabilization, object 

detection and tracking in FLIR sequences. Proceedings 2000 International Conference on 

Image Processing, Piscataway, NJ, USA, Vol 3, 78–81, 2000. 

18. Gregois, D. J.; Yu, S.K.W.; Tritchew, S.; Sevigny, L. Wavelet transformation-based 

filtering for the enhancement of dim targets in FLIR images. Wavelet Applications (5-8 

April 1994: Orlando, FL, USA), Proceedings of SPIE 2242, 573–583, 1994. 

19. Der, S.; Nasrabadi, N.; Chan, A.; Kwon, H. Spatial Anomaly detection algorithm for FLIR 

imagery. Advances in Imaging Sciences and Engineering: Algorithms, Systems, and 

Applications, edited by Bahram Javidi, Spring-Verlag, 2002. 

20. Torrieri, D. The eigenspace separation transform for neural-network classifiers. Neural 

Networks 1999, 12, 419–427. 

21. Krauskopf, J. Effect of retinal image stabilization on the appearance of heterochromatic 

targets. Journal of the Optical Society of America 1963, 53, 741–744. 

22. Hubel, D. H. Eye, Brain, and Vision, Scientific American Library, W. H. Freeman, New 

York, 1988. 
20 

Distribution List

ADMNSTR 

DEFNS TECHL INFO CTR 

ATTN DTIC-OCP (ELECTRONIC COPY) 

8725 JOHN J KINGMAN RD STE 0944 

FT BELVOIR VA 22060-6218 

DARPA 

ATTN IXO S WELBY 

3701 N FAIRFAX DR 

ARLINGTON VA 22203-1714 

OFC OF THE SECY OF DEFNS 

ATTN ODDRE (R&AT) 

THE PENTAGON 

WASHINGTON DC 20301-3080 

US ARMY TRADOC 

BATTLE LAB INTEGRATION & TECHL 

DIRCTRT 

ATTN ATCD-B 

10 WHISTLER LANE 

FT MONROE VA 23651-5850 

US MILITARY ACDMY 

MATHEMATICAL SCI CTR OF 

EXCELLENCE 

ATTN MADN-MATH LTC T RUGENSTEIN 

THAYER HALL RM 226C 

WEST POINT NY 10996-1786 

SMC/GPA 

2420 VELA WAY STE 1866 

EL SEGUNDO CA 90245-4659 

US ARMY ARDEC 

ATTN AMSTA-AR-TD 

BLDG 1 

PICATINNY ARSENAL NJ 07806-5000 

COMMANDING GENERAL 

US ARMY AVN & MIS CMND 

ATTN AMSAM-RD W C MCCORKLE 

REDSTONE ARSENAL AL 35898-5000 

US ARMY INFO SYS ENGRG CMND 

ATTN AMSEL-IE-TD F JENIA 

FT HUACHUCA AZ 85613-5300 

US ARMY NATICK RDEC 

ACTING TECHL DIR 

ATTN SBCN-TP P BRANDLER 

KANSAS STREET BLDG 78 

NATICK MA 01760-5056 

US ARMY SIMULATION TRAIN & 

INSTRMNTN CMND 

ATTN AMSTI-CG M MACEDONIA 

12350 RESEARCH PARKWAY 

ORLANDO FL 32826-3726 

US GOVERNMENT PRINT OFF 

DEPOSITORY RECEIVING SECTION 

ATTN MAIL STOP IDAD J TATE 

732 NORTH CAPITOL ST., NW 

WASHINGTON DC 20402 

HICKS & ASSOC INC 

ATTN G SINGLEY III 

1710 GOODRICH DR STE 1300 

MCLEAN VA 22102 

US ARMY RSRCH LAB 

ATTN AMSRD-ARL-CI-OK-TP TECHL LIB 

 T LANDFRIED 

ABERDEEN PROVING GROUND MD 

 21005-5066 

DIRECTOR 

US ARMY RSRCH LAB 

ATTN AMSRD-ARL-RO-D JCI CHANG 

ATTN AMSRD-ARL-RO-EN W D BACH 

PO BOX 12211 

RESEARCH TRIANGLE PARK NC 27709 

US ARMY RSRCH LAB 

ATTN AMSRD-ARL-D J M MILLER 

ATTN AMSRD-ARL-CI-OK-T TECHL PUB 

 (2 COPIES) 

ATTN AMSRD-ARL-CI-OK-TL TECHL LIB 

 (2 COPIES) 

ATTN AMSRD-ARL-SE-SE S S YOUNG 

 (7 COPIES) 

ATTN IMNE-ALC-IMS MAIL & RECORDS 

 MGMT 

ADELPHI MD 20783-1197