United States

Department of

Agriculture

Forest Service Research Note

Southern Forest

Experiment Station

SO-380

September 1995

Correction of Line Interleaving Displacement in

Frame Captured Aerial Video Imagery

B. Cooke and A. Saucier

SUMMARY

Scientists with the USDA Forest Service are currently

assessing the usefulness of aerial video imagery for various purposes including midcycle inventory updates. The

potential of video image data for these purposes may be

compromised by scan line interleaving displacement

problems. Interleaving displacement problems cause

features in video raster datasets to have jagged edges

due to pixel offset on adjacent scan lines. Researchers

at the Southern Forest Experiment Station, Forest Inventory and Analysis Unit (SO-FIA), have developed a

computer program, “Pixshift,” written in C, that shifts alternate raster lines (even or odd) to the right or left by a

user-specified integer value.The program allows a maximum of four channels to be shifted simultaneously. Improvements in band-to-band registration and rectification

to a map base have been observed, and classification

improvements are expected.

INTRODUCTION

Video imaging is a developing remote sensing tool that

is receiving increased application in resource management (King and Vlcek 1990). Research is being conducted at SO-FIA to investigate uses of aerial video for

landcover classifications, forest canopy percent measurements, sub-pixel prediction models for lower resolution

datasets, and generation of digital elevation models

(DEM’s) from video stereo pairs. Operational use of airborne video at SOiFIA has been documented for forest

damage assessment resulting from ice storms and hurricanes (Jacobs and Eggen-McIntosh, in press) and for

satellite data classification verification (Eggen-McIntosh

and Jacobs 1993). Uses of aerial video in forestry are

becoming commonplace due to low acquisition costs,

the ease of system setup and operation, and digital capabilities to incorporate the imagery into computer databases and geographic information systems (GIS) (Evans

and Beltz 1992). Although much research has been done

that validates multispectral video as a tool for providing

forest inventory data, more research is needed to compare technologies and publish their limitations (Walker

1993).

In each of the previously mentioned applications, traditional digital image processing functions are hindered

to some extent by line interleaving displacement problems. Some video frame grabbing software allows the

user to capture only one of the two fields comprised by

full video frames. This process eliminates the line interleaving problem, but the cost is a 50-percent reduction

in data. A typical video scene captured in digital form

(fig. 1) has been magnified in order to emphasize the

interleaving problem. Interleaving is manifested by a more

jagged edge appearance than is normally expected with

raster data. In this example, alternate lines in the magnified area are noticeably “out of sync” by a factor of one

pixel. Notable problems include difficulty in locating accurate control points for scene registration to a map coordinate system and problems associated with defining

training fields for classification purposes, particularly in

areas where image features of interest may contain few

pixels.

The purpose of this study was to develop the “Pixshift’

program and to demonstrate qualitative visual improvement of images and subsequent improvement in bandto-band registration of images after shifting.

METHODS

Captured digital video images used in this study were

generated from a four-camera system. Separate images

Southern Forest Experiment Station/T-l0210 U.S. Postal Services Bldg., 701 Loyola Avenue, New Orleans, La. 70113

Forest Service, U.S. Department of Agriculture.

Servlng Alabama, Arkansas, Louisiana, Mississippi, E. Oklahoma, Tennessee, E. Texas, Puerto Rico, U.S. Virgin Islands.

1
Figure 1 .-Single frame of uncorrected video data captured in digital form; boxes at the bottom depict

various zoom factors applied to the area of interest.

representing reflectance in the blue, green, red, and nearinfrared portions of the electromagnetic spectrum were

obtained by lens filtering. Each camera recorded images

on a charge coupled device (CCD) sensor capturing

350,000 pixels per spectral image in a full-frame format.

Spatial resolution was approximately 1 meter per pixel.

The single band images were not registered to each

other, and band-to-band registration was performed by

the researcher. A primary concern in the registration process was accurate visual depiction of ground features

used for control points common to each set of two bands.

Correction of the line displacement problem was considered essential to qualitative visual improvement of images and accurate visual depiction of ground features.

The Pixshift program was designed to take a raster

image in ASCII format with one to four channels of spectral information and shift either even or odd rows to the

left or right by a specified integer value. The resultant

image is enhanced from a visual and an analytical perspective. The program should be applied prior to any

rectification process.

The Pixshift program requires conversion of video images to ASCII file format as a preparatory step.The ASCII

file may contain a maximum of four data channels

(bands), and the current maximum number of pixels per

line is 5,000. There is no limit on the number of lines

other than practical limits of available disk space.

The Pixshift program is currently designed to accept

an ASCII file created in ERDAS; however, any generic

‘Use of company, product, or trade names is solely for information

and does not constitute official endorsement by the USDA Forest

Service.

Bill Cooke is a research forester, and Allen Saucier is a computer specialist, U.S. Department of Agriculture, Forest Service, Southern Forest

Experiment Station, Forest Inventory and Analysis, Starkville, MS 39759.

2
ASCII file of the same design will work as well. The

Pixshift program is user interactive.The user must specify

the number of channels, the number of pixels (as an integer) and direction (left or right) of the shift, and whether

even or odd lines are to be shifted. The output file is an

ASCII file of the same size as the input file. A batch routine has been created that automatically calls the ERDAS

programs and passes the data to the appropriate program. The original image file is preserved as a binary

data file with a “.lan-ori” file name extension. The output

file has the same name as the original file.

Subsequent to qualitative visual improvement of images, image registration improvement prior to and after

shifting was tested. Comparison of band-to-band registration was performed on a single scene.The green band

was chosen as the band to which the other bands were

registered. It is impossible to remove all subjectivity from

manually choosing control points. Each ground feature’s

reflectance characteristics are unique to the recorded

spectral band. Analyst subjectivity in choosing points

common to different spectral bands was decreased by

using the same zoom factor (16) for each set of two

bands. The same set of control points was used for each

registration test. Differences in file coordinates result from

initial capture differences, analyst subjectivity in selection, and pixel shifting. A rigorous test of the improvement in band-to-band registration was not the focus of

this paper; however, current research will allow comparison of average root mean square (RMS) errors for two

adjacent flight lines.

RESULTS

Comparison of the unshifted image (fig. 1) with the

shifted image (fig. 2) is indicative of the qualitative visual

improvement obtained by using the “Pixshift” program.

This image was created by moving every even-numbered

line to the left by a factor of one pixel. Quantitative improvement in band-to-band registration for the multispecFigure P.-Single frame of corrected video data captured in digital form; boxes at the bottom depict

various zoom factors applied to the area of interest.

3
tral video imagery is evidenced by improvement of RMS

errors for sets of control points common to the unregistered green and near infrared bands (table 1).

DISCUSSION

The video camera incorporates a trick to fool the eye

and reduce transmission band width: the video frame is

divided into two fields, half taken at time 1 and half taken

at l/60 second later at time 2 (Lake 1994). Line interleaving displacement in full video frames may be introduced by aircraft vibration, motion, or some combination of

the two variables that may occur between field capture.

The Pixshift program is an important preparatory tool

for “cleaning up” video imagery that exhibits line interleaving displacement in advance of traditional image processing techniques. Visual improvement of imagery is

obvious, and future implications are promising. Classification efforts are expected to improve subsequent to the

Table 1 .-Comparison of root mean square (RMS) errors for unshifted and shifbd

video imagev using ground control points (GCP) common to both images

Green Near infrared RMS error

GCP Total RMS

No. X Y X Y X Y error

________________________________ Unshifted videoimagery ____-_-----_--__-__________________

1 100.0 72.0 102.0 77.0 0.09 -0.36 0.37

2 326.5 69.5 326.5 96.5 -0.17 0.30 0.35

3 594.7 56.0 597.9 64.0 0.40 -0.23 0.46

4 50.0 235.9 50.9 240.9 0.05 -0.21 0.21

5 294.5 292.0 295.4 299.4 -0.06 0.69 0.69

6 657.6 249.0 659.0 257.9 -0.66 0.17 0.70

7 115.0 425.0 115.1 431.0 0.19 0.25 0.32

S 436.2 469.0 436.0 476.0 -0.42 -0.67 0.79

9 721.4 461.6 722.6 471.2 0.60 0.05 0.61

__________________________________ Shifted videoimagefyt ____________________________________

1 97.0 76.0 101.0 77.0 0.30 -0.06 0.31

2 323.5 92.5 327.5 96.6 -0.56 0.25 0.62

3 590.5 56.9 597.5 63.9 0.51 0.14 0.53

4 46.0 239.0 49.0 240.0 0.17 -0.12 0.21

5 294.0 294.9 295.0 299.0 -0.16 -0.12 0.19

6 657.0 249.0 660.0 257.0 -0.59 -0.33 0.66

7 116.2 428.3 114.3 430.9 -0.02 -0.01 0.02

6 436.0 469.0 435.0 476.0 -0.10 0.33 0.34

9 722.0 461.0 723.0 471.0 0.45 -0.05 0.45

'x,y RMS = (0.46, 0.47),Total RMS = 0.66

tx,y RMS = (0.46, 0.42),Total RMS = 0.52

improved spatial integrity of features. Choice and delineation of training fields for supervised classification of

multispectral video data should be more easily accomplished, and increased precision of frame-to-frame registration has been demonstrated. Improvement in

automatic generation of OEM’s from video stereo pairs

is being investigated at this time, and improvement is

expected due to corrections made by this program. Measurement work, including stereo modeling for height determination and area measurements, is expected to yield

increased accuracy.

Work is continuing on automatic calculation of shift

distance and direction parameters. Determination of variable shift factors across an image, based on autocorrelation results, is also being investigated.The Pixshift

program, when used in conjunction with traditional image enhancement techniques, should enable Forest

Service researchers and others to expand the usefulness of video imagery for a variety of forestry applications. The Pixshift program is available from SO-FIA in

Starkville, MS.
LITERATURE CITED

Eggen-McIntosh, Susan; Jacobs, Dennis M. 1993. Acquisition of airborne videography of Mexico for verification of FAO forest resources assessment Landsat

interpretations. In: Proceedings of the 14th biennial

workshop on color photography and videography for

resource monitoring; 1993 May 25-29; Logan, UT.

Bethesda, MD: American Society for Photogrammetry and Remote Sensing: 135-l 44.

Evans, David L.; Beltz, Roy C. 1992. Aerial video and

associated technologies for forest assessments. In:

Greer, Jerry D., ed. Remote sensing and natural resource management: Proceedings of the 4th Forest

Service remote sensing appplications conference;

1992 April 6-l 1; Orlando, FL. Bethesda, MD: American Society for Photogrammetry and Remote Sensing: 301304.

Jacobs, Dennis M.; Eggen-McIntosh, Susan. [In press].

Airborne videography and GPS for assessment of forest damage in southern Louisiana from Hurricane

Andrew. In: Proceedings of the IUFRO conference on

inventory and management in the context of catastrophic events; 1993 June 21-24; University Park, PA.

International Union of Forestry Research Organizations.

King, Doug; Vlcek, Jerry. 1990. Development of a multispectral video system and its application in forestry.

Canadian Journal of Remote Sensing. 16(l): 15-22.

Lake, Don. 1994. Pictures in time: Camera issues when

you exploit motion video clips. Advanced Imaging.

January: 47-51.

Walker, Kim-Marie. 1993. Airborne digital multispectral

imaging. Earth Observation Magazine. April: 66-69.

5