COMPARING 30 METER IMAGERY FROM LANDSAT 5 AND 7 FOR CROP AREA

ESTIMATION

Michael E. Craig, Senior Remote Sensing Analyst

U.S. Department of Agriculture

National Agricultural Statistics Service

3251 Old Lee Hwy, Room 305

Fairfax VA 22031

E-mail: mcraig@nass.usda.gov

ABSTRACT

This paper compares digital, multi-spectral imagery from the Landsat 5 Thematic Mapper with that of the Landsat 7

Enhanced Thematic Mapper when used to discriminate crop types for area estimation. Comparisons are done for

different types of crop areas in several states, using images that are only one day apart. The overlaps between

adjacent paths in several major crops states are used to define the analysis areas. A simple non-parametric paired

sample sign test is used to determine statistical significance of differences. Standardized techniques as used in the

Agency's Cropland Data Layer Project are used for image processing, including a modified supervised pattern

recognition /clustering approach for cover type signatures and maximum likelihood for categorization. Ground truth

data consist of 211 one and two square mile areas selected in a stratified random sample and visited in June of the

corresponding crop year. Sampling rates for the ground data range from one in 30 to one in 166 depending on the

state and land use stratum.

INTRODUCTION

The National Agricultural Statistics Service (NASS) of the U.S. Department of Agriculture has used digital

multi-spectral satellite imagery since the 1970's to aid in the acreage estimation of large area crops in major

producing states. Early research and later operational NASS remote sensing programs used imagery from the

Landsat multi-spectral scanner instruments until the late 1980's; research on the Landsat Thematic Mapper started in

1986 and it was adopted for operational acreage estimation use in 1991. The current NASS Remote Sensing

Acreage Estimation Program (Craig 2001b) encompasses Landsat-based major crops estimates at the state and

county levels plus digital Cropland Data Layer GIS products for six major crop producing states. Numerous

documents about the history of this Program are included in the References section.

Landsat Digital Imagery 

Two Landsat satellites are currently operational; Landsat 5 with the Thematic Mapper (TM) sensor, and Landsat

7 with a somewhat upgraded Enhanced Thematic Mapper (ETM) sensor. Imagery from both sensors was at 30 meter

resolution, and delivered in the UTM coordinate system. Landsat data from both sensors is obtained through a

cooperative agreement with the USDA Foreign Agriculture Service (Bethel and Doorn 1998). The polar orbits of

these two satellites are controlled to give eight day coverage to any area from one of the sensors. The eight day

repeat coverage is very important to any vegetation related remote sensing analysis, and is particularly important for

real time, within season crop acreage and or yield estimation. Crop types are distinguishable from each other for

only a short period during the crop season, and thus have a very limited number of chances to be covered by 

cloud-free satellite imagery.

There is a side by side overlap of approximately 30 percent of a scene between adjacent paths of the satellites;

this overlap can be covered by scenes (one from each satellite) that are one day apart. This research compares the

two types of digital imagery using only the areas in the overlap of scenes separated by one day. These overlap areas

define ‘analysis districts’ which will be processed the same way for each Landsat satellite sensor; highlights of the

image processing and estimation methodology will be discussed in a later section.

Ground Truth Data

Ground truth for the Remote Sensing Acreage Estimation Program comes from a pre-existing Agency

nationwide survey program, called the June Agricultural Survey (JAS). One part of the JAS utilizes an area frame to

select areas of land, called segments, for field visits. The area frame for a given state is a land use stratification of
the state based on percent cultivation of crops. The sample is randomly drawn by strata, with an emphasis toward

minimizing the variances of major crops and livestock estimates. For the 2001 crop season, sample sizes ranged

from 260 in Indiana to 452 in Iowa for our six Program states. Segments are visited in early June, with data

collected directly from personal contacts with farmers. The size of an individual segment is approximately one

square mile in high percent cultivation strata, larger in less cultivated but open area strata, and small in urban areas. 

Field boundaries are drawn on black and white aerial photo enlargements of the segment area, and later entered into

digital form and geo-registered to a Landsat scene in the NASS field office. It should be noted here that crops

estimates at the state (but not county) level can be produced using only the ground survey data without the

application of remote sensing imagery. Table 1 below gives an idea of the normal sampling rates for the states

involved in this analysis, plus the number of samples actually used. Two states have two analysis districts (AD’s)

included, the same center scene with an overlap area on each side.

Table 1. Area frame description and sampled segment* information for overlap areas.

State Year Strata

Strata

Percent

Cultivated

State

Number

Segments

State

Selected

Segments

State

Expansion

Factor

Seg’s in

First

AD

Seg’s in

Second

AD

Arkansas 1999 11 76-100 11673 260 45 80 n/a

“ 21 25-75 2718 24 113 5 n/a

Mississippi 1999 11 76-100 3875 40 39 7 n/a

“ 20 15-50 16003 78 82 8 n/a

North Dakota 1999 11 76-100 29271 231 127 23 19

“ 12 51-75 7469 90 83 10 13

“ 20 15-50 7990 85 94 15 20

2000 11 76-100 29271 231 127 15 n/a

Iowa 2000 13 76-100 43415 378 115 38 48

“ 20 25-75 10963 66 166 10 n/a

* Note: Average segment sizes for strata 12 and 20 in North Dakota are 2 square miles each, all other

state/strata combinations shown in this analysis average 1 square mile each.

Analysis Districts

Seven analysis districts (AD’s), based on path overlap regions and one day date differences, were selected from

the 1999 and 2000 crop year estimation analyses. These AD’s offer a selection of most major crops and crop

producing regions, with the exception of the winter wheat great plains states. Four states (and eight crops) were

selected: North Dakota (barley, canola, sunflower, all wheat, and all small grains), Arkansas (rice, cotton, soybean),

Mississippi (cotton and soybean), and Iowa (corn and soybean). Note that ‘all small grains’ includes all wheat,

barley, oats, and rye for the purposes of this analysis. Table 2 shows the states, imagery dates, imagery path/row

locations and number of segments used in the comparisons.
Table 2. Definitions and specifications of analysis districts

AD State Project Crops

Number

Segments

TM Path/Row(s)

Overpass Date

ETM Path/Row(s)

Overpass Date

1 Arkansas AR99 Rice, Cotton,

Soybean

85 P:23 R:35,36,37

August 13, 1999

P:24 R:35,36,37

August 12, 1999

2 Mississippi MS99 Cotton, Soybean 15 P:23 R:36

July 28, 1999

P:22 R:36

July 29, 1999

3 North Dakota ND99 All Wheat, All

Small Grains

52 P:34 R:26,27,28

July 25, 1999

P:33 R:26,27,28

July 26, 1999

4 North Dakota ND99 Sunflower, All

Wheat, All

Small Grains

48 P:32 R:26,27,28

July 27, 1999

P:33 R:26,27,28

July 26, 1999

5 North Dakota ND00 All Wheat,

Canola, Barley

15 P:32 R:26

August 14, 2000

P:31 R:26

August 15, 2000

6 Iowa IA00 Corn, Soybean 48 P:27 R:30,31,32

August 11, 2000

P:26 R:30,31,32

August 12, 2000

7 Iowa IA00 Corn, Soybean 48 P:27 R:30,31,32

August 11, 2000

P:28 R:30,31,32

August 10, 2000

HIGHLIGHTS OF IMAGE PROCESSING AND ESTIMATION METHODOLOGY

The current NASS system for processing satellite remote sensing data and the statistical methodology used for

estimation of crop acreage was discussed in detail in the ASPRS 2001 proceedings (Craig 2001a). This section

presents the highlights of these procedures for review.

Software and Hardware

NASS uses the PEDITOR software system (Ozga 1995,2000) to process Landsat digital imagery on high-end

desktop computers using the Windows NT operating system. There is usually at least one dual processor Win NT

computer at each site for large volume batch processing. PEDITOR is an in-house created and maintained software

system of over 100 program modules, some of which are expert systems designed to automate parts of the process by

running other modules in sequence. There is one main difference between this software and commercial software:

the regression analysis procedure and its concept of a segment as a clustered sample (as opposed to just as individual

fields for training) from a specific stratum.

Imagery Import and Review of Sample Segments

Location and review of sampled areas in the current years imagery starts with the reformatting (import) of

selected imagery into PEDITOR format. NASS uses all seven Landsat TM and/or ETM bands for training and

classification. Where available, multi-temporal (two date) imagery is created using an automated block correlation

approach. Segment and internal field boundaries are overlaid on the new imagery, and are reviewed on a field by

field basis to identify any ‘bad’ fields where the reported data does not match the information seen in the digital

satellite imagery. This process includes such things as cropland already harvested, poor crop stands, cloud and haze

affected areas, poorly drawn boundaries, and areas where the planted cover observed during enumeration does not

match the farmer reported cover. Fields with problems that cannot be corrected are identified and later marked as

‘bad for training’. Cloud affected segments are removed from consideration with respect to the specified scene.
Training the Classifier Based on Known Sample Information

 The NASS automated procedure uses a modified supervised approach in the creation of cover type signatures.

In this approach interior (non-boundary) pixels from known fields, not labeled as bad for training, are sorted

according to cover type into separate files. Using a principle components analysis, additional outlier pixels are

deleted (‘clipped’) from the files (Winings, 1990). Each updated file is then clustered, using a modified ISODATA

algorithm which allows cluster splitting and merging (Bellow, 1991a). A similar process is used to cluster ‘extra’

cover types, such as clouds, urban, and deep water. The resultant signature statistics for clusters from all cover types

are then combined into one statistics file for input to the maximum likelihood classifier. All available known pixels

from the sample segments are categorized using this classifier for regression analysis.

Evaluation of the Maximum Likelihood Classifier

Several statistics are calculated for evaluation of the classifier. Known pixels are tabulated comparing ground

truth labels with the category assigned during classification. Percent correct, commission errors, and kappa

coefficients are calculated automatically from the tabulation. These statistics are calculated (and reported) based on

the ‘not bad for training’ set of known pixels from the segments. A field by field analysis is also performed to check

which signatures cause the most classification errors. Another module calculates regression coefficients by

crop/cover type for farmer reported versus categorized data at the segment level. Segments identified (by the

software) as regression outliers can be reviewed and deleted from the analysis for final estimation purposes. Table 3

below shows percent correct and kappa comparisons over all cover types plus the number of signatures used in the

final classification.

Table 3. Overall Percent Correct and Kappa Statistics

State AD Sensor

Percent

Correct Kappa

# Signatures

with ‘extra’

#Signatures

from Seg’s

AR99 1 ETM+ 75.46 67.94 103 83

AR99 1 TM 79.97 73.42 107 73

MS99 2 ETM+ 92.49 90.77 87 36

MS99 2 TM 93.43 91.97 56 23

ND99 3 ETM+ 62.83 51.13 137 106

ND99 3 TM 58.91 46.53 118 102

ND99 4 ETM+ 61.97 48.58 95 95

ND99 4 TM 63.80 52.13 93 93

ND00 5 ETM+ 82.01 77.85 27 23

ND00 5 TM 79.67 75.22 57 25

IA00 6 ETM+ 88.81 84.29 44 44

IA00 6 TM 88.67 83.99 39 39

IA00 7 ETM+ 96.26 93.49 51 51

IA00 7 TM 96.29 93.54 40 40

Estimation

Full scene classifications provide the last piece of the estimation process. The classification output is
summarized based on land use strata and boundaries from the area sampling frame. A regression estimation

approach is then applied, using classified pixels as the auxiliary variable and farmer reported acres as the dependent

variable in a set of area frame stratum based simple linear regressions. The county and state crop acreage estimates

produced by this system greatly reduce the sampling variation (‘error’) found in the estimates produced by the

ground data alone; and have the added benefits of producing county estimates with measurable precision. Table 4

below displays the various statistics available to compare at the crop type level. Regression line slopes were also

calculated, but are not shown in the table. In order to compare data with the least possible analyst interference, no

segment outlier analysis was performed on any of the AD’s. No Kappa statistics were available for the grouped

categories (all wheat and all small grains) due to software limitations.

Table 4. Percent correct and combined strata regression R-squared statistics by crop type

State AD Crop Type Sensor

Percent

Correct

Commission

Error Kappa*

Combined

R-squared

AR99 1 Cotton ETM+ 90.25 29.78 89.53 .926

AR99 1 Cotton TM 90.72 13.15 90.17 .909

AR99 1 Rice ETM+ 96.17 15.74 94.34 .865

AR99 1 Rice TM 90.92 9.19 87.32 .858

AR99 1 Soybean ETM+ 71.07 7.15 57.99 .673

AR99 1 Soybean TM 78.50 8.61 67.05 .624

MS99 2 Cotton ETM+ 97.09 6.29 96.28 .770

MS99 2 Cotton TM 96.59 10.98 95.61 .609

MS99 2 Soybean ETM+ 95.28 3.95 93.26 .424

MS99 2 Soybean TM 90.60 2.71 87.00 .299

ND99 3 All Wheat ETM+ 69.58 23.78 n/a .621

ND99 3 All Wheat TM 63.64 33.19 n/a .527

ND99 3 Small Grains ETM+ 73.88 22.37 n/a .753

ND99 3 Small Grains TM 73.23 28.21 n/a .655

ND99 3 Sunflowers ETM+ 93.46 3.04 93.37 .985

ND99 3 Sunflowers TM 87.51 8.54 87.32 .973

ND99 4 All Wheat ETM+ 38.10 25.66 n/a .609

ND99 4 All Wheat TM 36.02 27.73 n/a .546

ND99 4 Small Grains ETM+ 51.81 24.78 n/a .768

ND99 4 Small Grains TM 44.35 25.91 n/a .643

ND99 4 Sunflowers ETM+ 46.09 32.48 44.77 .644

ND99 4 Sunflowers TM 66.07 41.79 64.70 .707

ND00 5 All Wheat ETM+ 83.46 6.60 n/a .022
State AD Crop Type Sensor

Percent

Correct

Commission

Error Kappa*

Combined

R-squared

ND00 5 All Wheat TM 76.24 4.24 n/a .515

ND00 5 Barley ETM+ 90.62 18.78 87.13 .098

ND00 5 Barley TM 93.00 23.03 90.04 .487

ND00 5 Canola ETM+ 91.59 6.23 90.85 .874

ND00 5 Canola TM 92.68 23.56 91.86 .730

IA00 6 Corn ETM+ 97.96 3.69 96.55 .887

IA00 6 Corn TM 96.63 5.04 94.25 .832

IA00 6 Soybean ETM+ 98.99 1.29 98.52 .924

IA00 6 Soybean TM 98.96 2.16 98.49 .903

IA00 7 Corn ETM+ 96.82 2.76 93.62 .705

IA00 7 Corn TM 97.27 2.06 94.57 .618

IA00 7 Soybean ETM+ 98.04 0.90 96.68 .846

IA00 7 Soybean TM 98.08 2.22 96.71 .882

* No Kappa statistics were available for the grouped categories: all wheat and all small grains.

COMPARISON OF STATISTICS FOR LANDSAT 5 TM AND LANDSAT 7 ETM

Statistical comparisons in this analysis consist of simple, non-parametric sign tests (Siegel 1956)since the data

are all based on paired samples (ETM+ versus TM for the same areas), and there are not enough pairs to assume

normal distributions for paired ‘t’ tests. Under the sign test, the null (H0) hypothesis is that the median difference

between ETM+ and TM is zero, with a two sided alternative (HA) hypothesis that it is non-zero. Critical values for

the sign test are based on the binomial probability function.

Hypothesis Tests

This set of hypotheses was tested for percent correct (by crop type and overall), commission error (by crop

type), kappa statistic (by crop type and overall), and combined strata r-squared. For the percent correct and kappa

statistics, both by crop type and overall, there was no reason to reject the H0 of the median difference between TM

and ETM+ being zero. However, the same tests, using commission error and combined r-squared do show a

significant difference at the 5% level. Table 5 below shows the sign test values associated with these tests. 

Although not shown in this paper, comparisons were also done using only large analysis districts, as defined by

those analysis districts with 45 or more segments. The sign test results for the large AD only analyses mirrored the

‘all AD’ results shown in Table 5. Also considered were number of signatures created from the segment data, both

at the AD level, and by crop. The differences using number of signatures were not significant for either AD level or

by major crop type.
Table 5. Sign Test Results

Variable

Summary Level

(Within AD)

(+)

ETM+ > TM

(-)

TM > ETM+

Significant at

P=X%

Percent Correct Overall Cover Types 3 4 n/s

Kappa Overall Cover Types 3 4 n/s

Percent Correct Major Crop Type 11 7 n/s

Commission Error Major Crop Type 5 13 10%

Kappa * Major Crop Type * 6 7 n/s

Combined Regr. Slope Major Crop Type 6.5 11.5 n/s

Combined R-Squared Major Crop Type 14 4 5%

Slope Single Regression Major Crop & Stratum 18 20 n/s

R-Squared Single Regr. Major Crop & Stratum 29 9 5%

* No Kappa statistics were available for the grouped categories: all wheat and all small grains.

CONCLUSION

The importance of the Landsat 5 TM sensor is that it allows an eight day repeat coverage for imagery

acquisition. This fact is particularly important for crop area estimation, because most intensively cultivated areas

have cloud cover problems. You might think of this as the relationship between intensive cultivation and rain fed

cropping practices. Half of the NASS statewide analysis would be missing if not for the presence of Landsat 5 TM.

Percent correct related statistics are important for ‘map’ and visual image creation, such as for the Cropland

Data Layer distributed by NASS. These statistics include percent correct, commission error, and kappa coefficients. 

Although the commission error statistics are different, this measure is usually considered only a complement to the

percent correct, and not as a stand-alone statistic. Considering only the visual side of remote sensing based outputs,

the two sensors would not be significantly different. 

However, the r-squared and slope statistics are more important when the aim is to produce crop acreage

indications (estimates) as these are directly related to the standard error of the estimates. The slope statistics are not

significantly different between the two sensors, but the r-squared regression statistics are significantly different. The

r-squared significant differences are perhaps the result of the better commission errors, but for whatever reason, the

differences are small but significant. Under the acreage estimation scenario, the ETM+ data seems to be slightly

better, and given a choice of similar dates between the two sensors, the ETM+ should be used. An interesting point

to consider is how close in performance the data from Landsat 5 TM, launched in March 1984, is to the data from

the Landsat 7 ETM+ sensor, launched in April, 1999.
REFERENCES

Allen, J., and Hanuschak, G. (1988). The Remote Sensing Applications Program of the National Agricultural

Statistics Service: 1980-1987. SRB Staff Report Number SRB-88-08, USDA, National Agricultural Statistics

Service, Washington, DC-USA.

Allen, J. (1990). Remote Sensor Comparison for Crop Area Estimation Using Multitemporal Data. SRB Staff

Report Number SRB-90-03, USDA, National Agricultural Statistics Service, Washington, DC-USA.

Bellow, M., and Ozga, M. (1991a). Evaluation of Clustering Techniques for Crop Area Estimation Using Remotely

Sensed Data. In: 1991 Proceedings of the Section on Survey Research Methods, American Statistical

Association, Atlanta, GA-USA, pp. 466-471.

Bellow, M. (1991b). Comparison of Sensors for Corn and Soybean Planted Area Estimation. SRB Research Report

Number SRB-91-02, USDA, National Agricultural Statistics Service, Washington, DC-USA.

Bellow, M. and Graham, M. (1992). Improved Crop Area Estimation in the Mississippi Delta Region Using Landsat

TM Data. In: Proceedings of ASPRS/ACSM/RT'92 Convention, Washington, D.C., pp. 423-432.

Bellow, M. (1994a). Application of Satellite Data to Crop Area Estimation at the County Level. NASS Research

Report Number STB-94-02, USDA, National Agricultural Statistics Service, Washington, DC-USA.

Bellow, M. (1994b). Large Domain Satellite Based Estimators of Crop Planted Area. In: 1994 Proceedings of the

Section on Survey Research Methods, August, 1994, American Statistical Association, Toronto, Canada,

pp.194-199.

Benz, S. (2000). Area Frame Design Information - 2000 Edition. Internal document: USDA, National Agricultural

Statistics Service, Area Frame Section, Washington, DC-USA.

Bethel, G. and Doorn, B. (1998). USDA Remote Sensing Technical and Systems Support for Operational

Worldwide Agruiculture Analysis. In: Proceedings of the 1st International Conference: Geospatial Information

in Agriculture and Forestry, ERIM International Inc., Ann Arbor, MI-USA, pp. 44-51.

Cook, P., Mueller, R., and Doraiswamy, P. (1996). Southeastern North Dakota Landsat TM Crop Mapping Project.

In: 1996 ASPRS/ACSM Annual Convention Technical Papers, Vol. 1, Baltimore, MD-USA, pp. 600-614.

Craig, M. (2001a). A Resource Sharing Approach to Crop Identification and Estimation. Available on the CD: 

Proceedings of the ASPRS 2001 Conference, ASPRS, Bethesda, MD-USA. 

Craig, M. (2001b). The NASS Cropland Data Layer Program. Presented at the Third International Conference on

Geospatial Information in Agriculture and Forestry, Denver, Colorado, 5-7 November 2001.

Day, C. (2001). A Compilation of PEDITOR Estimation Formulas. RDD Research Paper RDD-01-01, USDA, National

Agricultural Statistics Service, Washington, DC-USA.

Graham, M. (1993). State Level Crop Area Estimation Using Satellite Data in a Regression Estimator. In: Survey

Methods for Businesses, Farms, and Institutions, ICES Part I, NASS Research Report No. SRB-93-10, USDA,

Washington, DC-USA, pp. 7-11.

Hanuschak, G., Allen, R., and Wigton, W. (1982). Integration of Landsat Data Into The Crop Estimation Program of

USDA’s Statistical Reporting Service. In: Proceedings of the 1982 Machine Processing of Remotely Sensed Data

Symposium, Eighth International Symposium, LARS, Purdue University, West Lafayette, IN-USA, pp. 45-56.

Hanuschak, G., Hale, R., Craig, M., Mueller, R., and Hart, G. (2001). The New Economics of Remote Sensing for

Agricultural Statistics in the United States. To be included in the: Proceedings of the CAESAR Conference, June

2001, Italian Statistics Agency (ISTAT), Rome, Italy

Harris, J., Winings, S., and Saffell, M. (1989). Remote Sensor Comparison for Crop Area Estimation. In: Proceedings

of the IGARSS '89 Symposium, Vancouver, Canada, Vol. 3, pp. 1860-1863.

Harris, J. (1990). The Comparison of Emulated Multispectral Scanner Data Sets. SRB Staff Report Number SRB-90-10,

USDA, National Agricultural Statistics Service, Washington, DC-USA.

Mueller, R. (2000). Categorized Mosaicked Imagery from the National Agricultural Statistics Service Crop Acreage

Estimation Program. Available on the CD: Proceedings of the ASPRS 2000 Conference, ASPRS, Bethesda, MDUSA.

Ozga, M., and Craig, M. (1995). PEDITOR - Statistical Image Analysis for Agriculture. Presentation at the Washington

Statistical Society (WSS) Seminar, April, 1995, USDA, National Agricultural Statistics Service, Washington, DCUSA.
Ozga, M. (2000). Batch Processing of Remote Sensing Jobs on the PC. Available on the CD: Proceedings of the

ASPRS 2000 Conference, ASPRS, Bethesda, MD-USA.

Siegel, S. (1956). Non-Parametric Statistics. McGraw-Hill, New York, pp. 68-75.

Stup, C., and Allen, J. (1991). The Construction of a Dry Bean Area Sampling Frame in Michigan. In: Michigan Dry

Bean Digest, Vol. 15, No. 4, pp. 8-12.

Vogel, F., and Bange, G. (1999). Understanding USDA Crop Forecasts. In: NASS/WAOB Miscellaneous Publication

No. 1554, U.S. Department of Agriculture, Washington DC-USA.

Winings, S., Ozga, M., and Stakenborg, J. (1990). The Effects of the Applications of Smoothing and Orthogonal

Transforms to SPOT and TM Data on Regression Based Crop Acreage Estimates. In: Proceedings of the IGARSS

'90 Symposium, College Park, MD-USA, Vol. 1, pp. 647-650.