1

Research Note RMRS-RN-59. 2013

Rocky Mountain Research Station

Research Note

United States Department of Agriculture

Forest Service

Rocky Mountain Research Station

Research Note RMRS-RN-59

October 2013

Citation:

Goeking, Sara A.; Patterson, Paul L. 2013. Stratifying to reduce bias 

caused by high nonresponse rates: A case study from New Mexico’s 

forest inventory. Res. Note RMRS-RN-59. Fort Collins, CO: 

U.S. Department of Agriculture, Forest Service, Rocky Mountain 

Research Station. 22 p.

Authors:

Sara A. Goeking, Inventory and Monitoring Program, Rocky 

Mountain Research Station, Ogden, Utah. 

Email: sgoeking@fs.fed.us; phone 801-625-5193.

Paul L. Patterson, Inventory and Monitoring Program, Rocky 

Mountain Research Station, Fort Collins, Colorado. 

Email: plpatterson@fs.fed.us; phone 970-498-2618.

Stratifying to Reduce Bias Caused by 

High Nonresponse Rates: A Case Study 

From New Mexico’s Forest Inventory

Sara A. Goeking and Paul L. Patterson

Abstract

The USDA Forest Service’s Forest Inventory and Analysis (FIA) Program applies specific sampling and analysis procedures to estimate a variety of forest attributes. FIA’s 

Interior West region uses post-stratification, where strata consist of forest/nonforest 

polygons based on MODIS imagery, and assumes that nonresponse plots are distributed at random across each stratum. The sample of New Mexico’s forests during 

2005-2013 did not meet this assumption due to a large number of private landowners who denied permission to access plots on their properties. We developed a 

modified stratification scheme to minimize bias in our estimates of forest attributes. 

The purpose of this paper is twofold: first, to document the customized stratification 

methods used for the New Mexico forest inventory of 2005-2013, and second, to 

provide sufficient instructions for application of this method to other states or inventories where high nonresponse rates occur.

Keywords: Forest inventory, FIA, stratification, nonresponse, bias, Interior West
2

Research Note RMRS-RN-59. 2013

Introduction

The USDA Forest Service’s Forest Inventory and Analysis (FIA) program is designed to produce timely, objective estimates of forest attributes on an ongoing basis 

(Gillespie 1999). A comprehensive set of statistical sampling and estimation procedures have been developed to meet this objective (Bechtold and Patterson 2005). 

The FIA sample design consists of an array of field plots distributed across the entire 

United States, where each plot represents roughly 6,000 acres, and the sample represents a variety of land cover types and ownership groups. Field measurements provide 

the basis for broad-scale estimation of forest attributes such as forest area, numbers of 

trees, volume, growth, and mortality. To reduce the variance of these estimates, FIA 

applies a stratification scheme on a state-by-state basis, where strata typically consist 

of forest, nonforest, and noncensus water, as determined by satellite imagery (Scott 

and others 2005).

FIA’s stratification and estimation methods assume that nonresponse occurs at random within each stratum. Nonresponse is defined as the portion of the sample that 

cannot be measured, and it occurs because some plots are deemed too hazardous to 

survey safely or, more typically, permission to access plot locations cannot be obtained. We recognize that nonresponse may not occur randomly within strata for 

two reasons. First, past and current rates of nonresponse are higher on private lands, 

where landowners have the right to deny access to their property, than on non-private 

lands where FIA and the appropriate land managers have typically entered into memoranda of understanding that facilitate field sampling. Second, plots that are not 

designated for field sampling due to their obvious nonforest status have a different 

probability of nonresponse than plots that are designated for field sampling. Prior 

to field sampling, photo-interpreters examine each plot location using aerial photographs, and plots that are unquestionably nonforest are designated as non-field plots 

and are not included in the field sample. The purpose of this prefield discrimination 

between “field” and “non-field” plots is to decrease inventory costs by minimizing 

unnecessary field sampling at nonforest plot locations. However, non-field plots have 

zero probability of nonresponse, while field plots have a positive probability of nonresponse because they may be inaccessible due to hazardous conditions or lack of 

permission to access the plot. Therefore, the assumption of random nonresponse is 

violated in any stratum that has a combination of non-field plots, sampled field plots, 

and nonresponse field plots.

Recent work by Patterson and others (2012) demonstrates that forest estimates may 

be biased when the assumption of random nonresponse is not met. For example, in 

a state with a high rate of nonresponse, the estimated forest land area will be lower 

than the true forest land area. In order to reduce this bias, FIA must modify its approach to stratification such that nonresponse rates are homogeneous within each 

stratum (Patterson and others 2012). Because most nonresponse plots in the Interior 

West occur on private lands due to landowners’ right to deny access to their property, a modified stratification protocol should distinguish private from non-private 

strata, in addition to the forest/nonforest distinction that is already used for variance 

reduction.
3

Research Note RMRS-RN-59. 2013

We applied a revised stratification approach to the New Mexico forest inventory, and 

the resulting estimates from the national FIA database (FIADB; see Woudenberg and 

others 2010) are available to the public at http://www.fia.fs.fed.us/tools-data/default.

asp. The two purposes of this paper are (1) to provide documentation of the customized stratification methods used to estimate the publicly available population-level 

forest attributes in New Mexico, and (2) to provide a roadmap to other analysts faced 

with constructing such estimates in the face of high nonresponse. We also illustrate 

discrepancies in statewide estimates of forest land area based on the standard FIA 

stratification versus the modified approach used here.

Methods

Overview of FIA Sample Design

FIA’s annual inventory system is designed to provide spatially and temporally balanced estimates of forest attributes via a sample plot grid that is subdivided into 

spatially interpenetrating panels, where each panel is scheduled for a specific inventory year. In the western United States, the entire sample grid is measured over a 

10-year period and therefore each panel consists of 10% of the grid. Sample plots 

are pre-assigned to specific panels in the 10-year cycle, such that a plot scheduled for 

inventory year 2013 will be remeasured in 2023, 2033, etc.

FIA produces estimates at the scale of individual states, which can then be aggregated 

into regional estimates, as well as at smaller scales within each state. Within-state 

population estimates are constructed at two scales: survey units that are comprised of 

groups of counties, and smaller estimation units that typically represent individual 

counties. The eight states in FIA’s Interior West region contain between one and five 

survey units per state. The number of estimation units is equal to the number of 

counties per state, except in cases where minimum sample size requirements necessitate lumping counties into larger estimation units.

Study Area

We developed and tested our modified stratification procedure using forest inventory data from the State of New Mexico. Four FIA survey units comprise all of New 

Mexico and are further subdivided into 33 estimation units corresponding to individual counties (Figure 1). After FIA began New Mexico’s annual inventory in 2008, 

funding became available through the American Recovery and Reinvestment Act of 

2009 to accelerate the inventory in that state. Under a compressed data collection 

schedule, plots that were scheduled for the 2005-2007 and 2009-2013 inventory 

years were measured during 2010-2012. Thus the inventory years for this analysis 

were 2005-2013, or 90% of the total sample grid, and actual measurement years 

were 2008-2012. The dataset used for our stratification is identical to that available 

from the public FIADB (Woudenberg and others 2010) as well as the forthcoming report on New Mexico’s Forest Resources (Goeking and others, in preparation). 
4

Research Note RMRS-RN-59. 2013

Figure 1—Map showing New Mexico’s four FIA survey units and 33 estimation units, which correspond to individual 

counties. Dots show approximate FIA plot locations from the national Forest Inventory and Analysis Database 

(FIADB).
5

Research Note RMRS-RN-59. 2013

Preliminary analysis revealed high nonresponse rates in the private land category, 

which warranted customization of New Mexico’s stratification procedures in an effort to better satisfy the assumption of random nonresponse. We focused on plots 

that were entirely nonresponse because only 32 plots, or less than 0.3% of all plots, 

contained a combination of response and nonresponse, and because FIA’s estimators 

already adjust for partial nonresponse.

Statistical Principles

Our stratification was based on two statistical premises. First, a minimum sample size 

is required in each population or subpopulation of interest. Sample size equates to 

the number of plots that were sampled by either a prefield photo interpreter or a field 

crew, and therefore have been assigned a definitive status as either forest or nonforest. 

Nonresponse plots are not included in the sample size. The minimum sample size requirements apply to each estimation unit, or county, as well as the strata within each 

county. Although Scott and others (2005) recommended a minimum sample size of 

4 response plots per stratum, recent work by Westfall and others (2011) revealed that 

fewer than 10 plots per stratum may yield a biased variance estimate. We followed 

the latter recommendation and used a minimum sample size of 10 response plots per 

stratum in our stratification scheme.

The second premise is that all plots in a stratum should have an equal likelihood of 

nonresponse. As mentioned above, FIA estimation uses stratification for variance 

reduction, with the assumption that nonresponse occurs at random within each 

stratum. If the assumption of random nonresponse within strata is not met, then 

estimates may be biased (Patterson and others 2012). Therefore the goal of our modified approach to stratification was to produce strata wherein each plot has an equal 

probability of nonresponse. We refer to this characteristic as response homogeneity, following the response homogeneity group (RHG) model of Särndal and others 

(1992). We approached the New Mexico stratification with the guiding principle 

that strata should be defined using predetermined stratification variables that are 

known to affect response probability, in addition to FIA’s traditional stratification 

classes. Within our newly defined strata, we applied a threshold of 5% to distinguish 

high versus low nonresponse rates. Therefore strata with nonresponse rates of 5% or 

greater were examined for subdivision, while strata with less than 5% nonresponse 

were left intact and not split.

Stratification Steps

We developed a stratification guide (Appendix) to translate theoretical principles of 

minimum sample size and response homogeneity into practical application. The guide 

prescribes the order of operations for stratification and specifies the decision rules for 

each step. We tested the guide on the New Mexico data from 2005-2013, with the 

objective of producing a repeatable statewide stratification procedure wherein every 

stratum consists of at least 10 response plots that have similar response probabilities. 

The stratification process consisted of four distinct steps.

1. We selected classification variables to define our stratification such that final 

strata would have greater response homogeneity.
6

Research Note RMRS-RN-59. 2013

2. We combined populations and subpopulations that did not meet the 

minimum sample size requirement, with the constraint that nonresponse rates 

in the strata being combined should be uniformly high (≥5%) or low (<5%) 

whenever possible.

3. We examined each stratum once again, by estimation unit, splitting strata with 

nonresponse rates greater than 5% into non-field/field substrata, provided that 

the resulting substrata had at least 10 response plots.

4. We calculated stratum weights for strata with spatially quantifiable areas, and 

estimated stratum weights for strata that had been subdivided into non-field 

and field plot substrata.

Each of these steps is described in more detail in the following sections.

Step 1: Selecting Stratification Variables—In the Interior West, traditional FIA 

stratification schemes distinguish predicted forest/nonforest classes developed from 

250-m MODIS satellite imagery. We refer to these strata individually as green (predicted forest) and brown (predicted nonforest), and collectively as Phase 1 strata 

(Bechtold and Patterson 2005). We retained the Phase 1 classes and added ownership as a second stratification variable because preliminary analysis showed that 

nonresponse rates were higher on private lands than on any other ownership category 

(Figure 2). Classes for the ownership variable were non-private and private. Using 

geographic information system tools, we computed the spatial intersection of the 

Phase 1 green/brown layer with a statewide ownership layer (USDOI BLM 2012) to 

produce four initial strata: (1) green/non-private, (2) green/private, (3) brown/nonprivate, and (4) brown/private (Figure 3).

Figure 2—Proportion of plots recorded as forest, nonforest, and nonresponse, by owner group (number of 

plots in group). 
7

Research Note RMRS-RN-59. 2013

Figure 3—Maps showing (a) two Phase 1 strata, which were modeled based on MODIS imagery as 

green (predicted forest) and brown (predicted nonforest); (b) two ownership strata, recoded from 

a statewide ownership layer as non-private and private; and (c) the four strata resulting from the 

spatial intersection of Phase 1 strata and ownership strata.
8

Research Note RMRS-RN-59. 2013

Although this four-stratum scheme accounts for different nonresponse rates due to 

Phase 1 and ownership classes, it does not account for the unequal response probabilities between field plots versus non-field plots. Any stratum that has high nonresponse 

as well as large numbers of both non-field and field plots is likely to violate the 

assumption of random nonresponse. Because non-field plots have zero probability 

of nonresponse, splitting a high-nonresponse stratum into non-field/field substrata 

produces one substratum with zero nonresponse (i.e., the non-field substratum) and 

another substratum with higher nonresponse but greater response homogeneity than 

the initial stratum. Thus, given sufficient sample sizes, separating strata with high 

nonresponse rates into non-field and field substrata is necessary to increase response 

homogeneity within each substratum and thereby reduce bias. Unlike the stratification variables of green/brown and non-private/private, the field status variable was 

not considered until Step 3, where we examined only strata with high nonresponse. 

This is because Phase 1 classes and ownership classes are spatially continuous variables, whereas non-field/field status is known only for point locations. This difference 

has statistical implications for both strata weights, as described in Step 4, and variance estimators, which will be addressed in future research.

Step 2: Combining Strata to Achieve Minimum Sample Size—To meet the minimum sample size constraint, we first examined each estimation unit and then each 

stratum within each estimation unit. None of the estimation units in New Mexico 

had fewer than 10 response plots, so combining estimation units was not necessary. 

However, some estimation units would have had fewer than 10 response plots if our 

sample encompassed fewer than nine panels. FIA stratifications in the Interior West 

are done state by state, often using only five panels, so we developed rules for combining estimation units (see Step 2.1 of the Appendix). For example, if a very small 

county contained only nine response plots, combining all of that county’s strata into 

a single stratum would not achieve the minimum sample size. In this case, the county 

must be lumped with the nearest county that falls within the same survey unit, where 

geographic distance is based on the counties’ centroids, to create a new estimation 

unit. The rules regarding the minimum sample size per stratum then apply to the 

strata in the combined estimation unit.

Within each estimation unit, many strata had fewer than 10 response plots and were 

therefore combined with other strata in the same estimation unit. The following decision rules governed the pairing of strata that could be combined (see Step 2.2 of the 

Appendix for details). First, if a stratum had less than 10 response plots and nonresponse was less than 5%, then the stratum was combined with another stratum that 

also had less than 5% nonresponse. The first priority was to combine such a stratum 

with respect to its Phase 1 class (i.e., green/brown); if that was not possible or did not 

meet the minimum sample size requirement, then the second priority was to combine with respect to ownership status (i.e., non-private/private).
9

Research Note RMRS-RN-59. 2013

Examples of small strata, one with low nonresponse and the other with high nonresponse, illustrate the decision rules for combining strata. To demonstrate the situation 

of low nonresponse, let Stratum 1 be the green/non-private stratum. Stratum 1 has 

fewer than 10 response plots and a nonresponse rate of zero. We would first attempt 

to combine the green/non-private stratum with the green/private stratum in the same 

estimation unit, provided that the green/private stratum also has a nonresponse rate 

less than 5%. If these conditions are not met in their entirety, or if the combined 

stratum has fewer than 10 response plots, then we consider combining the green/

non-private stratum with the brown/non-private stratum, where the same constraints 

apply. If the green/non-private stratum and the brown/non-private stratum do not 

have similar response levels, or if combining them does not create a stratum with 

at least 10 response plots, then all strata in the estimation unit are combined into a 

single stratum. In New Mexico, this situation occurred in only one estimation unit, 

where there was one nonresponse plot and each stratum had fewer than 10 response 

plots.

Similarly, if a stratum has high nonresponse and fewer than 10 response plots, then 

the stratum should be combined with another stratum with high nonresponse. For 

example, Stratum 2 is the green/private stratum, and if its nonresponse rate is high, 

then we would first attempt to combine it with the green/non-private stratum to obtain a combined green/non-private/private stratum. However, to combine these strata 

the green/non-private stratum must have more than 5% nonresponse and combining 

green/private and green/non-private strata must yield at least 10 response plots. If 

these criteria are not met, we would instead attempt to combine the green/private 

stratum with the brown/private stratum to create a green/brown/private stratum, 

provided that the brown/private stratum has high nonresponse and the combined 

green/brown/private stratum has at least 10 response plots.

Step 3: Splitting Strata to Achieve Response Homogeneity—After achieving 

minimum sample sizes based on the rules for combining strata, we examined the 

nonresponse rate of each stratum within each estimation unit. To account for different nonresponse probabilities among non-field versus field plots, we attempted to 

split strata with high nonresponse rates into non-field/field substrata. For any stratum 

with greater than 5% nonresponse, if there were at least 10 response plots in both 

non-field and field classes, then we split the stratum into non-field/field substrata. 

However, if either the non-field or field substratum was too small, then we retained 

the original stratum with high nonresponse because splitting the stratum would have 

resulted in substrata with less bias but also much less precision. Within any stratum 

in the green Phase 1 class, this concession was acceptable because nearly all plots in 

the green class (more than 95%) were designated as field plots.
10

Research Note RMRS-RN-59. 2013

Step 4: Calculating or Estimating Strata Weights—Strata weights are used to expand plot data to an area of interest such as an estimation unit. When strata are 

spatially continuous across the area of interest, the strata weights can be calculated 

as the proportion of the total area in each stratum. When strata are defined by point 

variables, however, their weights must be estimated as the product of the total area and 

the proportion of plots in that stratum’s class. Therefore, we calculated strata weights 

for all strata defined by only Phase 1 and ownership status, because green/brown and 

non-private/private classes exist as spatially explicit, continuous statewide datasets 

(Figure 3). In contrast, the strata weights for non-field/field substrata were estimated 

as the initial stratum weight, prior to subdivision, multiplied by the proportion of 

non-field and field plots within that stratum. Therefore the sum of the non-field/field 

substrata weights is equal to the weight of the initial stratum from which they were 

split. Within each estimation unit, the sum of the weights for all strata, including 

the estimated weights for non-field/field substrata but not the weights of their initial 

strata, is equal to one.

Results

The New Mexico sample for inventory years 2005-2013 included 11,792 plots 

statewide, where 3,444 (29%) met FIA’s definition of forest land, 7,340 (62%) were 

nonforest land, and 1,008 (9%) were nonsampled (i.e., nonresponse). Nonresponse 

plots were spatially dispersed throughout the state and among estimation units 

(Figure 1). Using the stratification guide in the Appendix, we did not encounter any 

ambiguous scenarios during the stratification process using the New Mexico 2005-

2013 dataset. Thus we met the objective of developing repeatable decision rules for 

stratification that incorporated the principles of minimum sample size and response 

homogeneity.

Statewide Nonresponse Rates Among Strata

Table 1 presents the statewide sample size, proportion of field plots, and nonresponse 

rate, by each of the four initial strata as well as aggregated among green/brown and 

non-private/private classes. To assess the ability of a modified stratification scheme to 

meet the response homogeneity objective at a statewide scale, we compared the nonresponse rates within the traditional strata to those within our newly created strata. 

The green and brown groups showed similar nonresponse rates (10.6% and 8.0%, 

respectively), while the non-private and private groups showed a difference of more 

than 10 percentage points (2.1% and 13.9%). Strata 2 and 4 represent the private 

portions of Phase 1 green and brown classes, and both had nonresponse rates greater 

than 10%. Both Stratum 1 and Stratum 3 had nonresponse rates less than 5%. This 

signifies that including ownership class as a stratification variable produced strata 

with more internal response homogeneity than using green/brown classes alone.
11

Research Note RMRS-RN-59. 2013

These results confirm that the factors affecting nonresponse are ownership status and 

non-field/field status; private plots clearly have a higher probability of nonresponse 

than non-private plots, and all non-field plots are designated as nonforest and therefore have zero nonresponse. Pairwise comparisons of the four initial strata illustrate 

the relative importance of these factors in each stratum. For example, Strata 1 and 

2 respectively comprise the non-private and private segments of the Phase 1 green 

category, and both strata had 98% of plots included in the field survey but had very 

different nonresponse rates (2.9% and 22.0%). These two metrics collectively imply 

that nonresponse disparities between the two strata are due to ownership differences 

rather than their proportions of non-field/field plots. A comparison of Strata 3 and 

4 (see Table 1) corroborates the influence of private ownership on nonresponse rates. 

In contrast, a similar comparison of Stratum 1 and Stratum 3 indicates that the small 

difference in their low nonresponse rates is due to the fact that (a) nonresponse is low 

on non-private lands, and (b) nonresponse was lower in the stratum with a smaller 

proportion of field plots (i.e., potential nonresponse plots). Similarly, while Stratum 

2 and Stratum 4 both have high nonresponse rates, nonresponse is higher in the green 

stratum where the proportion of field plots is much larger.

Description of Final Strata Among Estimation Units

The final stratification scheme maintained the original 33 estimation units, which 

had total sample sizes ranging from 11 to 611 response plots. After the four initial 

strata were combined due to sample size constraints, and then split when necessary 

and possible for response homogeneity, there were 17 final strata (Table 2). Because 

Steps 2 and 3 were conducted independently for individual estimation units, not every stratum exists in every estimation unit. The number of final strata per estimation 

unit varied from one to five with strata sample sizes ranging from 10 to 423 response 

plots. The most common strata among estimation units were the original strata 1, 2, 

and 3, along with strata 41 and 42, which represent the non-field/field substrata of 

the original stratum 4. Eight of the final strata represent non-field/field substrata that 

were split from a previous stratum with high nonresponse; their final strata numbers 

are denoted with two digits (e.g., strata 31 and 32).

Table 1—Statewide sample size, percentage of plots designated for field sampling, 

and percentage of plots that are nonresponse, by stratum, for New Mexico’s 

Forest Inventory, 2005-2013. 

Ownership classes

Nonprivate Private

Phase 1 

Classes

Green Stratum 1

1,528

98.0%

2.9%

Stratum 2

1,041

97.8%

22.0%

Green total

2,569

97.9%

10.6%

Brown Stratum 3

3,811

48.6%

1.8%

Stratum 4

5,412

37.5%

12.3%

Brown total

9,223

42.0%

8.0%

Nonprivate total

5,339

62.7%

2.1%

Private total

6,453

47.2%

13.9%

Stratum

Sample Size

Field Plot %

Nonresponse Plot %
12

Research Note RMRS-RN-59. 2013

Nonresponse rates in individual strata, by estimation unit, ranged from zero for several strata to a maximum of 59% for one private/field stratum with 91 total plots, 

which was formed after the initial green/private and brown/private strata (Strata 2 

and 4, respectively) were combined during Step 2 for sample size reasons and then 

split during Step 3 into non-field/field categories to address high nonresponse. Of 

the 33 estimation units, 17 had high nonresponse in either Stratum 1 or Stratum 2, 

yet none of these could be sub-stratified into non-field/field substrata because they 

did not have 10 or more non-field plots. As mentioned above, this situation requires 

prioritizing practical sample size constraints over the statistical purity of response 

homogeneity. This is a reasonable concession for Strata 1 and 2 because having nearly 

100% field plots equates to response homogeneity within the strata, at least with 

respect to the non-field/field distinction. To investigate whether this concession was 

also reasonable for the brown strata, we examined estimation units where Strata 3 and 

4 had high nonresponse and could not be split into non-field/field substrata. There 

were only two of these situations, and the vast majority of plots in both cases were 

field plots. Therefore we concluded that when the small sample size of a potential 

substratum precluded splitting a high nonresponse stratum into non-field/field substrata, preserving the initial stratum with high nonresponse was acceptable.

Effects of Modified Stratification on Statewide Estimates of 

Forest Area

To gage the effects of the new stratification on estimates of forest attributes, we contrasted estimated forest land area by ownership group and by forest type, based on 

the standard Phase 1 versus modified stratification procedures. The two stratification 

schemes produced estimates of total forest land area that differed by more than one 

million acres, where the smaller estimate resulted from the standard Phase 1 stratification (Figure 4a). This result is consistent with the findings of Patterson and others 

Table 2—List of final strata in the New Mexico 2005-2013 inventory. Estimation units contain 

between one and five strata; frequency indicates the number of estimation units that contain 

each stratum.

 Constituent

Final stratum Description initial strata Substrata Frequency

1 Green/Non-private 1 n/a 17

2 Green/Private 2 n/a 15

3 Brown/Non-private 3 n/a 22

4 Brown/Private 4 n/a 7

5 Green 1, 2 n/a 2

6 Brown 3, 4 n/a 1

7 Non-private 1, 3 n/a 7

8 Private 2, 4 n/a 1

9 Green/Brown 1, 2, 3, 4 n/a 1

31 Brown/Non-private/Non-field	 3	 non-field plots	 1

32 Brown/Non-private/Field	 3	 field plots	 1

41 Brown/Private/Non-field	 4	 non-field plots	 16

42 Brown/Private/Field	 4	 field plots	 16

71 Non-private/Non-field	 1, 3	 non-field plots	 1

72 Non-private/Field	 1, 3	 field plots	 1

81 Private/Non-field	 2, 4	 non-field plots	 7

82 Private/Field	 2, 4	 field plots	 7
13

Research Note RMRS-RN-59. 2013

(2012) regarding the bias, or underestimation of forest attributes, that can result 

when nonresponse is high. The private/tribal ownership group had the highest nonresponse rate of all ownership groups (Figure 2) and also exhibited the 

largest difference in the forest land area estimates produced by the two stratification methods (Figure 4a). Relative to the modified stratification, the standard 

stratification underestimated forest area on private/tribal lands by nearly two 

million acres, or 22% of the standard stratification’s estimate, and overestimated forest area on non-private lands by an aggregate of roughly 830,000 acres. 

Figure 4—Graph showing the forest land area, in acres, estimated using standard versus 

modified stratification schemes, by (a) ownership group, and (b) forest type.
14

Research Note RMRS-RN-59. 2013

In general, the proportional differences between the two stratification schemes 

for lands categorized as National Forests, other Federal lands, and state/local 

government lands were inversely related to their nonresponse rates. This may be because plots in ownership groups with low nonresponse (e.g., National Forests) carried 

more weight in the standard stratification than in the modified scheme and therefore 

had more forest land area attributed to them when high nonresponse occurred on 

private lands. By including ownership status as a stratification variable, the modified 

stratification reduced the expansion factor of each non-private plot.

The two stratifications produced different estimates of forest land area not only by 

ownership group but also by forest type (Figure 4b). Forest land area was underestimated by the standard stratification, relative to the modified scheme, within the 

following forest types: the three types representing various juniper and pinyon/juniper woodlands, deciduous oak woodlands, mesquite woodlands, and nonstocked 

stands. These forest types occur most frequently on either private lands or Federal 

lands other than National Forests (Table 3). The forest types that were overestimated 

were Douglas-fir, ponderosa pine, white fir, and Engelmann spruce, all of which occur most frequently on National Forest lands. However, the differences in forest types 

that were overestimated, as well as in the underestimated deciduous oak woodland 

and nonstocked stands, were very small.

The magnitude of the nonresponse bias shown by the standard stratification highlights the potential ambiguity of FIA’s standard statewide summary tables. Every 

FIA state report contains several prescribed tables that report on particular variables and categories within the state of interest. The first of these tables reports the 

Table 3—Number of plots in each forest type, by ownership group.

 National Other State/local Private/

Forest type Forest Federal government tribal Total

Rocky Mountain juniper 21 7 11 77 116

Juniper woodland 68 75 40 175 358

Pinyon / juniper woodland 498 203 127 616 1,444

Douglas-fir	 104	 0	 5	 38 147

Ponderosa pine 268 14 12 111 405

White fir	 49	 0	 2	 8 59

Engelmann spruce 30 0 0 8 38

Engelmann spruce / subalpine fir	 22	 0	 0	 10 32

Blue spruce 6 0 0 1 7

Southwestern white pine 9 0 0 1 10

Foxtail pine / bristlecone pine 1 0 0 2 3

Limber pine 2 0 0 3 5

Cottonwood 1 0 2 8 11

Aspen 49 0 1 14 64

Other hardwoods 0 0 1 0 1

Deciduous oak woodland 63 0 4 69 136

Evergreen oak woodland 62 4 1 11 78

Mesquite woodland 2 204 123 136 465

Intermountain maple woodland 1 0 0 0 1

Other exotic hardwoods 0 0 0 2 2

Nonstocked 33 109 27 54 223

Nonforest 305 2,086 1,134 5,467 8,992

Total 1,594 2,702 1,490 6,811 12,597
15

Research Note RMRS-RN-59. 2013

“percentage of area by land status” for specific inventory years within the state, along 

with the total land area of the state. Land status categories include accessible forest 

land; nonforest and other land; and nonsampled land (i.e., nonresponse). However, 

the percentages reported in the table actually refer to the percentage of sample plot 

area rather than the percentage of statewide land area in each class. Presenting them 

alongside the total state land area may be misleading to FIA data users, as this implies 

that the statewide forest land area, for example, is the product of the total state land 

area and the ostensible percentage of forest land area. For New Mexico’s 2005-2013 

forest inventory, this table lists percentages of 27.4, 63.9, and 8.7 for forest, nonforest, and nonsampled land areas, respectively, and a total state land area of 77,817,000 

acres (Goeking and others, in preparation). A casual observer may conclude that the 

estimated forest land area for New Mexico is 27.4% of this land area, or 21.3 million 

acres, despite the unstated implication that over 6.7 million acres are neither forest 

nor nonforest (i.e., nonsampled). This is obviously erroneous; the estimated forest 

land area, based on stratification for variance reduction and bias reduction, is over 

24.8 million acres. Indeed, the latter estimate is reported in several other standard 

summary tables. This discrepancy may cause confusion, or worse, misinterpretation 

of FIA’s estimates, and therefore should be remedied to remove ambiguity regarding 

sample plot area versus state land area.

Conclusions

The modified stratification methods presented here improved our ability to meet the 

assumption that nonresponse occurs randomly within each stratum and therefore 

minimized nonresponse bias in the New Mexico forest inventory of 2005-2013. We 

developed a stratification guide based on the statistical principles of minimum sample 

size and response homogeneity, and then applied that guide to generate a stratification scheme where no stratum had fewer than 10 response plots and individual strata 

were defined based on similarity of nonresponse rates. No ambiguous situations were 

encountered throughout the four steps stipulated in the guide, implying that the 

stratification’s decision rules are objective and repeatable. This modified stratification 

was used to generate FIA’s publicly available estimates of forest attributes in New 

Mexico for 2005-2013.

Under the modified stratification, estimates of forest land area were over one million 

acres higher than the estimated area under the standard green/brown stratification. 

This considerable difference can be attributed to the inclusion of ownership status, 

which is a major determinant of each plot’s likelihood of nonresponse, as a stratification variable in the modified scheme. New Mexico’s forest inventory had a statewide 

nonresponse rate for private lands of 13.9%, while non-private lands had only 2.1% 

nonresponse. The fact that non-private versus private strata had very different rates of 

nonresponse, whereas green versus brown strata did not, underscores the importance 

of incorporating ownership as a stratification variable in situations where high nonresponse occurs on certain ownership categories. The large range of nonresponse rates 

among our final strata (0-59%) signified that including ownership status increased 

response homogeneity within strata.
16

Research Note RMRS-RN-59. 2013

The differences in estimated forest land area from the two stratification schemes reinforces the recommendations of Patterson and others (2012) to increase response 

homogeneity within strata and thus reduce bias caused by nonrandom nonresponse. 

In states where nonresponse rates are related to factors other than Phase 1 strata, forest estimates based only on green/brown status may underestimate attributes such as 

forest land area, volume, growth, and mortality. Several states, including all of those 

within the Interior West region, have recently exhibited more than 10% nonresponse 

on private lands (Patterson and others 2012). Thus, to minimize nonresponse bias in 

forest estimates, these states should be stratified based on ownership groups in addition to existing Phase 1 classes.

The factors affecting nonresponse rates should be identified and included in modified stratification schemes whenever possible, and they may include land ownership 

status or other variables such as non-field/field status, topographic classes, or other 

variables. For example, extremely steep topography, such as that on the Hawaiian 

island of Kauai or the rugged karst region of Puerto Rico, may encompass a high proportion of physically inaccessible plots. Given an adequate sample size, plots in these 

areas could be stratified using slope, where slope values would be classified as either 

greater than a critical threshold or lower than that threshold. Selection of stratification categories should favor variables that exist as spatially continuous datasets so that 

stratum weights can be calculated directly, based on raster and/or polygon datasets, 

rather than estimated from sample points.

Future Research about Nonresponse and Stratification

Future research will investigate the sensitivity of forest estimates to our thresholds for 

sample size and nonresponse rate. First, we need to verify that a sample size threshold 

of at least 10 response plots per stratum is adequate and does not produce an unacceptably large variance. For strata that were not split into non-field/field substrata, 

the variance is calculated as described in Bechtold and Patterson (2005). However, an 

estimated variance must be derived for strata that use estimated rather than known 

strata weights (i.e., those that were split into non-field/field substrata). If the estimated variance is unacceptably large for small strata, then we may need to reconsider the 

minimum sample size as described in Westfall and others (2011). Second, we plan to 

simulate various levels of nonresponse to assess our definition of “high nonresponse.” 

Although we suspect 5% is a reasonable threshold for categorizing high-nonresponse 

versus low-nonresponse strata, we will compare estimates of forest attributes produced under other simulated nonresponse thresholds to evaluate this definition.

Finally, customized stratification methods can be implemented on a trial basis using 

the Forest Inventory Estimation for Analysis (FIESTA) software package currently 

in development (Frescino and others 2012). FIESTA will enable forest analysts to 

perform custom analyses on forest inventory data, including the ability to simulate 

alternative stratification schemes with user-defined stratification variables, minimum 

sample sizes, and nonresponse thresholds. The ability to simulate the consequences of 

modifying FIA’s stratification schemes will surely prove useful as we incorporate the 

concept of response homogeneity in other states with high nonresponse.
17

Research Note RMRS-RN-59. 2013

References

Bechtold, William A.; Patterson, Paul L., eds. 2005. The enhanced forest inventory and analysis program: national sampling design and estimation procedures. 

Gen. Tech. Rep. SRS-80. Asheville, NC: U.S. Department of Agriculture, Forest 

Service, Southern Research Station. 85 p.

Frescino, Tracey S.; Patterson, Paul L.; Freeman, Elizabeth A.; Moisen, Gretchen G. 

2012. Using FIESTA, an R-based tool for analysts, to look at temporal trends in 

forest estimates. In: Morin, Randall S.; Liknes, Greg C., comps. Moving from 

status to trends: Forest Inventory and Analysis (FIA) symposium 2012; 2012 

December 4-6; Baltimore, MD. Gen Tech. Rep. NRS-P-105. Newtown Square, 

PA: U.S. Department of Agriculture, Forest Service, Northern Research Station. 

[CD-ROM]: 74-78.

Gillespie, Andrew J.R. 1999. Overview of the annual inventory system established by 

FIA. Journal of Forestry 97(12): 16-20.

Goeking, Sara A.; Shaw, John D.; Witt, Chris; Werstak, Charles; Thompson, Michael 

T.; Menlove, Jim; Amacher, Michael C.; Stuever, Mary; Morgan, Todd A.; Hayes, 

Steven W.; McIver, Chelsea P.; Sorenson, Colin B.; Bailey, Robert G. [In preparation]. New Mexico’s Forest Resources. Fort Collins, CO: U.S. Department of 

Agriculture, Forest Service, Rocky Mountain Research Station.

Patterson, Paul L.; Goeking, Sara A. 2012. Estimators used in the New Mexico forest inventory: practical implications of “truly” random nonresponse within each 

stratum. In: Morin, Randall S.; Liknes, Greg C., comps. Moving from status to 

trends: Forest Inventory and Analysis (FIA) symposium 2012; 2012 December 

4-6; Baltimore, MD. Gen Tech. Rep. NRS-P-105. Newtown Square, PA: U.S. 

Department of Agriculture, Forest Service, Northern Research Station. [CDROM]: 330-333.

Patterson, Paul L.; Coulston, John W.; Roesch, Francis A.; Westfall, James A.; Hill, 

Andrew D. 2012. A primer of nonresponse in the US Forest Inventory and Analysis 

program. Environmental Monitoring and Assessment 184(3): 1423-1433.

Särndal, C.; Swensson, B.; Wretman, J. 1992. Model assisted survey sampling. New 

York: Springer-Verlag. 694 p.

Scott, Charles T.; Bechtold, William A.; Reams, Gregory A.; Smith, William D.; 

Westfall, James A.; Hansen, Mark H.; Moisen, Gretchen G. 2005. Sample-based 

estimators used by the forest inventory and analysis national information management system. In: Bechtold, W.A.; Patterson, P.L., eds., The enhanced forest 

inventory and analysis program: national sampling design and estimation procedures. Gen. Tech. Rep. SRS-80. Asheville, NC: U.S. Department of Agriculture, 

Forest Service, Southern Research Station: 53-77.

USDOI Bureau of Land Management. 2012. BLM New Mexico surface ownership. 

Digital GIS dataset published Sept. 28, 2012. Available: http://www.nm.blm.gov/

shapeFiles/SURFACE_OWN.zip. (Accessed Oct. 3, 2012.)

Westfall, James A.; Patterson, Paul L.; Coulston, John W. 2011. Post-stratified estimation: within-strata and total sample size recommendations. Canadian Journal 

of Forest Research 41: 1130-1139.
18

Research Note RMRS-RN-59. 2013

Woudenberg, Sharon W.; Conkling, Barbara L.; O’Connell, Barbara M.; LaPoint, 

Elizabeth B.; Turner, Jeffery A.; Waddell, Karen L. 2010. The Forest Inventory 

and Analysis Database: Database description and users manual versions 4.0 for 

Phase 2. Gen. Tech. Rep. RMRS-GTR-245. Fort Collins, CO: U.S. Department 

of Agriculture, Forest Service, Rocky Mountain Research Station. 336 p.
19

Research Note RMRS-RN-59. 2013

Appendix—Guide to Stratification Based 

on Response Homogeneity Groups.

Procedures for creating custom stratification schemes in 

states with high nonresponse

When Applied: In any state inventory that has ≥5% nonresponse among 

all plots within any discernible group (e.g., private plots)

Procedure Overview: Acquire green/brown spatial data layer (forest/nonforest 

strata) and statewide surface management layer (or 

other layer that allows discerning the group(s) of high 

nonresponse); then follow the steps below, in order, 

to produce initial strata, combine strata to achieve 

the minimum sample size, and split strata with high 

nonresponse to achieve response homogeneity.

Step 1: Select stratification variables and cross-tabulate with 

the standard Phase 1 strata

1.1 Acquire the spatial layer representing the standard stratification scheme of green/

brown/blue stratification, where strata represent potential forest/nonforest/

census water. (In the Interior West FIA region, blue is typically lumped with 

brown to yield a green/brown stratification.)

1.2 Identify a classification variable that is an important determinant of response 

probability. In New Mexico, that variable was ownership (non-private/private). 

Spatially intersect the Phase 1 green/brown strata with a statewide surface 

management GIS layer. Recode all non-private, non-tribal ownerships into 

a single stratum; recode all private ownership classes (including tribal) into 

another single stratum. This creates four initial strata:

1) green/nonprivate,

2) green/private,

3) brown/nonprivate, and

4) brown/private.

From the resulting spatial layer, we can calculate stratum weights and produce plot 

stratum assignments.

Step 2: Combine estimation units and/or strata where the 

sample size is too small

2.1 For any estimation unit with less than 10 response plots, combine the estimation 

unit with another estimation unit, where the target estimation unit is the one 

whose centroid is nearest the centroid of the source estimation unit and is in 

the same survey unit as the source estimation unit. Maintain the initial strata 

from both source and target estimation units. Combine the strata of the two 
20

Research Note RMRS-RN-59. 2013

estimation units and then proceed to Step 2.2 to consider each combined 

stratum’s sample size and nonresponse rate.

For example, if one estimation unit has plots in strata 1, 3, and 4, and another 

estimation unit has plots in strata 1, 2, and 3, the combined estimation unit will 

have all four strata, and strata 1 and 3 will include plots from both estimation 

units. Each of the four strata will be examined in Step 2.2.

2.2 The purpose of this step is to eliminate strata with small sample sizes by 

combining them with the appropriate strata. For descriptive purposes, when 

strata are combined we define the stratum with <10 sampled plots as the 

“source” stratum; the stratum into which we combine the source stratum is 

referred to as the “target” stratum.

Begin with the first estimation unit (ESTN_UNIT=1; see Woudenberg and 

others 2010). Consider each stratum separately within the estimation unit, in 

order of the stratum numbers, and complete Step 2.2 for each stratum. When all 

strata in the estimation unit have been examined, proceed to the next sequential 

estimation unit and repeat Step 2.2 for all strata; continue until all estimation 

units and strata have been examined.

(A) Is the number of sampled plots in the stratum <10?

No: The original stratum stands. Proceed to Step 2.2 for the next stratum.

Yes: Is the level of nonresponse in the stratum <5%?

No: Go to Step 2.2(B).

Yes: Is there another stratum with <5% nonresponse in the same green/

brown category, and if so, will combining by green or brown yield ≥10 

response plots?

Yes: Combine based on green/brown (e.g., combine green/private 

with green/nonprivate). Proceed to the next stratum and repeat 

Step 2.2.

No: Is there another stratum with <5% nonresponse in the same 

ownership category, and if so, will combining by ownership 

yield ≥10 response plots?

Yes: Combine based on ownership (e.g., combine green/

private with brown/private). Proceed to the next stratum 

and repeat Step 2.2.

No: Does combining based on green/brown, regardless of 

nonresponse levels, result in ≥10 sampled plots in the new 

stratum?

Yes: Combine based on green/brown. Proceed to the 

next stratum and repeat Step 2.2.

No: Does combining based on ownership, regardless of 

nonresponse levels, result in ≥10 sampled plots in 

the new stratum?
21

Research Note RMRS-RN-59. 2013

Yes: Combine based on ownership. Proceed to the 

next stratum and repeat Step 2.2.

No: Does combining all strata in the estimation 

unit result in ≥10 sampled plots in the 

estimation unit?

Yes: Combine all strata in the estimation 

unit. Proceed to the first stratum in the 

next estimation unit and repeat Step 

2.2.

No: The estimation unit should have been 

combined with another estimation 

unit. Return to Step 2.1.

(B) The number of sampled plots in the source stratum is <10 and nonresponse is 

≥5%. Is there is another stratum with ≥5% nonresponse in the same green/brown 

category, and if so, does combining by green or brown yield ≥10 response plots?

Yes: Combine based on green/brown (e.g., combine green/private with green/

nonprivate). Proceed to the next stratum and repeat Step 2.2.

No: Is there another stratum with ≥5% nonresponse in the same ownership 

category, and if so, will combining by green/brown yield ≥10 response plots?

Yes: Combine based on ownership (e.g., combine green/private with 

brown/private). Proceed to the next stratum and repeat Step 2.2.

No: Does combining based on green/brown, regardless of nonresponse 

levels, result in ≥10 sampled plots in the new stratum?

Yes: Combine based on green/brown. Proceed to the next stratum 

and repeat Step 2.2.

No: Does combining based on ownership, regardless of nonresponse 

levels, result in ≥10 sampled plots in the new stratum?

Yes: Combine based on ownership. Proceed to the next 

stratum and repeat Step 2.2.

No: Does combining all strata in the estimation unit result in 

≥10 sampled plots in the estimation unit?

Yes: Combine all strata in the estimation unit. Proceed 

to the first stratum in the next estimation unit and 

repeat Step 2.2.

No: The estimation unit should have been combined 

with another estimation unit. Return to Step 2.1.
22

Research Note RMRS-RN-59. 2013

Step 3: Split strata that have high nonresponse

After completing this step for each stratum within each estimation unit, complete 

Step 4. Again, consider each stratum within each estimation unit separately, and 

proceed through estimation units and strata in ascending numerical order.

(A) Is nonresponse in the stratum ≥5%?

No: Do not split the stratum. Proceed to Step 3 for the next stratum.

Yes: If the stratum is split into non-field/field groups, will there be ≥10 sampled 

plots in both the non-field and field strata?

No: Do not split the stratum. Proceed to Step 3 for the next stratum.

Yes: Split the stratum further into non-field and field strata (e.g., split 

brown/private into brown/private/non-field and brown/private/field). 

Proceed to Step 3 for the next stratum.

Step 4: Calculate and/or estimate stratum weights

Consider each estimation unit separately.

4.1 Calculate strata weights for strata that were not split into non-field/field 

substrata. Calculate the weights as the proportion of the estimation unit’s area 

that occurs within each stratum (i.e., divide the stratum’s area by the estimation 

unit’s area).

4.2 Estimate strata weights for strata that were split into non-field/field substrata. 

First calculate the hypothetical stratum weight for the two substrata in 

aggregate, as though they had not been subdivided, as described in Step 4.1. 

Then calculate the proportion of plots in the aggregate stratum that are in the 

non-field and field substrata. Multiply these two proportions by the calculated 

aggregate stratum weight to obtain the estimated strata weights for the 

respective non-field/field substrata.
The Rocky Mountain Research Station develops scientific information 

and technology to improve management, protection, and use of the 

forests and rangelands. Research is designed to meet the needs of 

the National Forest managers, Federal and State agencies, public and 

private organizations, academic institutions, industry, and individuals. 

Studies accelerate solutions to problems involving ecosystems, range, 

forests, water, recreation, fire, resource inventory, land reclamation, 

community sustainability, forest engineering technology, multiple use 

economics, wildlife and fish habitat, and forest insects and diseases. 

Studies are conducted cooperatively, and applications may be found 

worldwide. For more information, please visit the RMRS web site at: 

www.fs.fed.us/rmrs.

Station Headquarters

Rocky Mountain Research Station

240 W Prospect Road

Fort Collins, CO 80526

(970) 498-1100

Research Locations

The U.S. Department of Agriculture (USDA) prohibits discrimination against its 

customers, employees, and applicants for employment on the bases of race, 

color, national origin, age, disability, sex, gender identity, religion, reprisal, and 

where applicable, political beliefs, marital status, familial or parental status, 

sexual orientation, or all or part of an individual’s income is derived from any 

public assistance program, or protected genetic information in employment 

or in any program or activity conducted or funded by the Department. (Not all 

prohibited bases will apply to all programs and/or employment activities.) For 

more information, please visit the USDA web site at: www.usda.gov and click on 

the Non-Discrimination Statement link at the bottom of that page.

Reno, Nevada

Albuquerque, New Mexico

Rapid City, South Dakota

Logan, Utah

Ogden, Utah

Provo, Utah

Flagstaff, Arizona

Fort Collins, Colorado

Boise, Idaho

Moscow, Idaho

Bozeman, Montana

Missoula, Montana

To learn more about RMRS publications or search our online titles:

www.fs.fed.us/rm/publications

www.treesearch.fs.fed.us