United States

Department

of Agriculture

Forest Service

Rocky Mountain

Research Station

Research Paper

RMRS-RP-87

April 2011

Understory Vegetation Data Quality 

Assessment for the Interior West 

Forest Inventory and Analysis Program

Paul L. Patterson 

Renee A. O’Brien
You may order additional copies of this publication by sending your 

mailing information in label form through one of the following media. 

Please specify the publication title and number.

Publishing Services

Telephone (970) 498-1392

FAX (970) 498-1122

E-mail rschneider@fs.fed.us

Web site http://www.fs.fed.us/rmrs

Mailing Address Publications Distribution

Rocky Mountain Research Station

240 West Prospect Road

Fort Collins, CO 80526

Patterson, Paul L.; O’Brien, Renee A. 2011. Understory vegetation data quality assessment for the Interior West Forest and Inventory Analysis program. Res. Pap. RMRSRP-87. Fort Collins, CO: U.S. Department of Agriculture, Forest Service, Rocky Mountain 

Research Station. 9 p.

Abstract

The Interior West Forest Inventory and Analysis (IW-FIA) program of the USDA Forest Service 

collects field data on understory vegetation structure that have broad applications. In IW-FIA 

one aspect of quality assurance is assessed based on the repeatability of field measurements. 

The understory vegetation protocol consists of two suites of measurements; (1) the structure 

by lifeform and (2) the structure for individual species with cover greater than five percent. The 

measurements of structure by lifeform and species are highly repeatable; however, since species 

identification is confounded with cover threshold, the repeatability of species identification cannot 

be evaluated using FIA’s tolerance and compliance rate protocol for measuring repeatability. The 

protocols used to collect the data analyzed in this paper are specific to IW-FIA. A standardized 

national understory vegetation procedure that is very similar to the IW-FIA procedure described 

here will be implemented in 2011 for all FIA units. 

Keywords: understory, vegetation, species, cover, forest inventory

Authors

Paul L. Patterson, Mathematical Statistician, USDA Forest Service, Rocky Mountain Research 

Station, Fort Collins, Colorado.

Renee A. O’Brien, Deputy Program Manager, USDA Forest Service, Rocky Mountain Research 

Station, Interior West Forest Inventory and Analysis Unit, Ogden, Utah.

Acknowledgments

This research was funded by the USDA Forest Service Forest Inventory and Analysis program. 

The authors thank all of the reviewers; their comments greatly improved the quality of this paper.
Contents

Introduction.....................................................................1

Methods............................................................................2

Data..........................................................................2

Analysis....................................................................3

Results...............................................................................5

Matched Records...................................................5

Extra Records .........................................................7

Species Variable......................................................7

Discussion/Conclusions................................................8

Lifeform Suite of Variables..................................8

Species Suite of Variables.....................................8

References.........................................................................8

Research Summary

There are two major conclusions from this research: (1) the ocular estimates of the percent 

crown canopy cover by a lifeform and layer are very repeatable; and (2) the species protocol 

is structured in such a way that it is not currently possible to conduct a quality assessment of 

the species variable using the standard FIA MQO of a specified compliance rate for a given 

tolerance level.

USDA Forest Service Res. Pap. RMRS-RP-87. 2011 1

Introduction

The Forest Inventory and Analysis (FIA) program of 

the USDA Forest Service conducts forest inventories to 

meet the requirements for national assessments mandated 

by the Forest and Rangeland Planning Act of 1974 and 

other legislation. FIA inventories provide a statistically 

defensible, probability-based sample of forest resources 

across all ownerships that can be used for planning and 

analysis at the National Forest, State, regional, or national 

level. The FIA sample was designed to meet national 

standards for precision in state and regional estimates 

of forest attributes. Field crews conduct the field phase 

of the inventory on forest land and on some non-forest 

land. The sampling intensity is approximately one field 

plot every 5900 acres.

The Interior West FIA (IW-FIA) program of the 

Forest Service’s Rocky Mountain Research Station, as 

part of its national inventory activities, collects data on 

understory vegetation (O’Brien and Van Hooser 1983). 

Ocular estimates of percent crown canopy cover and 

height layer are recorded for four lifeforms and for some 

individual species on all FIA plots. Hereafter “crown 

canopy cover” is called “cover” and “height layer” is 

called “layer”. Percent cover is the area of ground surface 

covered by the canopy of a plant, presented as a percent 

of the total area sampled. Understory information, in 

combination with other data collected on FIA field 

plots, provides a structural picture of the total plant 

community that can be used in many applications such 

as estimating wildlife habitat, fuel characteristics, grazing potential, or presence of noxious or invasive plants. 

The FIA understory vegetation data have been used to 

predict indicators of rangeland health and functionality 

(O’Brien and others 2003), estimate biomass (Mitchell 

and others 1987), estimate forest fuels (Gebert and others 2008), inform vegetation types and maps of canopy 

cover for LANDFIRE (Toney and others 2007), and 

quantify woodpecker habitat (Witt 2009).

FIA’s quality assurance program entails extensive 

crew training and nationally consistent protocols. Quality control (QC) procedures include direct feedback to 

the field staff to provide continual real-time assessment 

and improvement of crew performance. In addition to 

extensive QC activities, data quality is assessed and 

documented using performance measurements and 

post-survey assessments. The quality assessment measurements are stated as measurement quality objectives 

(MQOs) and are designed to provide a range of accuracy 

that is allowable for any given field measurement. The 

MQOs are used to identify areas of the data collection 

process that need improvements or refinements in order to meet the quality objectives of the FIA program. 

Quality assessment analysis has been conducted for all 

FIA national core variables (Pollard and others 2006) 

and for all forest health variables (Westfall 2009); and 

at the time of this paper, it is being conducted on IWFIA variables.

Quality assurance of IW-FIA understory vegetation 

data is based on a second, independent measurement 

called a “blind check” that is conducted on a random 

subset of IW-FIA field plots. The blind check allows for 

the difference between the measurements to be compared 

against a pre-specified tolerance, and it allows for the 

calculation of an overall compliance rate that is used 

to assess the repeatability of the understory vegetation 

measurements. The MQOs for the IW-FIA understory 

vegetation protocols are stated in terms of tolerance and 

compliance rate.

The purpose of this paper is to (1) present a quality 

assessment of the IW-FIA understory vegetation protocols using quality assurance data and the stated MQOs; 

and (2) show if a quality assessment of the protocol to 

measure species variables can be conducted using the 

Understory Vegetation Data Quality 

Assessment for the Interior West Forest 

Inventory and Analysis Program 

Paul L. Patterson and Renee A. O’Brien
2 USDA Forest Service Res. Pap. RMRS-RP-87. 2011

current MQOs. All data collection experts and users of 

IW-FIA understory vegetation data will find the results 

on the repeatability important and useful. The protocols used to collect the data analyzed in this paper are 

specific to IW-FIA. A standardized national understory 

vegetation procedure that is very similar to the IW-FIA 

procedure described here will be implemented in 2011 

for all FIA units. The analyses in this paper can be 

used in establishing the MQO standards for the national 

protocols.

Methods

Data

There are two types of data; the first is data collected 

using the IW-FIA understory vegetation protocols, 

and the second is data used to conduct the quality assessment of the understory vegetation protocols. The 

IW-FIA understory vegetation variables are measured 

on all forested FIA plots; the FIA plot is comprised of 

four subplots. Percent cover is estimated for the tree, 

shrub, forb, and graminoid lifeforms by three layers 

(figure 1) on each sampled subplot. Crews also estimate 

and record a summary percent “aerial” cover for each 

lifeform, that is, all the height layers are collapsed and 

the crew estimates the percent cover from a bird’s eye 

view. For example, in figure 1, the tree lifeform has 5 

percent cover for Layer 1, 6 percent cover for Layer 2, 

9 percent cover for Layer 3, and 16 percent aerial cover 

(for the aerial, the trees on the right side of the subplot 

have 12 percent aerial cover, not 15 percent); while the 

grass lifeform has 9 percent cover in Layer 1, 0 percent 

cover in Layers 2 and 3, and 9 percent aerial cover. In 

addition to the lifeform estimates, percent cover and 

layer estimates are recorded for each species with 5 

percent or greater cover on each sampled subplot; the 

layer assigned is the one where the bulk of the cover 

occurs. For another example, in figure 1, there are two 

tree species with cover greater than 5 percent. For tree 

species 1, the cover is 11 percent and the bulk of the 

Figure 1. Adapted from the FIA field manual (USDA 2008), this figure shows how percent crown canopy cover for 

vegetation on a subplot is estimated by layer.
USDA Forest Service Res. Pap. RMRS-RP-87. 2011 3

cover occurs in Layer 3; for tree species 2, the cover is 

8 percent and the bulk of the cover occurs in Layer 2. 

As one last example, in figure 1, the forb lifeform would 

have 4 percent cover for both Layer 1 and the aerial and 

0 percent cover for Layer 2 and 3, but since the cover 

is less than 5 percent, no forb species would be listed. 

More detailed field protocols are documented in the 

IW-FIA Field Procedures manual (USDA 2008).

The data used for data quality assessment of any 

FIA variable are generated by a second independent 

measurement of a field plot called a blind check. This 

involves the re-installation of an inventory plot by 

another field crew without the first field crew’s data 

in hand. The first measurement of the plot by the field 

crew is referred to as the field measurement (FM), and 

the second measurement by another crew is referred to 

as the quality assessment (QA). This QA measurement 

is considered to be a “blind” measurement because the 

QA crew does not have knowledge of the FM crew’s 

original measurement. In addition, FM crews do not 

know when or which of their plots will be re-measured 

by a QA crew and, therefore, cannot alter their performance on a given plot. This type of blind measurement 

provides a direct, unbiased observation of measurement 

precision from two independent crews. Blind check plots 

are randomly selected to be a representative sub-sample 

of all plots measured.

Typically, the blind check plots are measured within 

two weeks of the FM crew measurement to avoid the 

confounding effects of seasonal changes. The FM data 

for all plots usually go through more edit checks than 

QA data. For this analysis, so that the data are comparable, we used the version of the FM data that had 

not gone through the additional edit checks. The blind 

check data used in this study are from the annual inventory system for the following states and years: Arizona 

(2001 to 2008), Colorado (2002 to 2008), Idaho (2004 

to 2008), Montana (2003 to 2008), and Utah (2001 to 

2008). There is a total of 493 plots or 1972 subplots.

Analysis

Quality assessment is accomplished by calculating 

the differences between the FM crew and QA crew 

data and comparing the differences to pre-established 

MQOs. The standard FIA model for an MQO of a data 

element is a combination of a tolerance limit and a compliance rate for each data element. The tolerance limit 

or acceptable measurement error is selected based on a 

balance between the accuracy necessary for scientific 

or analytical purposes and the ability of the crews to 

make repeatable measurements or observations within 

the assigned tolerance. In the analysis of blind check 

data, an observation is within tolerance when the difference between the FM crew and QA crew observations 

does not exceed the assigned tolerance for that data 

element. The compliance rate is the minimum percentage of observations that should be within tolerance. 

For example, the tolerance for the diameter of a tree at 

breast height (DBH) is ± 0.1 inch for each 20.0 inches 

in diameter of a live tree, and the compliance rate is 95 

percent. The objective is that 95 percent or more of the 

DBH observations are within ± 0.1 inch for each 20.0 

inches of diameter for all live trees measured by both the 

FM and QA crews. Results can be displayed as a simple 

percent of the computed differences that fall within 

the program tolerances. This percentage is referred to 

as the observed compliance rate. The tolerance limits 

and compliance rates are documented in the IW-FIA 

field procedures manual (USDA 2008). Tolerance limits 

have been established for understory variables, but there 

are no established compliance rates for the understory 

variables.

The first step in analyzing blind check data is to match 

the FM and QA measurements. Because of the blind 

aspect of the protocols there is the possibility of “extra” 

observations, that is, one crew records an observation that 

the other crew does not. For example, in the understory 

vegetation protocols, a species is recorded if it covers at 

least 5 percent of the plot, so either the FM or QA crew 

may assess a species cover at 7 percent and record the 

species while the other crew assesses the species cover 

at 4 percent and does not record the species. The extra 

observations are analyzed separately for each crew type 

(FM and QA). The extra records are presented as total 

and as a percentage of the recorded observation, that is, 

the number of recorded observations is equal to the sum 

of the matched records and extra records. In equation 

form, the percentage extra is:

Percent extra = (Number of extras)+ (Number of matches)

Number of extras ; E * 100

The understory data are collected in two general 

groups—a lifeform suite of variables and a species suite 

of variables; the two suites use different identifiers in the 

matching process, so the matching process is described 

separately.

Lifeform Suite of Variables—For each of the four 

lifeforms, percent cover by three layers plus the percent 

aerial cover is recorded. The FM and QA measurements 

can be matched for the 16 variables using a unique 
4 USDA Forest Service Res. Pap. RMRS-RP-87. 2011

identifier number for each subplot. There is a possibility 

of some unmatched subplots; this occurs when either 

the FM or QA crew records lifeform data for a subplot 

but the other crew does not. There are two reasons 

why this can occur. First, the blind check protocols do 

not require that the QA crew collects data on all of the 

subplots, even though the FM crew sampled them. The 

subplots that the QA crew does not collect data for are 

recorded as non-sampled. Second, one crew may record 

the subplot center condition as “non-forested,” which 

means the crew won’t measure the lifeform variables, 

while the other crew may record the subplot center 

condition as “forested,” which means they will sample 

the subplot. In this paper, the extra lifeform observations are subplots that one crew recorded as non-forest 

and the other crew recorded as forest. For this paper, 

only the subplots that were jointly measured were used 

in the analysis of the MQOs for the lifeform suite of 

variables. The stated tolerance for all lifeform variables 

is ± 10 percent (USDA 2008); there is no stated compliance rate. To better understand and formulate a possible 

compliance rate, we analyzed the matched data using 

tolerances of ± 5 percent, ± 10 percent, ± 15 percent, 

and ± 20 percent.

Species Suite of Variables—Crews only record 

species data on subplots where they record lifeform 

data, so the analysis of the species suite of data was 

restricted to the subplots where both crews recorded 

lifeform data. The species suite of variables consists 

of four variables: species, lifeform, cover, and vegetation layer. For each subplot, the crew records up to 

four of the most dominant plant species within each 

lifeform that have at least 5 percent cover. The plants 

are recorded using the alphanumeric code, as listed in 

the USDA Natural Resources Conservation Service 

PLANTS database (USDA 2000). Because the FIA 

protocols were developed to be easily implemented by 

non-botanists and because the FIA inventory field work 

is conducted in early spring and late fall, concessions 

have been made regarding species identification. Some 

plants require identification only at the genus level; for 

example, all graminoids are coded at the genus level 

only; while for trees, only Salix (willow) is recorded 

at the genus level—all other trees are recorded at the 

species level. For a complete list of plants identified at 

only the genus level see USDA (2008). For the purposes 

of this report, there are two uses of the word species; 

one use designates the species suite of variables, and 

the other is for the variable whose value is the species 

or genus code of the plant. The unique identifier for a 

species suite record is comprised of state, county, plot, 

and subplot numbers, plus the species code. We refer to 

this unique identifier as species-within-subplot or genuswithin-subplot, depending on the level of identification 

required. Because the crews identify species within the 

lifeforms, the matching of the species suite records is 

conducted separately for each lifeform. Also, the records 

are further divided into those species that are to be 

identified at the species level and those that are to be 

identified at the genus level. For each lifeform, after the 

initial matching, there are five data sets: (1) the matched 

records, (2) the FM unmatched species-within-subplot, 

(3) the FM unmatched genus-within-subplot, (4) the 

QA unmatched species-within-subplot, and (5) the QA 

unmatched genus-within-subplot.

If a crew is not able to identify a plant in the field, 

the species is recorded as UNKN1, UNKN2, etc., and 

a specimen is obtained to send to a specialist. If the 

specialist cannot identify the plant, the record remains 

in the database as an unknown species. For this analysis, 

any unknown species was compared by a plant identification specialist to any unmatched plant records to 

determine if the records with unknown species could 

be matched. This process is based on the percent cover 

and the vegetation layer and is compared to one of the 

remaining unmatched records with the same lifeform 

and opposite crew type. These “manual” matches were 

added to the data set of matched records.

The remaining QA unmatched species-within-subplot 

and FM unmatched species-within-subplot records are 

compared for matches based on the identifier genuswithin-subplot. Any genus-level matches of specieswithin-subplot are added to the matched data set and 

are removed from the appropriate unmatched data set; 

these genus-level matches of species-within-subplot are 

considered to be out of tolerance for the species variable.

At this point, all possible matches have been made 

directly by species, indirectly by genus, or by manual 

estimation, and the matching process is concluded. We 

refer to the data set of matched records as the matched 

species data set even though some were matched at the 

genus level. When the matching process is concluded, 

there are three data sets: the matched species, the extra 

FM species, and the extra QA species. Compliance rates 

for the variables lifeform, percent cover, and vegetation 

layer are analyzed using the matched species data set. 

There are no compliance rates for these variables. To 

better understand and formulate a compliance rate, the 

percent cover variable was analyzed using tolerances of 

± 5 percent, ± 10 percent, ± 15 percent, and ± 20 percent. 

The vegetation layer was analyzed using tolerances of 
USDA Forest Service Res. Pap. RMRS-RP-87. 2011 5

zero layers, ± one layer, and ± two layers. The lifeform 

variable was analyzed using a tolerance of zero.

In the FIA protocols, the species variable has a tolerance of no errors on species identification, but this 

tolerance level is confounded by the cover threshold. 

For example, if a crew estimates the cover of a species 

to be less than 5 percent, they won’t record that species. 

But the other crew may estimate the cover to be greater 

than 5 percent and will record the species. As a result, 

the standard FIA MQO model of a tolerance limit and 

compliance rate is not strictly applicable. There are 

two reasons for an extra species record: (1) the other 

crew determined the cover for the species to be less 

than 5 percent and did not record the species, or (2) the 

species was not identified correctly by one crew and, 

therefore, does not show up as a match. The former is 

within the tolerance of the species variable while the 

latter is outside of the tolerance of the species variable. 

Therefore, the extra records contribute to whether the 

compliance rate for the species variable is met.

The closer the cover percent of the extra record is to 

5 percent, the more likely that the record will be within 

tolerance. For example, it is more likely for 6 percent 

cover to be within tolerance than 15 percent cover. As 

an indication of the level of non-repeatability for the 

species variable, we compared the number of matched 

records to a frequency distribution of cover for each set 

of extra records.

Results

Matched Records

The term “compliance rate” refers to the compliance 

rate at a tolerance of 5 percent; when referring to the 

compliance rate for the other tolerances, the tolerance 

is stated along with the compliance rate.

Lifeform Suite of Variables—Data from 1723 

subplots where both the FM and QA crews recorded 

lifeform data were used to analyze the compliance rates 

for the 16 lifeform variables. Due to the lack of a compliance rate, the data were analyzed using tolerances of 

± 5 percent, ± 10 percent, ± 15 percent, and ± 20 percent 

(table 1). The compliance rate ranged from 77 percent to 

100 percent. For a ± 10 percent tolerance, the compliance rate ranged from 87 percent to 100 percent, and 

all but two layers had a compliance rate above 90 percent. The minimum compliance rate for a tolerance of 

± 15 percent was 93 percent with the majority of layers 

above 96 percent. Note that for graminoids, forbs, and 

trees, the compliance rate for the percent aerial cover 

estimate was close to the compliance rate where most of 

the vegetation occurred—Layer 1 for the graminoids and 

forbs and Layer 3 for the trees. The lowest compliance 

rates at any tolerance were for graminoid percent aerial 

cover, which were 76 percent, 87 percent, 93 percent, 

and 95 percent at tolerances ± 5 percent, ± 10 percent, 

Table 1—Percent observed compliance rate with a tolerance of 5 percent for 12 lifeform variables 

from 1723 matched subplots.

Percent observed compliance rate at tolerance level

Variable ±5% ±10% ±15% ±20% Records

Tree Cover Layer 1 97.1% 99.1% 99.8% 99.9% 1714

Tree Cover Layer 2 91.2% 96.9% 98.6% 99.3% 1720

Tree Cover Layer 3 80.9% 91.2% 95.5% 97.4% 1720

Tree Cover Aerial 77.7% 88.7% 93.9% 96.3% 1714

Shrub Cover Layer 1 87.0% 95.1% 97.4% 98.4% 1703

Shrub Cover Layer 2 88.3% 94.5% 97.5% 98.6% 1713

Shrub Cover Layer 3 98.0% 99.2% 99.6% 99.8% 1713

Shrub Cover Aerial 77.9% 88.8% 93.6% 96.4% 1703

Forb Cover Layer 1 87.3% 94.3% 97.3% 98.5% 1688

Forb Cover Layer 2 95.9% 98.1% 98.9% 99.2% 1701

Forb Cover Layer 3 99.9% 100.0% 1704

Forb Cover Aerial 85.0% 92.7% 96.2% 97.8% 1688

Graminoid Cover Layer 1 77.0% 87.7% 93.4% 95.9% 1689

Graminoid Cover Layer 2 94.2% 96.9% 98.2% 98.9% 1701

Graminoid Cover Layer 3 100.0% 1704

Graminoid Cover Aerial 75.8% 86.8% 92.8% 95.4% 1689
6 USDA Forest Service Res. Pap. RMRS-RP-87. 2011

± 15 percent, and ± 20 percent, respectively. Not all 

records were used for each variable; the last column of 

table 1 shows the number of records used. The number 

of records used was often less than 1723 subplots because there was snow on some of the subplots, which 

sometimes restricted the estimates.

When considering the compliance rates in table 1, 

a combination of the lifeform and the layer should be 

taken into consideration. For example, the compliance 

rate for graminoid cover in Layer 3 was 100 percent due 

to the fact that very few graminoids grow to that height, 

so the crews agreed the cover was zero.

Species Suite of Variables—Analysis of compliance 

rates for three of the species suite variables (percent 

canopy cover, vegetation layer, and lifeform) was restricted to the records that were matched by the process 

detailed in the Methods section. The fourth variable, 

species, could not be analyzed using the standard FIA 

MQO structure; aspects of quality assurance for the 

species variable are addressed in the Species Variable 

section.

There were 2509 matched species suite records 

across 1723 subplots. A breakdown of type of matches 

by lifeform is shown in table 2. There was 100 percent 

agreement between the crews for the lifeform variable. 

For the predominate vegetation layer of a species (that is, 

the “layer” variable), the tolerance levels were in agreement, ± 1 layer, and ± 2 layers, with compliance rates 

of 87.2 percent, 99.7 percent, and 100 percent, respectively. Due to the lack of a compliance rate, the percent 

cover was analyzed using tolerances of ± 5 percent, 

± 10 percent, ± 15 percent, and ± 20 percent (table 3). 

In addition to the compliance rate for all lifeforms, the 

compliance rates are also given for the percent cover by 

individual lifeform (table 3). The compliance rate for 

percent cover was 71 percent, and the compliance rate 

by lifeform ranged from 66 percent for graminoids to 

82 percent for forbs. At a tolerance of ± 10 percent, the 

compliance rate for all species was 86 percent, and the 

compliance rate by lifeform ranged between 82 percent 

and 92 percent. The highest compliance rate, regardless 

of tolerance level, was for forbs and the lowest was for 

graminoids.

Table 2—The number of matches by type of match and by lifeform, where “Species” 

are matches at the species-within-subplot level; “Genus” are matches at the 

genus-within-subplot level; “Manual” are species that are listed as unknown 

and matched by an expert based on other attributes such as cover; and 

“Genus-species” are matches at the genus-within-subplot level for species 

that were supposed to be matched at the species-within-subplot level. 

Type of match

Lifeform Species Genus Manual Genus-species Total

Graminoids — 518 25 — 543

Forbs 178 20 12 9 219

Shrubs 707 10 11 35 763

Trees 940 11 0 33 984

Table 3—Percent observed compliance rate at tolerances of ± 5 percent, ± 10 percent, ± 15 percent, 

and ± 20 percent for the species percent crown canopy cover variable. The compliance 

rate is given for all lifeforms as well as separately for each lifeform.

Percent observed compliance rate at tolerance level

Variable ±5% ±10% ±15% ±20% Records

All lifeforms 70.7% 86.4% 92.5% 95.5% 2509

Graminoids 65.9% 81.6% 89.1% 91.9% 543

Forbs 81.3% 92.2% 95.0% 98.2% 219

Shrubs 71.3% 87.0% 92.4% 95.9% 763

Trees 70.5% 87.2% 93.8% 96.5% 984
USDA Forest Service Res. Pap. RMRS-RP-87. 2011 7

Extra Records

Lifeform Suite of Variables—Out of 1972 subplots 

where at least one of the crews recorded lifeform data, 

225 subplots were non-sampled by the QA crew and one 

subplot was non-sampled by the FM crew, which was 

due to snow that had melted off by the time the QA crew 

arrived. For the 1746 subplots that both crews sampled, 

23 were extra lifeform subplots, leaving 1723 matched 

lifeform subplots. The 23 extra lifeform subplots consisted of 13 QA extra subplots and 10 FM extra subplots, 

with an extra percentage of 0.86 and 0.58, respectively.

Species Suite of Variables—A frequency distribution of extra species records based on four categories of 

cover is shown in table 4; the four categories are (1) 5 to 

9 percent, (2) 10 to 14 percent, (3) 15 to 19 percent, and 

(4) greater than 19 percent. Although the percentage of 

extra records for each crew type was between 17 percent 

(QA—trees) and 49 percent (FM—forbs), the majority 

were in the 5 to 9 percent cover category.

Species Variable

As mentioned, the analysis of compliance rates for the 

species variable was confounded because of the 5 percent 

cover threshold. In an attempt to facilitate more discussion on the QA of the species suite of variables, we took 

a closer look at various aspects of the data. The usual 

MQO for FIA consists of a tolerance and a compliance 

rate. Because species are only listed if they are at least 

5 percent cover, there are two characteristics that are 

measured for the one variable: (1) the fact that it meets 

Table 4—The number of extra FM and QA species or genus records along with the percentage calculated as 100%*(number extra records)/(number matches + number extra records).

Cover

	 Lifeform	 	 5% to 9%	 10% to 14%	 15% to 19%	 ≥20%	 Total

Graminoids FM extra records 183 (22.2%) 43 (5.2%) 15 (1.8%) 39 (4.7%) 280 (34.0%)

 QA extra records 195 (23.3%) 41 (4.9%) 27 (3.2%) 31 (3.7%) 294 (35.1%)

Forbs FM extra records 163 (42.7%) 26 (6.0%) 11 (2.6%) 12 (2.8%) 212 (49.2%)

 QA extra records 104 (32.2%) 14 (4.0%) 6 (1.7%) 6 (1.7%) 130 (37.2%)

Shrubs FM extra records 199 (20.7%) 22 (2.2%) 12 (1.2%) 10 (1.0%) 243 (24.2%)

 QA extra records 215 (22.0%) 15 (1.4%) 6 (0.6%) 7 (0.7%) 243 (24.2%)

Trees FM extra records 170 (14.7%) 19 (1.6%) 8 (0.7%) 8 (0.7%) 205 (17.2%)

 QA extra records 165 (14.4%) 19 (1.6%) 6 (0.5%) 7 (0.6%) 197 (16.7%)

the 5 percent cover threshold, and (2) the species that 

are present. The MQO structure of a compliance rate 

at a stated tolerance for the species name does not take 

into account the “at least 5 percent cover” requirement. 

Neither the FM nor QA crew listed any species for 202 (12 

percent) of the 1723 subplots. In the standard FIA MQO 

analysis of the species variable, these subplots would 

not contribute to the repeatability of the measurement 

of the species variable even though there was complete 

agreement between crews that none of the plant species 

on the subplots covered 5 percent of the ground surface. 

Any additional attempt to quantify the similarity of the 

sets of species recorded by the FM crew versus the sets 

recorded by the QA crew (for example, using similarity 

coefficients) will always be constrained by the 5 percent 

threshold for recording or not recording species, which 

will cause the coefficient to be low due to the number 

of extra species records.

The matched records are a source of data for determining the probability that an extra record occurred 

because the other crew correctly identified the species 

but determined that the cover was less than 5 percent; 

using the matched records is complicated, since the 

joint distribution of QA and FM cover variables—is a 

censored distribution with no values below the 5 percent 

level. 

With the exception of forbs, the numbers of extra FM 

and QA species records were approximately the same 

(table 4), indicating that the extra records did not occur 

due to an artifact of whether the crew was an FM or QA 

crew. For all lifeforms, a majority of the extra records 

were in the 5 to 9 percent category.
8 USDA Forest Service Res. Pap. RMRS-RP-87. 2011

Discussion/Conclusions

Lifeform Suite of Variables

For all lifeforms, the compliance rate for the percent 

aerial cover was lower than the compliance rate for all 

other layers (regardless of the tolerance); this is likely 

a result of the difficulty of combining the vegetation at 

various heights to obtain total aerial cover. For the percent 

aerial cover estimate, crews are not supposed to double 

count overlapping crowns, which may further contribute 

to differences between estimates. The percentage of extra 

lifeform subplots was low and occurred at approximately 

the same rate for each crew type. The observed compliance rates for the lifeform suite of variables indicated 

that the cover variable is very repeatable at either the 

± 5 or ± 10 percent tolerance level for all lifeforms and 

layers. These data, in combination with other data collected on FIA field plots, provide a structural picture of 

the total plant community that has many uses, such as 

estimating wildlife habitat, fuel characteristics, biomass, 

and forage availability; validating vegetation maps; and 

assessing ecosystem health.

Species Suite of Variables

The species (and sometimes genus) detail of the FIA 

understory vegetation protocol was originally intended 

to supplement the lifeform estimates by providing more 

specific detail on the genus and/or species that make up 

the lifeforms present on an FIA subplot. The ± 5 percent 

threshold for recording species detail was added to limit 

the time and expertise needed by FIA field crews while 

still capturing basic, general information. The species 

protocol was intended to get information on common, 

abundant, or habitat indicator plants throughout forested 

ecosystems. The results for the matched species (table 3) 

indicate that, for species that meet the 5 percent cover 

threshold, the lifeform, percent cover, and layer variables 

are highly repeatable at zero tolerance, ± percent tolerance, and ± one layer, respectively. For all tolerance levels 

(with the exception of the ± 20 percent tolerance level for 

forb and tree percent aerial cover), the compliance rate 

by lifeform for the species cover (table 2) was less than 

the compliance rate for the percent aerial cover (table 3) 

of the corresponding lifeform. The higher percentage of 

extra records in table 4 for graminoids and forbs indicates 

that species identification is less repeatable for these 

lifeforms. The identification of graminoids to even the 

genus levels seemed to be problematic. In addition, the 

percentage of extra species records with cover greater 

than 10 percent for forbs was quite high. We recommend 

emphasizing the assessment of graminoids and forbs 

cover during field crew training.

The species protocol is structured in such a way that 

it is not possible to conduct a quality assessment of the 

species variable using the standard FIA MQO of a specified compliance rate for a given tolerance level. If this 

part of the understory protocol is intended to be assessed 

the same way as all other FIA variables within the FIA 

MQO system, we recommend (1) that additional analyses 

be performed to determine exactly what kind and level 

of repeatability is needed for the species variable to be 

of use and (2) that a method for quality assessment be 

devised. That method must take into account proposed 

uses of the species suite of variables; what level of 

accuracy is needed for the proposed uses; and how to 

measure that accuracy in terms of a single, independent 

re-measurement of the species variable, that is, a blind 

check measurement.

References

Gebert, K.M.; Schuster, E.G.; Woudenberg, S.; O’Brien, R. 2008. 

Estimating forest fuels in the Southwest using forest inventory 

data. Gen. Tech. Rep. PSW-GTR-189. Albany, CA: U.S. Department of Agriculture, Forest Service, Pacific Southwest Research 

Station: 39-48.

Mitchell, J.E.; Bartling, P.; O’Brien, R.; 1987. Understory coverbiomass relationships in the Front Range ponderosa pine zone. 

Res. Note. RM-471. Fort Collins, CO: U.S. Department of 

Agriculture, Forest Service, Rocky Mountain Forest and Range 

Experiment Station. 5 p.

O’Brien, R.A.; Van Hooser, D.D. 1983. Understory vegetation inventory: an efficient procedure. Res. Pap. INT-323. Ogden, UT: 

U.S. Department of Agriculture, Forest Service, Intermountain 

Forest and Range Experiment Station. 6 p.

O’Brien, R.A.; Johnson, C.M.; Wilson, A.M.; Elsbernd, V.C. 2003. 

Indicators of rangeland health and functionality in the Intermountain West. Gen. Tech. Rep. RMRS-GTR-104. Ogden, UT: U.S. 

Department of Agriculture, Forest Service, Rocky Mountain 

Research Station. 13 p.

Pollard, J.E.; Westfall, J.A.; Patterson, P.L.; Gartner, D.L.; Hansen, 

M.H.; Kuegler, O. 2006. National forest inventory and analysis 

data quality assessment report. Gen. Tech. Rep. RMRS-GTR-181. 

Fort Collins, CO: U.S. Department of Agriculture, Forest Service, 

Rocky Mountain Research Station. 43 p.

Toney, C.; Rollins, M.; Short, K.; Frescino, T.; Tymcio, R.; Peterson, B. 2007. Use of FIA plot data in the LANDFIRE Project. 

Pages 309-319 in: McRoberts, R.; Reams, G.A.; Van Deusen, 

P.C.; McWilliams, W.H., eds. Proceedings of the Seventh annual 

Forest Inventory and Analysis symposium; October 3-6, 2005; 

Portland, ME. Gen. Tech. Rep. WO-77. Washington, DC: U.S. 

Department of Agriculture, Forest Service.

U.S. Department of Agriculture [USDA]. 2008. Interior West forest 

inventory and analysis, field procedures, version 3.02. Ogden, UT: 

U.S. Department of Agriculture, Forest Service, Rocky Mountain 

Research Station, Forestry Science Laboratory. Available: http://

www.fs.fed.us/rm/ogden/data-collection/pdf/p2_manual_08.

pdf. [March 2008].
USDA Forest Service Res. Pap. RMRS-RP-87. 2011 9

U.S. Department of Agriculture, Natural Resources Conservation 

Service [USDA NRCS]. 2000. The PLANTS database. National 

Plant Data Center, Baton Rouge, LA. For copy contact Arlington, 

VA: U.S. Department of Agriculture, Forest Service, National 

Office Forest Inventory and Analysis, 1601 North Kent Street, 

Suite 400.

Westfall, James A., ed. 2009. FIA national assessment of data quality 

of forest health indicators. Gen. Tech. Rep. NRS-53. Newtown 

Square, PA: U.S. Department of Agriculture, Forest Service, 

Northeastern Research Service. 80 p.

Witt, Chris. 2009. Quantification of Lewis’s woodpecker habitat 

using Forest Inventory and Analysis data. In: McWilliams, Will; 

Moisen, Gretchen; Czaplewski, Ray, comps. 2008 Forest Inventory and Analysis (FIA) symposium; October 21-23, 2008; Park 

City, UT. Proc. RMRS-P-56CD. Fort Collins, CO: U.S. Department of Agriculture, Forest Service, Rocky Mountain Research 

Station. 1 CDROM.


Federal Recycling Program Printed on Recycled Paper

Rocky

 Mountain

 Research Station

The Rocky Mountain Research Station develops scientific information 

and technology to improve management, protection, and use of the 

forests and rangelands. Research is designed to meet the needs of 

the National Forest managers, Federal and State agencies, public and 

private organizations, academic institutions, industry, and individuals. 

Studies accelerate solutions to problems involving ecosystems, range, 

forests, water, recreation, fire, resource inventory, land reclamation, 

community sustainability, forest engineering technology, multiple use 

economics, wildlife and fish habitat, and forest insects and diseases. 

Studies are conducted cooperatively, and applications may be found 

worldwide.

Station Headquarters

Rocky Mountain Research Station

240 W Prospect Road

Fort Collins, CO 80526

(970) 498-1100

Research Locations

Reno, Nevada

Albuquerque, New Mexico

Rapid City, South Dakota

Logan, Utah

Ogden, Utah

Provo, Utah

The U.S. Department of Agriculture (USDA) prohibits discrimination in all its 

programs and activities on the basis of race, color, national origin, age, disability, 

and where applicable, sex, marital status, familial status, parental status, religion, 

sexual orientation, genetic information, political beliefs, reprisal, or because all 

or part of an individual’s income is derived from any public assistance program. 

(Not all prohibited bases apply to all programs.) Persons with disabilities who 

require alternative means for communication of program information (Braille, 

large print, audiotape, etc.) should contact USDA’s TARGET Center at (202) 

720-2600 (voice and TDD). To file a complaint of discrimination, write to USDA, 

Director, Office of Civil Rights, 1400 Independence Avenue, S.W., Washington, 

DC 20250-9410, or call (800) 795-3272 (voice) or (202) 720-6382 (TDD). USDA 

is an equal opportunity provider and employer.

Flagstaff, Arizona

Fort Collins, Colorado

Boise, Idaho

Moscow, Idaho

Bozeman, Montana

Missoula, Montana