Mitigating Budget Constraints 

on Visitation Volume Surveys: 

The Case of U.S. National Forests

Ashley E. Askew, Donald B.K. English, Stanley J. Zarnoch, 

Neelam C. Poudyal, and J.M. Bowker

Forest Service

Research & Development

Southern Research Station

General Technical Report SRS-192

United States Department of Agriculture
May 2014

Southern Research Station

200 W.T. Weaver Blvd.

Asheville, NC 28804

www.srs.fs.usda.gov

The Authors:

Ashley E. Askew is a Postdoctoral Research Associate, 

University of Georgia, Warnell School of Forestry and Natural 

Resources, Athens, GA 30602; Donald B.K. English is a 

Program Manager, U.S. Department of Agriculture Forest Service, 

Recreation, Heritage & Wilderness Programs, National Visitor 

Use Monitoring Program, Washington, DC 20250; Stanley J. 

Zarnoch is a Mathematical Statistician, U.S. Department of 

Agriculture Forest Service, Southern Research Station, Clemson, 

SC 29634; Neelam C. Poudyal is anAssistant Professor, 

University of Tennessee, Department of Forestry, Wildlife, and 

Fisheries, Knoxville, TN 37996; and J.M. Bowker is a Research 

Social Scientist, U.S. Department of Agriculture Forest Service, 

Southern Research Station, Athens, GA 30602.

Cover photo

Bottomland hardwoods in eastern North Carolina. (Photo 

by Paul Bolstad, University of Minnesota, Bugwood.org)

Product Disclaimer

The use of trade or firm names in this publication is for 

reader information and does not imply endorsement by the 

U.S. Department of Agriculture of any product or service.
Mitigating Budget Constraints 

on Visitation Volume Surveys: 

The Case of U.S. National Forests

Ashley E. Askew, Donald B.K. English, Stanley J. Zarnoch, 

Neelam C. Poudyal, and J.M. Bowker

The Case of U.S. National Forests 1

abstract

Stratified random sampling (SRS) provides a scientifically based 

estimate of a population comprising mutually exclusive, homogenous 

subgroups. In the National Visitor Use Monitoring (NVUM) 

program, SRS is used to estimate recreation visitation and visitor 

characteristics across activities on National forests. However, 

with rising costs and declining budgets, carrying out an annually 

established SRS poses challenges in how and where to reduce 

sampling while maintaining statistical precision. Furthermore, any 

reductions must produce results that validly compare to previous 

years’ SRS in trend analyses. 

Accurate estimates are necessary to manage resources for current 

and future demand; we explore this need through simulations and 

describe a methodology which can be generalized to any SRSbased process facing budgetary challenges. In this research, recent 

historic NVUM responses serve as groundwork for simulating 

various reduction scenarios. The ultimate goal is a manageable set of 

strategies from which to systematically assign a suitable reduction 

scenario for each sampling unit (i.e., forest). Using baseline historic 

data, we determine the sampling error variability by reduction 

scenario and experiment with various sets of three or four predefined strategies to identify viable reduction candidates. A desirable 

candidate is one that achieves acceptable error bounds (based on 

historic values) and reductions overall while meeting specified 

budget constraints (e.g., 20 percent overall fewer sampling days).

Keywords: Budget, National Visitor Use Monitoring, precision, 

sampling, scenarios, simulations, strata. 

INTRODUCTION 

The sample is a valuable scientific tool in estimating 

features of a population of interest because it may be 

impractical or even unnecessary to carry out a complete 

census. A population may be too expansive to evaluate 

within a timeframe or budget. In another case, a population 

may comprise homogeneous but distinct subunits which 

can be sampled in smaller numbers while maintaining 

a high level of statistical precision. Re-sampling on a 

recurring periodic basis produces estimates which may 

be analyzed for trends over time and space, especially 

when the samples are part of a nationwide data collection 

system with scientific foundations and trained employees 

collecting the data. There can be fiscal and administrative 

challenges, particularly when the budget is nominally static 

through time but sampling costs rise. 

This research is motivated by the objective to maintain 

the same level of statistical precision while adjusting for 

budget constraints. Our particular application is to the 

National Visitor Use Monitoring (NVUM) program for 

U.S. National forests. However, the approach is applicable 

to a wider array of stratified random sampling designs. 

Though there are multiple approaches in meeting budget 

goals through cutbacks, we examine various strategies in 

reducing the number of sampling days.

The NVUM program uses a survey-based approach in 

estimating the volume and characteristics of recreation 

visitation to units within the National Forest System 

(Zarnoch and others 2011b). The goals include evaluating 

the number of visits and the mix of activities in which 

they engage on National forests, as well as describing 

recreationists in terms of demographics, satisfaction, 

travel patterns, spending patterns, and visit duration. 

Understanding recreation at National forests in terms of 

visits and activity participation across space and time is 

essential for long-range resource planning, and it allows for 

more efficient policymaking and proactive management of 

resources. 

In this research, we simulate various schematics for 

reducing the total number of sampling days. The goal is 

to accommodate budgetary reductions by reducing the 

volume of survey effort by 20 percent, while preserving 

the statistical validity of visitation estimates. Further, since 

it is not likely that a single strategy will be applicable to 

all National forests, we seek to develop a manageable set 

of reduction strategies so that one can be systematically 

assigned to each national forest in the future. Our context 

is the NVUM survey framework for estimating visitation 

volume, but simulating reductions as described here can 

be generalized to stratified random sampling designs with 

historic observations as a testing ground for comparing 

the accuracy of reduced samples. The methodology 

presented here allows for experimenting with sampling 

rate reductions at different levels according to some 

property of the stratum. The end product is a reduction 

strategy which has been tested on recent data as opposed 

to making completely ad hoc cuts in the field which could 

Mitigating Budget Constraints on Visitation Volume 

Surveys: The Case of U.S. National Forests

Ashley E. Askew, Donald B.K. English, Stanley J. Zarnoch, Neelam C. Poudyal, and J.M. Bowker
2 Mitigating Budget Constraints on Visitation Volume Surveys

disrupt a trend analysis. After establishing the background 

and motivation in further detail for this application to the 

NVUM, we discuss the results and future implications of 

this work.

BACKGROUND: SAMPLING 

STRATEGIES AND THE NVUM 

SAMPLING PROGRAM

The NVUM program, initiated in 1998 and still evolving, 

utilizes a stratified random sampling design for estimating 

and characterizing visitor volume (Cochran 1977). In 

general, a stratified sampling approach is appropriate when 

the population can be divided into subgroups known as 

strata, which are homogeneous and mutually exclusive. 

Together, the members of each stratum make up a subpopulation with its own distinguishing characteristics. If 

a single sampling method is not well-suited for all strata, 

then stratification allows for varying sampling methods 

by subgroup (Cochran 1977). For instance, if a stratum 

has a low and variable volume of participation, it would 

not be advisable to use a sampling method that works well 

for a large and steady inflow of participants. Furthermore, 

stratification is well-suited for studies that involve surveys 

administered by an agency with field offices responsible for 

surveying a portion of a population (Cochran 1977).

The step of stratification maximizes variability between 

strata but minimizes variability within each stratum. 

The resultant estimators have reduced variance (greater 

precision) and improved accuracy (de Vries 1986, Gregoire 

and Buhyoff 1999). Due to the estimators’ precision 

properties, we can use the individual strata estimates 

to produce a precise estimate for the overall population 

(Cochran 1977). If a simple random sample is carried out 

in each stratum, then the method is known as stratified 

random sampling. For each stratum, each member has an 

equally probable chance of being selected.

For a population consisting of L strata, suppose that there 

are N1

, N2, ..., NL members, so that the total population size 

is N = N1 + N2 + ... + NL and samples of size n1, n2, ..., nL are 

taken for each stratum. An important question in stratified 

sampling is how to allocate the sample sizes for each 

stratum, operating under the assumption that the subgroups 

have already been delineated. The appropriate allocation 

will maximize the precision of the estimators. A strategy 

known as the Neyman allocation provides a convenient 

representation for the optimal size ni (i = 1, ..., L) to sample 

for maximizing precision based on the size of the population 

in stratum i and the true variance of the variable measured/

calculated on the stratum (Cochran 1977). However, this 

formula assumes that the population size in stratum i is 

known to be Ni, which is not the case with the NVUM 

process—in fact, it is a component of the very estimates we 

are trying to obtain. Thus, a methodology used in optimal 

allocation of sample sizes for SRS does not apply to the 

NVUM sampling reduction problem.

As a result of collaborative research efforts, NVUM data are 

collected in a systematic fashion across all National forests, 

and the subsequent visitation estimates obtained from 

surveys are within a satisfactory level of statistical accuracy. 

The sampling procedure does not treat the exiting volume 

of visits in the population of site days (the combination of 

a given site and a calendar day) across the National forests 

as uniform. Rather, it accounts for the inherent variation 

imposed by site-type and expected exiting volume via a 

stratified random sampling methodology. 

The two measures of visitation used in NVUM are 

recreation site visits (SV) and national forest visits (NFV) 

(Zarnoch and others 2011b). Each NFV comprises one or 

more SV. SV is estimated as a combination of traffic counts 

on sampled site-days, the average number of people per 

vehicle, and the proportion of exiting traffic that is lastexiting recreationists (LER) (Zarnoch and others 2011b). 

An LER is an individual who has visited a national forest 

site or area for recreation and is concluding the visit by 

exiting the site for the last time for that day. Visit durations 

can vary, being as short as a few minutes or as long as 

several days; in addition, the individual can engage in one 

or more recreation activities on lands and/or waters in the 

national forest (at a single site or at multiple locations). 

NFV is estimated by dividing SV by the average number 

of sites visited per NFV. The simulations developed in 

the current analysis use statistics based on SV and NFV 

to observe which sampling reductions are most effective. 

Though not necessarily globally optimal, they are at 

least practical and computable, as we later discuss in the 

analysis.

The NVUM process identifies four site-types on each 

forest: day-use developed sites (DUDS), overnight-use 

developed sites (OUDS), undeveloped general forest areas 

(GFA), and designated wilderness 1 (WILD). The sites and 

areas in these four site-types make up the spatial dimension 

of the population to be sampled. The estimates from Federal 

fiscal year (FY) 2007 to 2011 indicate that the majority 

of recreation within National forests occurs in the GFA 

portions; exiting survey data suggest that 52 percent of SV 

and 63 percent of NFV were in the GFA portion of forests.

Another important identifier or aspect of sites is whether 

they are proxy or nonproxy. On a proxy site, recreation 

visitation may be assessed using data that characterize all 

users of the site, are exact counts rather than estimates, 

1 Wilderness includes lands and waters within the National Wilderness 

Preservation System; it does not include Wilderness Study Areas, 

Research Natural Areas, or other roadless areas.
The Case of U.S. National Forests 3

and are of an NVUM-approved type, such as fee receipts, 

fee envelopes, mandatory permits, permanent traffic 

counters, and ticket sales (English and others 2002). A 

much smaller portion of sampling occurs at these sites 

with visitation proxy data, which more accurately reflect 

the recreation traffic and do so over a period of months. 

This property reduces the variation of visitation estimates 

(Zarnoch and others 2011b). Proxy sites do not require use 

of a traffic counter, but some sampling of recreationists 

is needed to estimate conversion factors from the data 

(e.g., fee envelopes) to the relevant SV or NFV measure. 

Thus, interviews are carried out in a 6-hour period of 

daylight which maximizes the opportunities to survey 

LERs. Further details on how these data are incorporated 

into calculations of visitation estimates are available 

in publications devoted to or employing the NVUM 

methodology, such as in English and others (2002), White 

and others (2007), or Zarnoch and others (2011a).

A site which does not meet the above criteria is a nonproxy 

type, in which case estimates utilize 24-hour traffic counts 

as a measure of use for one sampled site day. Compared 

to nonproxy sites, a proxy site requires lower levels of 

sampling. For the present simulation exercise, it is important 

to note that proxy sites have very low costs for sampling 

and thus are not included in these exercises and subsequent 

discussions. We consider the challenge of reducing the more 

expensive sampling of nonproxy site days.

At each site, every day of the year is characterized by an 

expected exiting volume of visitation: No use (N), Low (L), 

Medium (M), High (H), and Very High (VH). These daily 

exiting volume levels define the temporal dimension of the 

population. For nonproxy sites, the combination of site-type 

and use-level defines a sampling stratum: for example, a 

Medium use day for a day-use developed site (DUDS) is 

denoted as DUDS-M. After dividing the sample space into 

these 4×4=16 strata with visitation, the stratified random 

sampling procedure conducts a simple random sample on 

each stratum (de Vries 1986) from the selection of site days 

(the combination of site-type and use-level). If desired, 

estimators may be determined for each of the strata, or 

forests, but the overall estimate of visitation on all National 

forests is the primary goal. Standard NVUM procedure 

prescribes no more than 50 proxy site days and appropriate 

sampling of nonproxy site days subdivided by site-type 

and use-level. The latter is dependent upon a minimum of 

8 site days (or fewer, if 8 are not available), computations 

from the site’s previous year’s estimates, and weighting by 

importance of strata.

A survey round is completed over the course of five years, 

where on-site survey questionnaires and data collection 

occur on approximately 20 percent of all National forests 

each FY, running from 1 October to 30 September. Thus, 

the monitoring process for a given national forest takes 

place on a 5-year cycle. The majority of sampling is carried 

out at sites where the amount of traffic for the 24-hour 

period of the interview day is recorded in conjunction 

with 6 hours of visual calibration of the counter and onsite 

interviews to estimate the proportion of those exiting 

who are LER. For this portion of the sample, the traffic 

tally from the counters is recorded, and surveys are 

administered by personnel during a randomly designated 

time period, chosen out of two possible time slots ideal for 

encountering exiting recreationists. 

The NVUM survey form consists of a core set of questions 

asked of all eligible, randomly chosen respondents; to 

qualify, the respondent must be at least 16 years of age 

and must be exiting the site or area. If multiple individuals 

are in a vehicle selected for the interview, the person who 

is at least 16 years of age with the most recent birthday 

is designated the primary respondent. The first set of 

questions determines what proportion of traffic is there 

for recreation versus other purposes, and what proportion 

are LER. For non-LERs, the survey terminates; for LERs, 

additional information includes demographics, activity 

participation, and duration. Approximately one-third are 

also asked questions related to the levels of satisfaction 

with the conditions and access of resources, as well as the 

perception of crowding; another one-third are asked about 

their economic spending during the trip. Responses provide 

insight into contribution of tourism to the local economy 

and satisfaction of recreationists, as well as identifying the 

primary recreation clients in terms of gender, age, ethnicity, 

and other socioeconomic characteristics.

CONSIDERATIONS AND CHALLENGES 

FOR A MODIFIED SAMPLE PROGRAM

Balancing survey administration and tighter budget 

constraints entails several challenges: (1) determining the 

mix of sampling rates across the strata; (2) spatial and 

temporal distribution of sample days to ensure adequately 

representing recreationists by stratum while minimizing 

loss of precision effected by a decrease in sampling days; 

and (3) meeting budget constraints while maximizing 

information obtained. As results are based on traffic 

counts and survey responses obtained on the site-days 

chosen for field sampling, two important considerations 

arise: one is statistical in nature and the other fiscal. First, 

there is the issue of how to best distribute the sampling 

days for forests across the 16 strata while maintaining 

accuracy of estimators. To achieve an accurate estimation, 

a forest or stratum that has high variability in visitation 

may require more sampling days versus one that has 

a consistent exit volume of recreationists. Second, the 

NVUM process must adapt to a static nominal dollar 

budget despite rising in-field costs, thereby necessitating 
4 Mitigating Budget Constraints on Visitation Volume Surveys

approximately 20-percent reduction in sampling days over 

all national forest sampling days.

A day of field data collection is currently budgeted at 

around $400, but actual costs average about 20 percent 

higher. The cost covers preparation by personnel, travel 

to and from sites, setup of traffic counters and appropriate 

signage, 6 hours of onsite interviewing, and retrieval of 

traffic counter data. Annual national workload assignments 

average around 5,500 days of sampling for approximately 

24 National forests per FY. The work described here is 

motivated by the need to reduce sampling days to remain 

within budget constraints, fully fund the fieldwork, and 

attempt to maintain the current confidence levels for 

visitation estimates. Another constraint is to allocate 

reductions so that each forest has some cutback in sampling 

days but none exceeds roughly 25 percent (the highest 

threshold we have set that allows for some flexibility with 

the goal of 20 percent but not excessively so).

Simulation Study

The FY 2011 data from 29 National forests, including 

some forest subunits that were sampled as independent 

units, were used as a baseline for simulating sample size 

reductions which meet budgetary constraints and statistical 

accuracy goals. The NVUM sample size allocation from 

the FY 2011 survey must be reduced in an appropriate 

manner to meet these objectives. Though the simulations 

were based on estimates, the computations were rooted in 

an established scientific methodology and thus were treated 

as reasonably accurate representations of the FY 2011 SV 

and NFV measures. Additionally, the original prework 

of NVUM calls for the use of previous year estimates 

(standard errors) in deciding nonproxy site day allocations 

by stratum; this dependency upon the recent historical 

survey was satisfied in the use of FY 2011 estimates as 

recent historical sample data. (The list of forests/subunits 

and days of FY 2011 sampling for each is found in the 

appendix.) We are able to select an appropriate sample size 

reduction by evaluating the properties of the SV and NFV 

estimators under various sample size reduction scenarios 

and comparing them against the estimators from the 

complete FY 2011 sample.

For a set of sampling units, we may achieve a most 

accurate estimate (under the necessary constraint of 

reduction) by tailoring the reductions to each forest. For 

29 National forests, this may produce at most 29 different 

reduction strategies. This decisionmaking by individual 

forests is an undesirable sampling reduction strategy: it 

is cumbersome, time-consuming, and non-systematic. 

Though it may produce more accurate estimators relative 

to the historic estimates, the costs make it impractical. 

A more general, practical approach is to maintain a 

reasonable set of scenarios and to assign each forest an 

appropriate reduction strategy from the set. Thus, we 

consider six general sampling scenarios (see table 1 

for descriptions). In Scenario 1, reductions are carried 

out equally across all strata, regardless of site-type and 

use-level. Scenario 2 reduces the sampling equally for 

the DUDS and OUDS strata, while Scenario 3 does that 

for the GFA and WILD strata. The reductions in sample 

size for Scenario 4 affect only strata based on the GFA 

Table 1—Sampling strategies by site-type and use-level for simulation scenario

Scenario Description of sampling rate modifications for simulation

Scenario 1 Reduce DUDS, OUDS, GFA, and WILD simultaneously from 100% to 70% in 

increments of 10%.

Scenario 2 Reduce DUDS and OUDS simultaneously from 100% to 70% in increments of 

10%; maintain 100% for GFA and WILD.

Scenario 3 Maintain 100% for DUDS and OUDS; reduce GFA and WILD simultaneously 

from 100% to 70% in increments of 10%.

Scenario 4 Maintain 100% for DUDS, OUDS, and WILD; reduce from 100% to 70% in 

increments of 10% for GFA.

Scenario 5 Reduce DUDS, OUDS, GFA, and WILD simultaneously from 100% to 10% in 

increments of 10% for use-level L only; maintain 100% otherwise.

Scenario 6 Reduce DUDS, OUDS, GFA, and WILD simultaneously from 100% to 25% in 

increments of 25% for use-level H and VH only; maintain 100% otherwise.

Site-types: DUDS = day-use developed site; OUDS = overnight-use developed site; GFA = undeveloped general 

forest area; WILD = designated wilderness.

Use-level: L=low; H=high; VH=very high.
The Case of U.S. National Forests 5

site-type. In Scenarios 5 and 6, reductions are based on 

use-level: only use-level low (L) in 5 and only high or 

very high (H or VH) in 6. Within each scenario there are 

subscenarios that produce different levels of reductions. 

An example subscenario would be Scenario 5 with 40 

percent sampling (60 percent sampling reduction) for strata 

DUDS-L, OUDS-L, GFA-L, and WILD-L; another would 

be Scenario 1 with 80 percent sampling from all strata (20 

percent sampling reduction). 

The sampling cost savings are assumed to be a linear 

function of the overall reduction in on-site interview days. 

For example, if a certain strategy produces 5 percent fewer 

sampling days overall than another, then the total cost will 

be 5 percent lower. From a cost standpoint, it is irrelevant 

whether the fewer days come from a reduction in only the 

use-level L or are spread across all strata. What is not clear 

is how the performance of the visitation estimators relates 

to a cutback in a certain site-type or use-level; we explore 

this challenging matter through simulations. 

We simulate sampling reductions based on the scenario/

subscenarios from table 1 using PROC SURVEYSELECT 

(SAS Institute Inc. 2009) to repeatedly sample from the 

actual FY 2011 survey data. However, we recognize the 

possibility of a scenario/subscenario diminishing the 

number of sampling days so low that it is no longer useful 

for estimation. To avoid that result, we impose a minimum 

requirement of at least 10 sample days per site-type uselevel stratum in order to represent members from each 

stratum. If a stratum has fewer than 10 sample days to 

start, then no reduction is allowed. All reductions occur 

in strata which can afford some loss in membership. If 

the initial stratum size is > 10 but the reduction would 

produce a count < 10, the reduction was forced to stop at 

a sample of size 10. In preliminary work, we determined 

that a minimum of five sampling days per stratum was 

insufficient; this resulted in underrepresentation in multiple 

strata. Setting the threshold too high can preclude some 

useful reduction strategies. However, care must be taken 

with PROC SURVEYSELECT to ensure that all strata are 

included in the sampling, specifically when the full stratum 

size is < 10. Without the appropriate programming steps in 

SAS, PROC SURVEYSELECT excludes strata with counts 

less than the minimum. 

We carried out 100 simple random sample replicates of site 

days without replacement using the appropriate sampling 

rates for each stratum, given by scenario/subscenario. For 

each forest i and sampling scenario, we compute the 

average visitation estimates ( , NFVi ), standard errors 

[ , ], and coefficients of variation 

[ , ] for SV and NFV, respectively, across 

the 100 replicates. To create a ratio of the statistic to the 

“baseline” value, i.e., the one from the complete sample 

(such as the SV estimate denoted by ), we compute 

for the 29 forests as follows:

(1)

(2)

(3)

(4)

All of the ratios are indexed at 100 relative to the baseline, 

essentially becoming a percentage of error relative to the 

full set. Averaging over 100 replicates avoids the possibility 

of basing all inference on a single or few samples that 

may fail to adequately represent the population. For any 

of these ratio statistics, a value > 100 indicates that the 

reduced sample exceeds the full-sample estimate; a ratio 

< 100 means that the reduced-sample estimate is less 

than the baseline value. For example, an indexed ratio of 

107 for equation (1) indicates that the reduced sample has 

exceeded the full sample SE of SV by 7 percent. From this 

we have a convenient measure for assessing relative error 

and designating a tolerance for error, or the maximum 

absolute percentage deviation permitted for further 

consideration. If the tolerance is set at 10 percent, then for 

an interval centered at indexed statistics of 100, we will 

accept sampling schemes which produce estimates between 

90 and 110. Preliminary analysis shows the means to be 

consistent, as the SV and NFV estimators are unbiased 

for any of the reduced sample sizes. The sums of squared 

errors (SSEs) provide guidance as to which scenarios 

are most desirable. The SSEs are based on the indexed 

statistics created by averaging over all 100 replicates; we 

sum over all forests the squares of the difference between 

the indexed statistics for the reduced sample (I

i,j) and the 

full set (Fi,j), or

i

(5)

where I

ij is equation (1) or (2) or (3) or (4) for forest i, that is, 

 Iij = or 

 or or 

The SSEs are computed for six statistics (j = 1, ..., 6, 

respectively): NFV mean, NFV standard error, NFV 

coefficient of variation, SV mean, SV standard error, 

and SV coefficient of variation. By using the SSEs, we 
6 Mitigating Budget Constraints on Visitation Volume Surveys

aggregate individual deviations relative to the full sample 

and provide a sense of the overall performance of the 

estimators. Thus, SSEs are a guideline for judging how 

well the estimators perform under a sampling scenario/

subscenario. The goal is to select those that produce lower 

values. If there is a large discrepancy between the statistics 

computed for the full and reduced samples, then we will 

see a larger contribution to the SSE. Though the summation 

alone does not determine which forests contribute the 

most or least error, we can ascertain a desirable sampling 

reduction (using the low-SSE sampling criterion) by 

observing a small overall SSE. 

We assess the performance of the scenario/subscenarios 

according to four criteria: (1) the ability to reduce an 

individual forest’s sampling days by 15 to 25 percent; (2) 

SV and NFV statistics (standard error and coefficient of 

variation) within a specified tolerance of error from the 

baseline value; (3) lower SSEs for the mean, SE, and CV 

of SV and NFV; and (4) overall reduction across all forests 

amounting to approximately 20 percent to satisfy the budget 

constraint. If no candidates reduce by 15 to 25 percent, we 

then consider those that reduce by less than 15 percent while 

remaining within the tolerance limits for error. 

SV and NFV estimates exist for the full FY 2011 data 

set for each forest. The reduced samples from 100 SRS 

provide a new estimate of SV and NFV for each scenario/

subscenario. The simulations allow us to observe the 

effects of various scenario/subscenario reductions, using 

the computed statistics for gauging accuracy. We examined 

a tolerance of error from the baseline ranging from 10 

percent down to 5 percent in 1-percent increments. With 

an 8-percent error tolerance, it was not possible to achieve 

a 20-percent reduction overall. Since the more accurate 

estimates are typically affiliated with lower sampling-day 

reductions, we did not consider a tolerance limit below 10 

percent. This meets the sampling reduction needs while 

maintaining a pool of desirable reduction strategies per 

national forest. With a 10-percent tolerance, the indexed 

estimates for consideration must fall between 90 and 110; 

occasionally, it was necessary to consider candidates 

slightly above the interval. 

This project is concerned with both accuracy and 

sample size reduction using the estimates of FY 2011 

as baselines in simulated reduction strategies. In most 

cases, accuracy was a function of sample size reduction; 

less reduction produces a more accurate sample estimate. 

We are seeking a sampling scenario assignment that 

balances accuracy and sample size allocations based on 

the reduction goals. The criteria of sample size reduction 

and membership in the error tolerance interval results 

in a pool of possible subscenario candidates of < 10, 

in most cases. Each forest produces a unique set of 

viable candidates within this range. The most desirable 

candidates for each forest are those whose reduction-ofdays percentage approaches 20 pecent, while remaining 

close to the baseline value on all six statistics. A number 

of subscenarios in table 1 can be readily eliminated, 

specifically those affiliated with reductions over 70 

percent. Since certain subscenarios work for multiple 

forests, the feasibility of a systematic allocation seems 

possible.

Each forest or forest subunit is associated with a set of 

candidate scenario/subscenarios; some of these do not 

respond well to a given forest, a result of characteristics to 

be determined. The following process determines which 

candidates to consider for a given forest:

1) The initial pool consists of candidates producing a 

15 – 25 percent reduction in sample size; occasionally, 

a lack of candidates necessitated consideration of some 

outside this interval.

2) Candidates are filtered by the criterion of being within 

10 percent of the baseline value for the SV and NFV 

statistical measures; occasionally, it was necessary to 

relax this value to slightly over 10 percent in order to 

consider a candidate that achieved greater reduction 

levels. 

3) The best candidates are those that both meet accuracy 

standards and produce the largest reduction in sampling 

days. 

4) The goal of overall 20-percent reduction in sample size 

is assessed after all forests are assigned a subscenario. 

This functions as the final criterion, which further 

narrows the field of candidates.

Triplet and Quadruplet Assignments

It is not likely that a single scenario applied to all 

forests will yield an optimal result in reducing the 

sample size while maintaining accuracy. Each national 

forest is unique on some set of characteristics, such as 

proximity to population centers, geographical influences 

on recreation, and seasonality. The next objective is to 

identify a small subset of subscenarios that will—as a 

set—sufficiently meet the objectives while achieving 

lower sums of squared errors for indexed statistics. That 

is, we seek a small pool of reduction scenarios that can 

be allocated methodically according to some features of 

the national forests, such as visitation volume, size, or 

location. For this, we consider three- or four-scenario 

sets (called triplets and quadruplets). Each forest is 

assigned one of the scenarios in the set. We included 

eight scenarios as possible set candidates, drawn from 

the ones that performed best on the individual forests. 

The eight were: (i) Scenario 1 with 20-percent reduction, 

(ii) Scenario 1 with 30-percent reduction, (iii) Scenario 2 
The Case of U.S. National Forests 7

with 30-percent reduction, (iv) Scenario 4 with 30-percent 

reduction, (v) Scenario 5 with 30-percent reduction, (vi) 

Scenario 5 with 40-percent reduction, (vii) Scenario 5 

with 50-percent reduction, and (viii) Scenario 5 with 

60-percent reduction. Almost all triplets and quadruplets 

formed from this group of eight included at least one 

subscenario that worked well for each forest. We did not 

include subscenarios that worked for only one forest, even 

if that scenario one was the best reduction strategy for 

the forest; we exchange customization for a generalizable 

approach at a slight cost in accuracy.

We created all possible sets of three and four from 

these eight scenarios, for a total of 56 triplets and 70 

quadruplets. For example, one triplet is the set of Scenario 

1 with 20-percent reduction, Scenario 1 with 30-percent 

reduction, and Scenario 5 with 60-percent reduction. 

An exhaustive simulation with unlimited computing 

power could examine all permutations of these three 

scenario/subscenarios for all 29 forests and obtain a best 

assignment. For efficiency, we further restricted the 

analysis to limiting each forest to candidates from the 

triplet which had been shown to work well.

In some triplets and quadruplets, there were clear leading 

candidates for a given national forest; in others, one 

candidate was best for accuracy but another for reduction. 

In the latter case, the simulations often considered both 

candidates just to ensure that both were included among 

the possibilities. In some cases, none of the triplet or 

quadruplet scenario/subscenario combinations worked 

strictly within the desired constraints; therefore, it was 

necessary to consider reducing an individual national 

forest’s sampling days by < 15 percent or slightly > 25 

percent, but never as much as 30 percent. Additionally, 

it was sometimes necessary to relax the 10-percent error 

tolerance to 13 percent in order to find which candidate 

from the triplet or quadruplet worked best. Under the 

relaxed conditions, if a triplet or quadruplet did not result 

in a viable assignment across the 29 National forests, it was 

discarded from consideration. 

There were 21 triplets and 35 quadruplets in the final set 

of assignments that we assessed. We assessed these by 

the ability to produce an overall sample-day percentage 

reduction of 19 – 20 percent and have lower SSEs for the 

statistical measures (mean, standard error, and coefficient 

of variation for SV and NFV). Among the statistical 

measures, the distinguishing feature is the SSEs, 

especially in the case of the indexed SV and NFV means. 

Accuracy is the sum of the variance and the squared bias, 

so a bias that is close to zero implies that accuracy is 

evaluated by the variance.

RESULTS

The simulations examined assignments within 21 triplets 

and 35 quadruplets for the 29 National forests. We used the 

following color code to represent the percentage reductions 

by scenario:

In supporting Excel® worksheets, this code is used to color 

cells to represent the percentage reduction while the text 

in the cell represents the scenario number. For instance, 

the following quadruplet involves Scenario 1 at 20-percent 

reduction, Scenario 1 at 30-percent reduction, Scenario 

4 at 30-percent reduction, and Scenario 5 at 40-percent 

reduction, respectively. 

The top candidates for triplets and quadruplets appear in 

tables 2 and 3, along with the SSEs for the six indexed 

statistics. Here, the triplet or quadruplet ID is the unique 

number used to reference an assignment in a convenient 

manner. The percentage reduction column shows the 

percentage reduction for overall sample size (sampling 

days). The next six columns contain the SSE measures 

for the indexed (at 100) statistics for the SV mean, SV 

standard error, SV coefficient of variation, NFV mean, 

NFV standard error, and NFV coefficient of variation, 

respectively. The SSE for any of the indexed statistics 

provides a measure of variability of the estimate adjusted 

to the baseline, and this may provide additional insight that 

is not available through the original estimate. As the SV 

mean estimate may be unbiased, it is more informative to 

consider how the estimates fluctuate under the scenario/

subscenario by the corresponding SSE. Furthermore, 

the SSEs for the SV, SE, and CV reflect the variability 

of the standard errors and coefficient of variation, which 

are related to the mean by computation. Thus, there is a 

relationship among the three SSEs under consideration that 

provide an overall assessment of the candidate. 

As in tables 2 and 3, the overall sample size percentage 

reduction appears in the 19- to 20-percent range. These 

top candidates were chosen on the basis of balancing the 

10% Reduction (Scenarios 1-5), 25% Reduction (Scenario 6)

20% Reduction (Scenarios 1-5), 50% Reduction (Scenario 6)

30% Reduction (Scenarios 1-5), 75% Reduction (Scenario 6)

40% Reduction (Scenario 5)

50% Reduction (Scenario 5)

60% Reduction (Scenario 5)

70% Reduction (Scenario 5)

80% Reduction (Scenario 5)

90% Reduction (Scenario 5)

1 1 4 5
8 Mitigating Budget Constraints on Visitation Volume Surveys

percentage reduction with the SSEs. First, we filtered 

the pool of combinations for each triplet or quadruplet, 

choosing those with overall reductions of at least 19 

percent. From this reduced set, we manually select the 

candidates which yield lower SSEs in both the SV and NFV 

measures (relative within the statistic over all candidates) 

at varying levels of percentages. We then chose the one 

which balanced the two goals. We also note that from an 

administrative standpoint it is more efficient to manage 

assigning forests to a set of three reduction scenarios than 

to four scenarios, which gives some preference for a triplet 

as the final choice. Adding in more scenarios creates more 

administrative overhead, becoming more cumbersome to 

manage which scenario to assign to each forest.

Figures 1 and 3 illustrate the plots for 50 randomly chosen 

candidates per triplet or quadruplet, respectively, showing 

the overall sampling day percentage reductions against the 

SSEs of the six statistics. Figures 2 and 4 are based on the 

same plot as in figures 1 and 3 but also show the top five 

candidates indicated by the nontransparent points. 

On the prescreened or preselected set of scenario/

subscenarios per triplet or quadruplet, we explore the 

possibilities in one of two ways. First, if the permutations 

from the prescreened triplet or quadruplet are less than 

100,000, then we carry out an exhaustive search of 

the prescreened pool; otherwise, we randomly choose 

100,000 unique assignments. For example, if we have 

determined that 10 forests respond favorably to two 

scenario/subscenarios from a triplet and the remaining 19 

forests to just one, then we will have 1,024 permutations 

to consider. If each forest works well with two scenario/

subscenarios from a triplet, then we have 229 = 536,870,912 

possibilities. In this case, we randomly choose 100,000 

unique assignments from the pool of candidates. This 

value was chosen arbitrarily large in order to explore more 

possibilities; in another study, we might observe the efficacy 

of using a smaller value and thus a lower running time.

We present 50 points in order to show the clusters which 

form by the triplet or quadruplet; otherwise, the point 

density is so great that any useful patterns are obscured. A 

convenient way to view the clusters is the convex hull on 

the set of those permutations we explore and not just on the 

50 points. A two-dimensional convex hull can be thought 

of as a polygon adjoining the outermost points such that a 

line drawn between any two points is contained within the 

set. Figure 5 is an example of the point density (A) forming 

the shape which delineates the convex hull shown in (B). 

The points are from the triplet containing the top candidate, 

which is indicated in red. Figures 6 and 7 are the convex 

hulls for each of the triplets and quadruplets under study, 

respectively. There is a poorly performing triplet in figure 

Table 3—Top quadruplet candidates for 29 forests from simulation study, with selected candidate indicated in 

orange

Quad ID % Sample Size 

Reduction

SSE SV 

Mean

SSE SV 

SE

SSE SV 

CV

SSE NFV

Mean

SSE NFV

SE

SSE NFV 

CV

13 0.1878104 18.93423 1394.297 1217.442 16.48626 1637.501 1485.249

33 0.2013544 23.59944 1971.89 1663.471 20.75768 2202.976 1918.124

55 0.1936795 17.08092 1435.854 1272.793 15.14705 1709.232 1552.95

80 0.1941309 20.25986 1434.657 1284.046 18.66755 1699.362 1572.586

101 0.2002257 23.71797 1560.447 1389.108 20.86359 1911.046 1737.189

SSE = sums of squared errors; SV = site visit; SE = standard error; CV = coefficient of variation; NFV = national forest visit.

Table 2—Top triplet candidates for 29 forests from simulation study, with selected candidate highlighted in orange

Triplet ID % Sample Size 

Reduction

SSE SV 

Mean

SSE SV 

SE

SSE SV 

CV

SSE NFV

Mean

SSE NFV 

SE

SSE NFV 

CV

17 0.1893905 16.17968 1438.769 1256.743 13.86722 1703.125 1536.33

19 0.1914221 20.03314 1435.095 1282.951 18.53394 1708.317 1583.459

32 0.2006772 24.33266 1645.481 1442.783 21.50427 1939.316 1732.364

35 0.1995485 24.13526 1489.247 1322.569 21.2975 1774.063 1610.153

39 0.2011287 24.52816 1940.505 1591.717 21.59005 2116.518 1841.076

SSE = sums of squared errors; SV = site visit; SE = standard error; CV = coefficient of variation; NFV = national forest visit.
The Case of U.S. National Forests 9

Figure 1—Plot of sums of squared errors (SSEs) for six indexed statistics computed for 50 randomly chosen members of 21 triplets 

under study. For convenience, the legend displays the triplets as a sequence of three numbers, such as 13_14_24. The first digit of 

each of the three numbers symbolizes the scenario, and the second the level of reduction (i.e., 1 = no reduction, 2 = one increment 

reduction, 3 = two increment reduction, and so on, where increments are in 10 percent for Scenarios 1 to 5 and 25 percent for 

Scenario 6). Thus, 13_14_24 is the triplet with Scenario 1 at 20-percent reduction, Scenario 1 at 30-percent reduction, and Scenario 2 

at 30-percent reduction. 
10 Mitigating Budget Constraints on Visitation Volume Surveys

Figure 2—Plots of 50 randomly chosen members of the 21 triplets under study showing the relative position of the top five triplets: 17, 

19, 32, 35, and 39. For convenience, the legend displays the triplets as a sequence of three numbers, such as 13_14_24. The first digit 

of each of the three numbers symbolizes the scenario, and the second the level of reduction (i.e., 1 = no reduction, 2 = one increment 

reduction, 3 = two increment reduction, and so on, where increments are in 10 percent for Scenarios 1 to 5 and 25 percent for 

Scenario 6). Thus, 13_14_24 is the triplet with Scenario 1 at 20-percent reduction, Scenario 1 at 30-percent reduction, and Scenario 2 

at 30-percent reduction.
The Case of U.S. National Forests 11

Figure 3—Plot of sums of squared errors (SSEs) for six indexed statistics computed for 50 randomly chosen members of 35 

quadruplets under study. For convenience, the legend displays the quadruplets as a sequence of four numbers, such as 13_14_24_44. 

The first digit of each of the four numbers symbolizes the scenario, and the second the level of reduction (i.e., 1 = no reduction, 2 = one 

increment reduction, 3 = two increment reduction, and so on, where increments are in 10 percent for Scenarios 1 to 5 and 25 percent 

for Scenario 6). Thus, 13_14_24_44 is the quadruplet with Scenario 1 at 20-percent reduction, Scenario 1 at 30-percent reduction, 

Scenario 2 at 30-percent reduction, and Scenario 4 at 30-percent reduction.
12 Mitigating Budget Constraints on Visitation Volume Surveys

Figure 4—Plots of 50 randomly chosen members of the 35 quadruplets under study showing the relative position of the top five 

quadruplets: 13, 33, 55, 80, and 101. For convenience, the legend displays the quadruplets as a sequence of four numbers, such as 

13_14_24_44. The first digit of each of the four numbers symbolizes the scenario, and the second the level of reduction (i.e., 1 = no 

reduction, 2 = one increment reduction, 3 = two increment reduction, and so on, where increments are in 10 percent for Scenarios 

1 to 5 and 25 percent for Scenario 6). Thus, 13_14_24_44 is the quadruplet with Scenario 1 at 20-percent reduction, Scenario 1 at 

30-percent reduction, Scenario 2 at 30-percent reduction, and Scenario 4 at 30-percent reduction.
The Case of U.S. National Forests 13

Figure 5—(A) Panel showing the plots of all 100,000 points for the SSEs computed on the indexed SE SV statistic for Triplet 13_14_57, 

the triplet from which the leading candidate 35 comes. The top candidate appears as a red point in both panels, and the outlined 

polygon represents the convex hull generated by the points. (B) Panel simplifying the plot and showing only the convex hull and the top 

candidate’s relative position.
14 Mitigating Budget Constraints on Visitation Volume Surveys

Figure 6— If a triplet has fewer than 100,000 candidates, then we exhaust all permutations. Otherwise, we take a random selection 

of 100,000. All points generate a two-dimensional convex hull per statistic, which gives a sense of the coverage of the corresponding 

triplet. In these plots, there are some triplets which perform very well in terms of SSE, but the levels of reductions do not meet the goal 

of around 20 percent. Likewise, there is a very poorly performing triplet, as shown in figure 1 to be 13_56_57, which includes Scenario 

1 at 20-percent reduction and Scenario 5 at high levels of reduction (50 and 60 percent). The top candidates are those which produce 

around 20-percent reductions overall while achieving desirable levels of SSEs relative to the others.
The Case of U.S. National Forests 15

Figure 7— If a quadruplet has fewer than 100,000 candidates, then we exhaust all permutations. Otherwise, we take a random 

selection of 100,000. All points generate a two-dimensional convex hull per statistic, which gives a sense of the coverage of the 

corresponding quadruplet. In these plots, there are some quadruplets which perform very well in terms of SSE, but the levels of 

reductions do not meet the goal of around 20 percent. The poorly performing triplet from figures 1 and 2 disappears because of the 

inclusion of an additional scenario/subscenario from which to choose.
16 Mitigating Budget Constraints on Visitation Volume Surveys

6, with a low percentage reduction and high SSEs on the SE 

and CV statistics. Conversely, most of the quadruplets do 

not show a similar pattern; instead, there is a tendency for 

the SSEs to go up with increasing percentage reductions.

Each assignment involves some permutation of the 

preselected scenario/subscenarios per forest (i.e., not the 

entire pool of exhaustive permutations of the triplet or 

quadruplet). Some candidates may produce a small SSE on 

one measure, but have a larger one on another. Since the 

budget implications are essentially the same, if candidates 

produce lower SSEs at 19-percent reduction, then we will 

opt for the improved accuracy at the cost of a percentage 

point lower reduction. The top five candidates chosen from 

all triplets and quadruplets would be those corresponding 

to triplets 17, 19, 32, 35, and 39 and quadruplets 13, 33, 55, 

80, and 101.

Among the top three triplets, the SSEs for the means are 

roughly equivalent; the SSEs for the SE and CV measures 

are relatively low for triplet 35, which also achieves the 

reduction goal (19.95 percent). Quadruplet 55 was chosen 

due to the low SSEs across all of the estimators compared 

to all other combinations. In some cases, quadruplets 

with minimum SSEs for a given NFV statistic resulted in 

a less accurate SV estimator. The choices reflect which 

quadruplet performs well overall, not just on an individual 

statistic basis. Quadruplet 55 also produces a near-goal 

reduction overall (19.4 percent).

Triplets 32, 35, and 39 use the same set of scenario/

subscenarios but with differing assignments to National 

forests. The other triplets differ by one scenario/subscenario. 

The top five candidates for the quadruplets have in common 

Scenario 1 at 20-percent and 30-percent reduction, which 

is also common to all the top triplet candidates. The 

overlapping of the scenario/subscenarios seen across the 

triplets and quadruplets lends a measure of confidence 

that we have identified a set that is among the fittest of the 

practical strategies. The question becomes which of the 

nonoverlapping sets best groups the forests by features 

which lends to a fast classification for future application. 

The groupings produced by these top four candidates 

across triplets and quadruplets are given in tables 4 and 5. 

In the triplet and quadruplets, there is a tendency to 

have a large group with two and three smaller groups, 

respectively. Triplet 35 (table 4) has a largest group, 

followed by a second-largest group, and a group with only 

two members. The subunits of a given national forest 

(say, A02012) may belong to separate categories, which is 

not unexpected given the differing levels of variability of 

sites within National forests. Quadruplet 55 (table 5) has 

a marked distinction between the largest group and three 

smaller groupings of decreasing size. 

Table 4—Assignment of scenario/subscenarios from 

Triplet 35 to 29 forests from simulation study

Triplet 35

1 1 5

U0401702 A01005 U0201201

A05001 A01016 U0201202

A01017 A02013

U0300301 U0300302

A03006 U0401701

A04010 U0800501

A05004 U0800502

A05009 U0800503

A05013 A08016

A06003 A09013

A06006

A06012

A06022

A08008

A09003

A09009

A10105

Table 5—Assignment of scenario/subscenarios from 

Quadruplet 55 to 29 forests from simulation study

Quadruplet 55

1 1 4 5

U0401702 A01005 U0800501 U0300302

A05001 A01016 U0800502 U0401701

A05009 A01017 A08016

A06022 U0201201

U0800503 U0201202

A02013

U0300301

A03006

A04010

A05004

A05013

A06003

A06006

A06012

A08008

A09003

A09009

A09013

A10105
The Case of U.S. National Forests 17

Evaluation

Looking at the top two candidates (Triplet 35 and 

Quadruplet 55), we now examine the behaviors of the 

indexed statistics for each of the 29 National forests (see 

figs. 8 and 9). This allows us to evaluate each subunit 

individually, as certain subunits could perform poorly even 

though the candidate performs well overall. The horizontal 

lines are the means of the statistics taken over all 29 forests. 

The indexed SV and NFV means are accurate for the most 

part over all the forests, partly due to the unbiased nature of 

the statistic. Nevertheless, the indexed standard errors and 

coefficients of variation reveal a more interesting story. The 

closer the index is to 100, the more accurate the statistic is 

relative to the full sample. In most cases, the standard 

errors and coefficients of variation were greater than the 

baseline, mainly due to reduction of sample size. The top 

candidates show how the individual percentage reductions 

behave on an individual forest basis. In some cases, it 

is necessary to make smaller reductions on the forests 

because they cannot afford to lose more sampling days.

Triplet 35 has the advantage of one less scenario/

subscenario to maintain, and the results are also 

satisfactory with respect to the SSEs and percentage 

reductions. Of these final candidates, Triplet 35 offers both 

simplicity (one less scenario/subscenario to maintain) and 

desirable properties in the statistics, specifically SSEs 

which are comparable to the top three quadruplets.
18 Mitigating Budget Constraints on Visitation Volume Surveys

Figure 8—Plots of individual values of six indexed statistics by forest for the top five triplet candidates: 7, 19, 32, 35, and 39.
The Case of U.S. National Forests 19

Figure 9—Plots of individual values of six indexed statistics by forest for the top five quadruplet candidates: 13, 33, 55, 80, and 101.
20 Mitigating Budget Constraints on Visitation Volume Surveys

SUMMARY AND CONCLUSIONS

With any survey procedure undergoing reductions in 

sampling days, it is prudent to conduct a preliminary 

statistical investigation before imposing ill-informed 

cutbacks in the field that could possibly produce inferior 

estimators. A survey methodology as complex as the 

NVUM system requires careful and strategic planning in 

reductions so as not to reduce sampling days vital to strata 

for each national forest. Such investigations are relegated 

to the simulation process described in this work using 

historic but recent data. The values under the full sample 

are treated as the true baseline values, and the reductions 

provide estimates which may be gauged against this in the 

form of indexed statistics. In the end, it is the SSEs of the 

indexed statistics and the goal percentage reductions which 

provide the best method for selecting candidates.

For the NVUM program, sampling and surveys are used to 

obtain adequate estimation of visitation because a complete 

census would be impractical and unrealistic. Now, facing 

nominally flat budgets, even the surveying procedure 

is in need of reduction. The most statistically accurate 

approach would be to lower the number of sampling days 

on those National forests that can still effectively provide 

estimates. However, having such customized reduction 

scenarios for each national forest is not feasible due to the 

time it takes to make such decisions, and this violates the 

goal of systematic assignments. The simulation procedure 

demonstrated herein is an effective means of investigating 

how to best adapt a stratified random sampling process to 

varied conditions. The goal for simulations is to identify a 

manageable set of three or four reduction strategies that are 

shown to have satisfactory results in reducing the sample 

size/cost with modest losses in reliability of the estimators. 

The simulations assigned to the National forests surveyed 

in FY 2011 were grouped by scenario/subscenario. An 

important next step would be the identification of one or 

more criteria (e.g., proximity to large population centers, 

visitation proportions in certain site-types or strata, or 

geographical or climatological features) which best explain 

the assignments. Determining what features best predict 

assignments of forests by scenario/subscenario can be 

accomplished through a discriminant analysis. Another 

option would be to use a logistic model with covariates 

based on the characteristics of the national forest subunit 

(size, location, etc.). This model would also yield insight 

into the influences of the factors, which best explain the 

groupings and are statistically significant. We can then 

generalize findings from this exercise to all forests in the 

National Forest System.

In conclusion, this simulation process serves as a useful 

means by which decisions about how best to implement 

sampling reduction decisions can be made beforehand. 

By deciding on a set of possible reduction strategies, it is 

possible to simulate the effects on a historic sample and 

to evaluate variability of estimates through SSEs. Though 

reduction strategies can be made individually, it becomes 

a time- and cost-prohibitive method, especially with large 

numbers of forests. The identification of a small set of 

strategies to be applied across all the forests to be sampled 

is essential to achieving an efficient, valid, and tractable 

solution to the problem. 

ACKNOWLEDGMENTS

This research benefitted from the constructive comments 

of Lynne Seymour, Ray Souter, and Eric White. Remaining 

errors and omissions are the sole responsibility of the 

authors. 

LITERATURE CITED

Cochran, W.G. 1977. Sampling techniques. 3d ed. New York: John Wiley. 

428 p.

de Vries, P. 1986. Stratified random sampling. In: Sampling Theory for 

Forest Inventory. Springer: Berlin Heidelberg: 31-55. 

English, D.B.K.; Kocis, S.M.; Zarnoch, S.J.; Arnold, J.R. 2002. Forest 

Service National Visitor Use Monitoring process: research method 

documentation. Gen. Tech. Rep. SRS–57. Asheville, NC. U.S. 

Department of Agriculture Forest Service, Southern Research 

Station. 14 p.

Gregoire, T.G.; Buhyoff, G.J. 1999. Sampling and estimating recreational 

use. Gen. Tech. Rep. PNW–456. Portland, OR: U.S. Department of 

Agriculture Forest Service, Pacific Northwest Research Station. 39 p.

SAS Institute Inc. 2009. SAS/STAT® 9.2 user’s guide. 2d ed. Cary, NC: 

SAS Institute Inc. 7,869 p. 

White, E.M.; Zarnoch, S.J; English, D.B.K. 2007. Area-specific 

recreation use estimation using the National Visitor Use Monitoring 

Program data. Research Note PNW–557. Portland, OR: U.S. 

Department of Agriculture Forest Service, Pacific Northwest Research 

Station. 26 p.

Zarnoch, S.J.; Bowker, J.M.; Cordell, H.K. [and others]. 2011a. 

Appalachian National Scenic Trail pilot survey. Gen. Tech. Rep. 

SRS–143. Asheville, NC: U.S. Department of Agriculture Forest 

Service, Southern Research Station. 56 p.

Zarnoch, S.J.; White, E.M.; English, D.B.K. [and others]. 2011b. The 

National Visitor Use Monitoring methodology and final results for 

Round 1. Gen. Tech. Rep. SRS–144. Asheville, NC: U.S. Department 

of Agriculture Forest Service, Southern Research Station. 74 p.
The Case of U.S. National Forests 21

Appendix—National forests and subunits sampled in FY 2011

Administrative 

code Forest name

FY 2011 

sampled days

A01005 Clearwater NF 211

A01016 Lolo NF 238

A01017 Nez Perce NF 202

U0201201 Pike–San Isabel, Forest portion 215

U0201202 Pike–San Isabel, Cimarron and Comanche Grasslands 112

A02013 Rio Grande NF 213

U0300301 Cibola NF, Forest portion 196

U0300302 Cibola NF, Black Kettle, Kiowa, and Rita Blanca Grasslands 58

A03006 Gila NF 219

A04010 Manti–LaSal NF 266

U0401701 Humboldt–Toyiabe NF, Spring Mountains NRA 156

U0401702 Humboldt–Toyiabe NF, outside of SMNRA 109

A05001 Angeles NF 291

A05004 Inyo NF 241

A05009 Modoc NF 124

A05013 Sequoia NF 243

A06003 Gifford Pinchot NF 235

A06006 Mt. Hood NF 237

A06012 Siuslaw NF 241

A06022 Columbia Gorge 165

U0800501 NFS in Florida, Apalachicola portion 64

U0800502 NFS in Florida, Ocala portion 132

U0800503 NFS in Florida, Osceola portion 68

A08008 George Washington–Jefferson NF 281

A08016 El Yunque NF 71

A09003 Chippewa NF 255

A09009 Superior NF 212

A09013 Chequamegon–Nicolet NF 223

A10105 Tongass NF, Sitka and Hoonah districts 128

You may request additional copies of this publication by email at pubrequest@fs.fed.us.

Number of copies is limited to two per person.

How do you rate this publication?

Scan this code to submit your feedback or go to 

www.srs.fs.usda.gov/pubeval

Askew, E.A; English, D.B.K.; Zarnoch, S.J. [and others]. 2014. Mitigating 

budget constraints on visitation volume surveys: the case of U.S. National 

forests. Gen. Tech. Rep. SRS-192. Asheville, NC: U.S. Department of 

Agriculture Forest Service, Southern Research Station. 21 p.

Stratified random sampling (SRS) provides a scientifically based estimate 

of a population comprising mutually exclusive, homogenous subgroups. 

In the National Visitor Use Monitoring (NVUM) program, SRS is used to 

estimate recreation visitation and visitor characteristics across activities on 

National forests. However, with rising costs and declining budgets, carrying 

out an annually established SRS poses challenges in how and where to reduce 

sampling while maintaining statistical precision. Furthermore, any reductions 

must produce results that validly compare to previous years’ SRS in trend 

analyses. 

Accurate estimates are necessary to manage resources for current and 

future demand; we explore this need through simulations and describe a 

methodology which can be generalized to any SRS-based process facing 

budgetary challenges. In this research, recent historic NVUM responses 

serve as groundwork for simulating various reduction scenarios. The ultimate 

goal is a manageable set of strategies from which to systematically assign a 

suitable reduction scenario for each sampling unit (i.e., forest). Using baseline 

historic data, we determine the sampling error variability by reduction 

scenario and experiment with various sets of three or four pre-defined 

strategies to identify viable reduction candidates. A desirable candidate is 

one that achieves acceptable error bounds (based on historic values) and 

reductions overall while meeting specified budget constraints (e.g., 20 percent 

overall fewer sampling days).

Keywords: Budget, National Visitor Use Monitoring, precision, sampling, 

scenarios, simulations, strata. 
Non-Discrimination Policy

The U.S. Department of Agriculture (USDA) prohibits discrimination against 

its customers, employees, and applicants for employment on the bases of race, 

color, national origin, age, disability, sex, gender identity, religion, reprisal, and 

where applicable, political beliefs, marital status, familial or parental status, 

sexual orientation, or all or part of an individual’s income is derived from any public assistance 

program, or protected genetic information in employment or in any program or activity 

conducted or funded by the Department. (Not all prohibited bases will apply to all programs 

and/or employment activities.)

To File an Employment Complaint

If you wish to fi le an employment complaint, you must contact your agency’s EEO Counselor 

within 45 days of the date of the alleged discriminatory act, event, or in the case of a personnel 

action. Additional information can be found online at http://www.ascr.usda.gov/complaint_

fi ling_fi le.html.

To File a Program Complaint

If you wish to fi le a Civil Rights program complaint of discrimination, complete the USDA 

Program Discrimination Complaint Form, found online at http://www.ascr.usda.gov/

complaint_fi ling_cust.html, or at any USDA offi ce, or call (866) 632-9992 to request the form. 

You may also write a letter containing all of the information requested in the form. Send your 

completed complaint form or letter to us by mail at U.S. Department of Agriculture, Director, 

Offi ce of Adjudication, 1400 Independence Avenue, S.W., Washington, D.C. 20250-99410, by fax 

(202) 690-7442 or email at program.intake@usda.gov. 

Persons with Disabilities 

Individuals who are deaf, hard of hearing or have speech disabilities and you wish to fi le either 

an EEO or program complaint please contact USDA through the Federal Relay Service at 

(800) 877-8339 or (800) 845-6136 (in Spanish). 

Persons with disabilities who wish to fi le a program complaint, please see information 

above on how to contact us by mail directly or by email. If you require alternative means of 

communication for program information (e.g., Braille, large print, audiotape, etc.) please 

contact USDA’s TARGET Center at (202) 720-2600 (voice and TDD).