A Temporal Importance-Performance Analysis 

of Recreation Attributes on National Forests

A Technical Document Supporting the Forest Service

Update of the 2010 RPA Assessment

Ashley E. Askew, J.M. Bowker, Donald B.K. English, Stanley J. Zarnoch, and Gary T. Green

Forest 

Service

General Technical August 2017

Report SRS–223

Southern Research 

Station

United States

Department of

Agriculture
The Authors:

Ashley E. Askew, Postdoctoral Research Associate, University of 

Georgia, Warnell School of Forestry and Natural Resources, Athens, 

GA 30602; J.M. Bowker, Research Social Scientist, U.S. Department of 

Agriculture, Forest Service, Southern Research Station, Athens, GA 30602; 

Donald B.K. English, National Visitor Use Monitoring Program Manager, 

U.S. Department of Agriculture, Forest Service, Washington Office, 

Washington, DC 20250; Stanley J. Zarnoch, Mathematical Statistician, 

U.S. Department of Agriculture, Forest Service, Southern Research Station, 

Clemson, SC 29634; and Gary T. Green, Assistant Dean of Academic 

Affairs and Professor, University of Georgia, Warnell School of Forestry 

and Natural Resources, Athens, GA 30602.

August 2017

Southern Research Station

200 W.T. Weaver Blvd.

Asheville, NC 28804

www.srs.fs.usda.gov

Product Disclaimer

The use of trade or firm names in this publication is for 

reader information and does not imply endorsement by the 

U.S. Department of Agriculture of any product or service.
A Temporal Importance-Performance Analysis 

of Recreation Attributes on National Forests

A Technical Document Supporting the Forest Service 

Update of the 2010 RPA Assessment

Ashley E. Askew, J.M. Bowker, Donald B.K. English, Stanley J. Zarnoch, and Gary T. Green

ABSTRACT 

The outdoor recreation component of the 2010 Resources Planning Act (RPA) Assessment provided projections 

and modeling of participation and intensity by activity. Results provided insight into the future of multiple outdoor 

recreation activities through projections of participation rates, numbers of participants, days per participant, and 

total activity days. These projections can be considered in managing potential shifts in outdoor recreation. In 

anticipation of projected trends, resources can be allocated appropriately by each outdoor recreation activity, or by 

the settings in which these activities take place. Decades of marketing and consumer research have established that 

an industry depends on and benefits from satisfied customers. In this study, we apply a widely used tool with origins 

in marketing research known as importance-performance analysis (IPA) to better understand visitor satisfaction 

with recreation settings on national forests. The results from IPA can supplement planning for improving efficient 

provision of recreation opportunities through management of setting attributes, which in turn could reinforce 

activity popularity and intensity. With the systematic and consistent data collection methodology in place from the 

National Visitor Use Monitoring Program, we assessed visitor ratings of importance and satisfaction on multiple 

attributes related to national forest recreation, as well as overall satisfaction and crowding ratings, over two recent 

survey periods (rounds 2 and 3) spanning 10 years. These attributes ranged from those in the natural domain (e.g., 

condition of natural environment and quality of scenery) to the developed setting (e.g., availability of parking and 

condition of parking lots). We classified the attributes into one of four IPA management actions and also conducted 

an intertemporal analysis to assess sustainability. The significance of changes in importance and satisfaction between 

rounds can provide insight into potentially problematic shifts, such as declining satisfaction in conjunction with 

increasing importance. Overall, users of the national forests were satisfied and found the attributes important. We 

identify relative overperformers and underperformers to better inform the allocation of resources within site type per 

attribute for the RPA regions and the Nation.

Keywords: Importance-performance analysis, National Visitor Use Monitoring (NVUM) Program, outdoor 

recreation, recreation setting attributes, Resources Planning Act (RPA) Assessment, visitor satisfaction.

A Technical Document Supporting the Update to the Forest Service 2010 RPA Assessment iii

CONTENTS

KEY FINDINGS. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iv

INTRODUCTION. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

IMPORTANCE-PERFORMANCE ANALYSIS. . . . . . . . . . . . . . . . . . . . . . . . . . . 2

Consumer Theory and Motivations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

Direct Measurements for Importance-Performance Analysis. . . . . . . . . . . . . 2

Data Description. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

RESULTS. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

NVUM Sample Characteristics. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

Purpose of Visitation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

IPA Results: Temporal Analysis and Attribute Classifications. . . . . . . . . . . . 10

Designated Wilderness. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

General Forest Areas. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

Day-Use Developed Sites. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

Overnight-Use Developed Sites. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

Overall Satisfaction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

Crowding. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

CONCLUSIONS. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

Limitations and Future Directions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

ACKNOWLEDGMENTS. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32

LITERATURE CITED. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
A Temporal Importance-Performance Analysis of Recreation Attributes on National Forests iv

KEY FINDINGS 

• Recreationists in all four RPA regions and across the Nation generally 

had high satisfaction levels with their national forest visits.

• Recreationists have the highest overall satisfaction scores on condition 

of the natural environment, quality of scenery, and perception of safety, 

all deemed very important to the outdoor recreation experience.

• Visitors to designated Wilderness were generally satisfied, with a 

possible need for attention to trail conditions in the South.

• For all RPA regions and the Nation, developed sites could benefit from 

improvement of restroom cleanliness and adequacy of signage. 

• Overall, visitor satisfaction increased between two visitor survey rounds 

conducted over 10 years, most notably by visitors to North Wilderness 

and South overnight-use developed sites.

• Visitors to the Pacific Coast region perceived increases in crowding, 

especially in Wilderness. Visitors in the South perceived either 

declining or static levels of crowding. 
A Technical Document Supporting the Update to the Forest Service 2010 RPA Assessment

INTRODUCTION

The Forest Service, U.S. Department of Agriculture, has 

conducted natural resource analyses for more than a century. 

The 1974 Forest and Rangeland Renewable Resources 

Planning Act (RPA) (P.L. 93378, 88 Stat 475, as amended) 

established a periodic reporting requirement and broadened 

coverage of the Forest Service analyses to include all 

renewable resources on forests and rangelands of the United 

States. The 155 national forests and 20 grasslands in the 

United States encompass 192 million acres across 43 States, 

Puerto Rico, and the Virgin Islands, with uses ranging from 

commercial and economic pursuits to outdoor recreation. 

The RPA Assessment provides reliable information every 

10 years, with 5-year Updates, on the status, trends, and 

projected future of the Nation’s renewable resources. The 

2010 RPA Assessment was the fifth assessment prepared 

under the mandate of the 1974 RPA legislation.

The outdoor recreation component of the 2010 RPA 

Assessment presented trends and futures (Cordell 2012) and 

included national projections, through 2060, of participation 

and use for 17 natural resource-based recreation activities 

across alternative futures, allowing variation in population 

growth, socioeconomic conditions, land use changes, and 

climate (Bowker and others 2012). The 2010 Assessment also 

provided detailed information about trends in and the current 

supply of recreation opportunities (Cordell and others 2013). 

Previous analyses on a decadal basis with similar directives 

included summaries of demand and recreation supply trends 

(Cordell and others 1999) and national projections for outdoor 

recreation to 2040 (Cordell and others 1990), 2050 (Bowker 

and others 1999), and 2060 (Bowker and Askew 2012, 

Bowker and others 2012). The methodology for formulating 

national-level outlooks has also been applied on a regional 

basis for the South in Bowker and others (2013, 2014) and for 

the North in Bowker and Askew (2013). The intent with all 

of these studies was to provide solid evidence of recreation 

trends and potential futures with outlooks toward long-term 

supply and demand.

The recreation experience is a product of complex interactions of human planning and psychology. Recreation 

planning and management should factor in users’ perceptions, 

improving what could be deterrents and maintaining the 

attractive features (O’Leary and Adams 1982). Quantifying 

the psychological and experiential aspects can present 

challenges, but approximations are available through survey 

data. This Update to the 2010 RPA supplement evaluates 

recreation from a different perspective than used in the 2010 

RPA Assessment, shifting from respondents’ participation 

indicators and days of activity to levels of satisfaction. 

Specifically, this study reports on importance-performance 

analyses (IPAs), also called importance-satisfaction analyses, 

applied to attributes of the recreation experience on national 

forests (NFs) across the four RPA regions and the Nation. 

Moreover, with data spanning two time periods, potential 

trends can be identified. In the case of declining satisfaction 

for important attributes, managerial decisions may be needed 

to improve and sustain visitor satisfaction. 

In this report we evaluate visitor perceptions of factors or 

attributes deemed relevant to outdoor recreation a priori, 

specifically ratings of attribute importance and satisfaction on 

NFs aggregated at the RPA regional and national levels. The 

report proceeds as follows. First, we discuss customer (visitor) 

motivations from consumer theory that underpin applications 

of IPA, followed by descriptions of IPA methodology. We 

then discuss the National Visitor Use Monitoring (NVUM) 

Program, which is the primary data source. We conclude with 

the Results section, covering not only the IPA implications 

regionally and nationally, but also the temporal changes 

between two successive periods of NVUM sampling 

spanning fall of 2005 to 2014. In closing, we note some 

caveats and potential future directions that could ensue from 

this work.

A Temporal Importance-Performance Analysis 

of Recreation Attributes on National Forests:

A Technical Document Supporting the Update to the Forest Service 

2010 RPA Assessment

Ashley E. Askew, J.M. Bowker, Donald B.K. English, Stanley J. Zarnoch, and Gary T. Green
2 A Temporal Importance-Performance Analysis of Recreation Attributes on National Forests

IMPORTANCE-PERFORMANCE ANALYSIS 

Consumer Theory and Motivations

Importance-performance analysis (IPA) in recreation relies 

on perceiving the recreation visitor as a customer. In business, 

satisfied customers are linked to profitability (Anderson and 

others 1994). Anderson and others (1994) define cumulative 

customer satisfaction as the comprehensive assessments of 

buying and consuming a good or service through time. The 

customers’ overall impressions of their transactions and 

returns may be used to gauge how the business is performing, 

serving also as markers of future potential. Additionally, 

positive customer impressions from multiple experiences 

can mean loyalty, favorable word of mouth, and further 

investments, translating to increased profits and business 

sustainability (Matzler and others 2004).

To develop strategies for sustained satisfaction in recreation, 

particularly on public lands, requires an understanding 

of participants’ motivation and incentives. The human 

decisionmaking process is multifaceted, but a simplified 

model is necessary for a generalized approach. Push-pull 

theory describes two kinds of factors that are drivers of 

recreation (Ziegler and others 2012). Push factors are those 

elements that encourage recreationists to fulfill emotional 

desires, such as spending time with family or friends, 

seeking the excitement of a challenging activity, exploring 

solitude in the wilderness, or enjoying the serenity of natural 

scenery. Pull factors pertain directly to the recreation venue 

or activity, such as how safe a participant feels, availability 

of opportunities and sites, and the associated expenses. 

The decision to seek recreation may be explained by push 

factors; in turn, pull factors determine the activity of choice 

and location. How well recreation experiences satisfy 

the dynamics of push and pull factors determines how 

participants will engage over the long term. Participants 

enter a recreation decision with a perception, perhaps based 

on word of mouth, prior experiences, or anticipation of a 

first visit. The “disconfirmation of expectations” paradigm 

describes an inverse relationship between expectations and 

the subsequent experience (Sever 2015). A participant who 

invests time, money, and planning for a recreation trip enters 

with an expectation that his or her motivation will be met; 

if the outcome falls below expectations, then he or she will 

experience a negative disconfirmation (dissatisfaction). If 

the participant’s perceptions exceed expectations, then the 

outcome is a positive disconfirmation (satisfaction).

A primary question in consumer science has been how to 

quantify and even define importance. Jaccard and others 

(1986) define importance as how consumers react to change. 

If a change in the consumer’s perception of an attribute’s 

performance changes the view of that product or service, 

then that attribute is deemed important. Performance is 

more straightforwardly measured; the evaluation may 

be done post-experience, with comparisons against the 

expectations that the consumer formulated before entering 

the transaction(s). Ratings of importance may be direct or 

indirect, quantifying various facets of importance depending 

on the measure used (Jaccard and others 1986). Direct 

measurements are based on self-stated ratings such as Likert 

scales (Lego and Shaw 1992, Sever 2015). To circumvent 

potential biases with self-rating, an alternative is the use of 

indirect methods, which involve more complex statistical 

formulations using models or correlations. An implicit 

derivation of importance may be achieved by linking overall 

satisfaction to the set of attribute satisfaction levels (Abalo 

and others 2007). Because of the comprehensive scope of 

the overall satisfaction measure, this can implicitly contain 

information about the relative importance of each attribute. 

If the survey asks for satisfaction ratings only (overall and on 

individual attributes), omitting the importance scale translates 

to reduced rater fatigue and fewer potential sources of bias. 

However, this necessitates a more complicated analysis that 

diverges from the simplicity of traditional IPA. Techniques 

based on linear regression can extract relative importance 

by using the relationship between overall satisfaction and 

the set of attributes. However, the use of linear regression is 

problematic when attributes are collinear, and when attributes 

are not necessarily linearly related to overall satisfaction. So 

a simpler survey, based only on satisfaction ratings, has the 

tradeoff of a more complex analysis, with potentially invalid 

assumptions. The advantage of using both importance and 

satisfaction scales as in this study is that fewer assumptions 

require validation, and the processing of results is more 

straightforward, despite limitations inherent in analyzing 

Likert-type variables.

Direct Measurements for 

Importance-Performance Analysis

Methodology—Martilla and James (1977) describe their 

development of IPA in the setting of an automobile dealer 

and examine, from a marketing perspective, the customer’s 

experience. IPA is broadly applicable because of the relative 

simplicity of analysis methodology, in its most basic form 

requiring little statistical expertise and facilitating graphical 

interpretations. Additionally, it reduces a multifaceted process 

of human decisionmaking and evaluation to a set of numbers 

that can be analyzed and compared. The simplification is 

necessary to assess results from a general standpoint (Lai 

and Hitchcock 2015). Consequently, the method is applicable 

to a variety of settings, including recreation and tourism 
3 A Technical Document Supporting the Update to the Forest Service 2010 RPA Assessment

(Albayrak 2015, Gill and others 2010, Huang and others 2015, 

Janes and Wisnom 2003, Lee 2015, Pan 2015, Sheng and 

others 2014, Sörensson and von Friedrichs 2013, To and others 

2015, Ziegler and others 2012). Its simplicity and applicability 

make the method convenient to implement, albeit with some 

validity considerations discussed in the next paragraph and 

in the Conclusions section. These concerns have given rise 

to modified strategies that incorporate statistical complexity, 

but then the ease of application suffers. As such, practitioners 

tend to prefer the simpler approach and the use of any 

available explanatory power. We employ two variations of 

IPA known as the data-centered quadrants approach and the 

means and diagonal line model (Lai and Hitchcock 2015). We 

apply the methodology to recreation settings on NFs; we view 

the recreation visitors as customers rating their experiences, 

specifically the perceived importance and satisfaction levels 

of these experiences for a variety of setting-related attributes.

The original 1977 methodology occurs in three phases: 

attribute selection relevant to the product or service, rating 

of attributes by respondents, and computing means and 

thresholds for a two-dimensional plot (Martilla and James 

1977). Essential to IPA are suitable designs for the survey 

and sampling techniques. Attributes used in evaluating 

responses through IPA are often formulated in advance 

based on expertise or prior research. These items should 

be relevant and responsive to managerial decisions, as 

improper or insufficient choices limit the detection powers 

of IPA (Martilla and James 1977). For example, if a survey 

includes an unimportant attribute, then the respondents’ 

mean importance rating is likely to be low. Additionally, 

sampling expenses and efforts are expended unnecessarily if 

unimportant attributes are presented. The selected items of 

relevance can subsequently be used to identify key features 

and areas of improvement. Asking for both importance 

and satisfaction ratings supplies researchers with a twodimensional perspective on the quality of experience for 

customers. The basic IPA methodology operates on a twofactor theory of customer satisfaction as outcomes based 

on the interactions of importance and satisfaction on a 

set of relevant attributes (Lai and Hitchcock 2015). IPA 

maintains an expectation-disconfirmation framework, where 

performance exceeding expectations produces satisfaction, 

and performance falling below expectations produces 

dissatisfaction (Sever 2015). Traditional IPA operates under 

the assumption of independence between importance and 

performance, as well as the linearity and symmetry of the 

relationship between overall satisfaction and performance 

(Lai and Hitchcock 2015). This approach may affect the 

validity of results from the IPA methodology applied without 

adjustments. However, some of the techniques developed 

to increase validity can be overly complex and are based on 

assumptions which are currently debatable.

Plotting importance against performance provides a snapshot 

of aggregate satisfaction and potential strategic planning for 

areas of improvement. Martilla and James (1977) formulated 

the quadrants approach for attributes, where thresholds 

established for importance and performance divide the plot 

into four sections:

1. Relatively high importance, relatively high 

satisfaction: The desired outcome. A quality that is 

deemed highly important and performing well means 

to “keep up the good work” (GW).

2. Relatively high importance, relatively low 

satisfaction: A highly important quality that is not 

performing well means that we should “concentrate 

here” (CH). Items falling into this quadrant over time 

suggest a situation in need of attention.

3. Relatively low importance, relatively high 

satisfaction: A potential indicator of overkill, in 

which resources are used on an area of low importance 

despite customers being satisfied; in a limited budget, 

the areas of “possible overkill” (PO) may be a starting 

point for shifting resources.

4. Relatively low importance, relatively low 

satisfaction: Though customers may not be as satisfied 

as they could be, the low importance attached to this 

quality indicates that this is an item of “low priority” 

(LP).

The difference between the threshold and mean of the 

attribute can indicate a deviation from the threshold in 

either a negative or a positive direction. Items performing 

very highly may appear in the extreme reaches of the 

GW quadrant, whereas attention may be needed for those 

appearing far into the CH section. However, the attributes 

whose means fall close to the thresholds may be inconclusive, 

especially with standard deviations such that confidence 

intervals of importance or performance span multiple 

quadrants. 

Setting the thresholds is “a matter of judgment,” with options 

available such as using the center of the Likert scale, medians, 

or means (Martilla and James 1977, p. 79). Having a variety 

of ways to determine thresholds can be advantageous in 

that the analyses are flexible and adjustable to specific 

purposes; a downside is that there is no one-size-fits-all 

method, necessitating some judgment calls and perhaps the 

examination of multiple approaches. Thresholds based on the 

Likert scale (e.g., the scale midpoint) are scale-dependent, 

whereas those based on the observations are data-centered 

(e.g., median or mean). If a goal threshold is predetermined, 
4 A Temporal Importance-Performance Analysis of Recreation Attributes on National Forests

such as “at least somewhat satisfied” and “at least somewhat 

important” being expressed at 4 on a 5-point scale, then 

the crosshairs are target-driven (Lai and Hitchcock 2015). 

If attributes were removed from analysis or observations 

produced different values, then the thresholds in scalecentered and target-driven analyses are not affected. One 

advantage is that thresholds are convenient and systematic; 

however, this can be problematic if the means of the attribute 

all fall into the same quadrant. For example, a tendency 

toward ratings of 5’s for importance suggests that attributes 

can be well above a scaled center of 3 on a 5-point scale, 

thereby yielding an insensitive test. Data-driven thresholds, 

especially based on means, ensure finer partitions for the data 

points into the quadrant spaces per the four categories. Even 

so, research continues into how to best establish thresholds 

and to subdivide observations into the decisionmaking 

classifications (Lai and Hitchcock 2015).

In addition to setting thresholds for quadrants, another 

evaluation aid used by IPA analysts is the iso-priority line 

(Lee 2015). This is a diagonal line on which performance 

equals importance. This diagnostic indicates which items 

are of highest concern by using pairwise differences of 

satisfaction and importance as the measures for a more 

sensitive test (Ziegler and others 2012). In other words, the 

iso-priority approach reduces the two-dimensional analysis 

to one dimension, as used in service sectors (Albayrak 

2015). A downside is that the test is less informative, using a 

division of only two spaces rather than the four classifications 

seen in traditional IPA (Sever 2015). A hybrid approach 

can be formulated by overlaying an iso-priority line on the 

conventional IPA based on either scale- or data-centered 

thresholds. This method expands the potential area of 

concern (fig. 1). The distances can then be used to gauge how 

conclusively an attribute falls into a quadrant.

IPA in recreation and natural resources—IPA 

originated in the context of the automotive industry to better 

understand drivers of customer loyalty by using surveys 

of importance and satisfaction ratings on select attributes 

(Martilla and James 1977). Though there are multiple 

aspects to measuring importance, Martilla and James (1977) 

developed IPA using direct self-stated measurements on the 

importance (and satisfaction) on a 5-point Likert scale. Since 

then, the technique has been applied to many industries, 

including tourism and recreation. The methodology is now 

nearly four decades old, but applications are ongoing. With 

the many applications of IPA in recreation through the years, 

the articles discussed next are not an exhaustive list but are 

highlights. Research can take one of two forms: evaluating 

how to sustain customer satisfaction or comparing IPA 

methods (and variations) by using an example dataset in a 

recreation context. Furthermore, IPA may be used to compare 

groups hypothesized to demonstrate different behaviors, to 

make temporal comparisons (Leeworthy and others 2004), 

or to evaluate different classification methodologies. First, 

we discuss some applications of IPA in a general recreation 

setting and then those most relevant to this report.

Figure 1—Traditional importance-performance analysis with thresholds (A) and a hybrid using the iso-priority line (B), shown as dashes, where 

the importance and performance are equal (Abalo and others 2007).

(A)

Concentrate here (CH)

Importance

Low priority (LP) Possible

overkill (PO)

Keep up the 

good work

(GW)

Performance

(B)

Concentrate here (CH)

Importance

Low priority (LP)

Possible

overkill (PO)

Keep up the 

good work

(GW)

Performance
5 A Technical Document Supporting the Update to the Forest Service 2010 RPA Assessment

An early application by O’Leary and Adams (1982) used 

IPA on attributes identified through literature as potentially 

important for urban river corridor recreation, such as water 

quality, safety, and transportation. The responses came from 

two applications of the Urban River Recreation Survey, one 

in 1980 and the other in 1981, to three distinct user groups 

differing in attitudes toward recreation. The classifications 

were a function of scaled composite rankings based on 

ranked median importance values and ordered absolute values 

of Pearson correlation coefficients (between importance 

and performance). A comparison between 1980 and 1981 

responses juxtaposed the attributes’ classifications by each 

quadrant. The 1981 responses facilitated a comparison of 

two user groups, black and white respondents, on their 

perceptions for that year.

IPA results depend on thresholds—and thus the plotting 

methodology. Crompton and Duray (1985) compared multiple 

plotting strategies on tourism data from British respondents. 

The respondents supplied their importance and performance 

ratings for various attributes of tourism in Texas, such as food 

and drink, availability of sandy beaches, and personal safety 

at their destination. Sörensson and von Friedrichs (2013) used 

IPA in the context of sustainable tourism, acknowledging the 

need to maintain satisfaction amid limitations on available 

resources. Their application of IPA compared two groups 

with potentially different recreation patterns, Italian and 

international visitors to Bologna, Italy. In 2012, Ziegler and 

others used IPA with iso-priority lines to evaluate tourist 

satisfaction with whale shark viewing in Isla Holbox, 

Mexico. Lee (2015) used IPA and gap analysis (difference 

between mean importance and mean satisfaction) to gauge 

recreationists’ perceptions of public zoos in Korea and also 

analyzed the reliability and validity of attributes. Another 

recent application of IPA by Albayrak (2015) integrated 

competitor information in the tourism setting, specifically 

in five-star hotels in Antalya, Turkey, as part of a revised 

methodology known as Importance Performance Competitor 

Analysis (IPCA). The additional aspect allowed for the 

comparison of a focal hotel and a competitor, a dimension 

not available with basic IPA. Though not directly applicable 

to this research due to the noncompetitive nature of NFs, this 

example demonstrates an adaptation of conventional IPA and 

may be used in competitive recreation markets.

The IPA studies most relevant to our research are listed in 

table 1. Our analyses use aspects of these approaches, with 

novel application to NVUM responses for assessment of 

Table 1—Select applications of importance-performance analysis (IPA) in recreation settings by year, author, general 

methodology, and context

Year Author(s) Methodology Context

Number of 

attributes Sample size

1982 O'Leary and 

Adams

IPA, ranking and Pearson 

correlation coefficients, 

separate analyses of two 

groups

Urban river recreation Year 1980: 48;

year 1981: 44

Year 1980: 181;

year 1981: 118

1985 Crompton and 

Duray

IPA, comparison of four 

plotting methods

British tourists on 

aspects of visits to 

Texas

28 544

2004 Leeworthy, Wiley, 

and Hospital 

IPA, intertemporal comparisons on two time 

periods for distinct user 

groups

Artificial and natural 

reef usage, recreation and tourism for 

Florida Keys National 

Marine Sanctuary 

25 1995–96: 917 visitors, 455 residents;

2000–01: 275 visitors, 609 residents

2010 Gill, Bowker, 

Bergstrom, and 

Zarnoch

IPA with weighting by 

inverse number of trips

Usage of Virginia 

Creeper Trail, Virginia 

17 1,308

2012 Ziegler, Dearden, 

and Rollins

IPA, iso-rating line, gap 

analysis (paired t-tests)

Whale shark tourism 

in Mexico

15 397 total, 90% response rate

2013 Sörensson and 

von Friedrichs

Sustainable tourism, 

comparison of two 

groups

Destination: Bologna, 

Italy

24 150 international tourists,

139 Italian visitors

2015 Albayrak Importance-Performance 

Competitor Analysis 

(IPCA)

Five-star hotels in 

Antalya, Turkey

38 478

2015 Lee IPA, factor analysis, 

gap analysis, attribute 

reliability

Public zoos in Korea 23 697
6 A Temporal Importance-Performance Analysis of Recreation Attributes on National Forests

satisfaction by RPA region. Gill and others (2010) applied 

IPA in the setting of the Virginia Creeper Trail (Virginia). 

They obtained a stratified random sample similar to 

NVUM. Length-of-stay bias and endogenous stratification 

bias (bias stemming from the fact that the probability of a 

recreationist being sampled at a given site is related to the 

number of times he or she visits; Gill and others 2010, Shaw 

1988) are inherent in onsite sampling. The former was not 

problematic for NVUM due to the policy of sampling lastexiting recreationists (LERs). The latter can be adjusted 

through inverse weighting by annual number of trips taken 

(see Data Description for a more detailed discussion). 

This approach treats the responses of a daily visitor as the 

same as those of an annual recreationist, so all responses 

are equally important. Gill and others (2010) found that 

weighting to account for endogenous stratification could 

lead to conclusions different from the naïve (or unweighted) 

approach, typically followed in IPAs, for several trail 

attributes including safety/security, crowding, parking, and 

trail structures.

Leeworthy and others (2004) not only examined distinct 

user groups (experienced vs. inexperienced, visitors vs. local 

residents) but also carried out an analysis between two time 

periods. The application was in the context of artificial and 

natural reefs in a Florida community, specifically as part of 

the recreation and tourism analysis for the Socioeconomic 

Research and Monitoring Program for the Florida Keys 

National Marine Sanctuary. The survey was applied over 

multiple years (1995–96 and 2000–01), similar to NVUM, 

but with shorter, nonconsecutive periods. The respondents 

in the two time periods were assumed independent, which 

is also similar to NVUM, so the responses did not warrant 

a paired before/after comparison. Leeworthy and others 

(2004) compared the two periods by t-testing attributes 

across time periods within the distinct user groups, noting 

whether satisfaction and importance levels changed and in 

what direction. Their methods did not account for endogenous 

stratification. 

In the present research, we used intertemporal comparisons 

testing as in Leeworthy and others (2004), specifically testing 

importance and satisfaction across two sampling periods (i.e., 

NVUM rounds 2 and 3). We also incorporated the weighting 

procedure put forth by Gill and others (2010). Our analyses 

were conducted across multiple regions and site types, 

evaluating for commonalities in test output.

Data Description

The data source for our IPA study was the NVUM Program. 

This program has the objective of estimating visitation 

volume for recreationists on NFs and analyzing visitation 

with respect to attributes such as activity participation, visit 

duration, demographics, and satisfaction levels (USDA Forest 

Service 2016). Only LERs are eligible to answer the survey, 

which ensures that responses are from those using the sites/

forests for recreation, and that double-counting is avoided. 

The NVUM sampling screens whether a person has recreated 

on the site/forest, proceeding with the interview only after 

an affirmative answer. The survey is onsite and collects 

information immediately after a visit (English and others 

2002). Though a mail-back or Internet response format may 

permit time to reflect and evaluate post-trip, the onsite survey 

asks recreationists to present their impressions from recent 

memory and recall upon concluding the trip. The survey 

includes information about the recreation site being exited 

and respondents’ home ZIP codes if supplied. Respondents 

are asked about their current visit, and also about their 

visitation to the NF over the previous 365 days.

The NVUM survey consists of three major variations: 

basic questions (asked of everyone), an economics module 

(asked of a subset along with the basic questions), and a 

satisfaction module (asked of a subset along with the basic 

questions). The basic module focuses on demographics and 

respondents’ recreation habits by activity. Those who receive 

the satisfaction module are asked about satisfaction and 

importance levels for attributes and perceptions of crowding. 

The NVUM methodology employs stratified random 

sampling (Cochran 1977) of site days on each forest, 

considering use levels and site types (English and others 

2002). There are four site types: day-use developed sites 

(DUDS), general forest areas (GFA), overnight-use developed 

sites (OUDS), and designated Wilderness (WILD). Persons 

using DUDS or OUDS encounter facilities with moderate, 

heavy, or high degrees of modification per the Forest 

Service’s Infrastructure (INFRA) development scale, 

with OUDS affording opportunities to stay overnight 

in campgrounds, cabins, lodges, or resorts (English and 

others 2002). Visitation to DUDS involves participation 

in activities that may occur in picnic areas, fishing sites, 

playgrounds, or visitor centers, to name a few. WILD consists 

of lands and waters under the jurisdiction of the National 

Wilderness Preservation System. GFA include any NF 

components remaining outside the DUDS, OUDS, and WILD 

classifications (English and others 2002). Variations may 

occur in site-type visitation due to activity opportunities on 

each. For example, motorized activities are not permitted in 

WILD, but could occur in either of the developed site types 

and in designated areas within GFA. 

A sample that is not homogeneous due to known 

characteristics must be segmented in order for IPA to be 

effective. This permits comparisons of distinct groupings for 

strategic planning (Lai and Hitchcock 2015, Leeworthy and 
7 A Technical Document Supporting the Update to the Forest Service 2010 RPA Assessment

others 2004). Distributions of sampling days by site type and 

Forest Service region under the NVUM methodology permit 

analysis of attributes relevant to each segment. By stratifying 

according to site type, the variations are more finely divided 

so that the sources of variation can be identified. In sampling 

design, accounting for use levels through exit volumes (low, 

medium, high, or very high) ensures sufficient sampling 

across site types within budgetary constraints (Askew and 

others 2014). Our analysis examined the site types by region 

as the finest resolution. However, as will be discussed in the 

Conclusions section, future work in testing mean ratings will 

need to adjust for the sampling being carried out at multiple 

use levels across site types.

The nature of the NVUM sampling procedure produces data 

that are endogenously stratified (Gill and others 2010, Shaw 

1988). The sampling process allocates a certain number of 

sampling days within a period for each site/forest. Morefrequent visitors have higher probabilities of being sampled, 

which necessitates adjustment to statistics describing 

individuals’ behaviors (as opposed to overall visitation). The 

problem of bias resulting from endogenous stratification is 

rarely recognized or acknowledged in studies applying IPA to 

onsite data. To our knowledge, the report by Gill and others 

(2010) has been the only IPA study to address this issue and 

offer a corrective procedure, weighting respondents’ values 

by the inverse of their reported annual visits. Weighting 

by the inverse of a respondent’s forest visits in a year, the 

statistics view the recreationist who participates 365 days 

annually in the same way as a one-time visitor. Furthermore, 

if the parameters of interest are dependent on trip frequency, 

then there is a possibility of trip frequency bias (Gill and 

others 2010), which again can be mitigated through weighting 

by the inverse of the annual number of trips.

The national data collection processes the site/forest of 

interview, so the observations can be refined at a regional 

level or even forest-by-forest analysis. The Forest Service 

regions shown in figure 2(A) can be aggregated into RPA 

regions as shown in figure 2(B), where the North and South 

RPA regions correspond to Forest Service Regions 9 and 

8, respectively. The Rocky Mountain RPA region is an 

aggregate of Forest Service Regions 1, 2, 3, and 4. Last, the 

Pacific Coast RPA region comprises Forest Service Regions 

5, 6, and 10. 

The NVUM survey was first implemented from 2000 to 

2003 (round 1). The methodology was significantly revised 

in 2004, with subsequent implementation in 2005. Because 

of the methodological change, round 1 is not comparable 

to subsequent rounds (5-year sampling cycles); thus, 

the responses for round 1 can be viewed as archival and 

descriptive only for that period. The methodological revisions 

improved on the consistency of estimation and sampling 

procedures (English and others 2002, Zarnoch and others 

2011). The end product of the periodic sampling is survey 

data collected from all NFs through appropriate prework and 

fieldwork. Approximately 20 percent of forests are surveyed 

in a given year, producing a full dataset after a completed 

round. The years from FY2005 to FY2009 make up round 

2, and FY2010 to FY2014 correspond to round 3. Because 

of the improvements, a consistent and nationally available 

framework is now in place that will permit temporal trend 

comparisons as more data and time points become available. 

The strength of this survey process makes it a desirable 

candidate for application to future recreation analyses for 

RPA Assessments and Updates. 

(A)

(B)

Figure 2—Delineation of the United States into (A) Forest Service 

(FS) regions (R) and (B) FS regional aggregates known as RPA 

regions.
8 A Temporal Importance-Performance Analysis of Recreation Attributes on National Forests

Figure 3(A and B) shows that for each of the two rounds, the 

three NVUM survey types were divided roughly equally 

among the respondents, with a somewhat greater percentage 

receiving only the basic component, as opposed to the basic 

plus economics module or the basic plus satisfaction module. 

The analyses in this study focus on the data collected for the 

satisfaction module. 

The NVUM survey contains a pool of 16 attributes generally 

describing conditions at DUDS, GFA, OUDS, and WILD. 

The attributes were determined by using expert knowledge, 

specifically from NVUM Program and National Forest 

System (NFS) staff in conjunction with forest-level personnel. 

The selection process incorporated prior satisfaction research 

on Federal lands and interviews with Forest Service managers 

0

5

10

15

20

25

30

35

40

Unweighted percentage of surveys

RPA region by site type

Unweighted Percentages of NVUM Response by Survey Type, Round 2

Basic Basic+Economic Basic+Satisfaction

(A)

0

5

10

15

20

25

30

35

40

Unweighted percentage of surveys

RPA region by site type

Unweighted Percentages of NVUM Response by Survey Type, Round 3

Basic Basic+Economic Basic+Satisfaction

(B)

Figure 3—Percentages of National Visitor Use Monitoring responses by survey type administered, 

by region and site type, for (A) round 2 and (B) round 3. RPA regions: N=North; S=South; RM=Rocky 

Mountain; PC=Pacific Coast. Site types: DUDS=day-use developed sites; GFA=general forest areas; 

OUDS=overnight-use developed sites; WILD=designated Wilderness.
9 A Technical Document Supporting the Update to the Forest Service 2010 RPA Assessment

on the most important aspects. Incorporating feedback from 

managers in the NVUM survey connects user responses to 

potential action items.

Respondents were presented with a set of predetermined 

relevant attributes and corresponding 5-point Likert scales 

for evaluating their visits to the NF. According to Martilla 

and James (1977), having a 5- or 7-point Likert scale presents 

a sufficient set of options for respondents, potentially 

dispersing the ratings more and producing a more useful 

range than a 3-point scale would. Though the 5-point scale 

is most common, Lai and Hitchcock (2015) suggested a 

new standard of at least a 7-point scale for subsequent IPA 

strategies. In the interest of continuity and the ability to 

compare NVUM responses through time, the current 5-point 

scale will be retained for NVUM. 

Presenting too many options can lead to choice overload, 

where respondents find it more difficult to make decisions 

or selections (Gourville and Soman 2005). The survey 

must balance asking for sufficiently well-defined ratings of 

attributes on an appropriate Likert scale and preventing rater 

fatigue. For each of the 16 available attributes in NVUM, 

respondents could rate their levels of satisfaction and the 

importance they attached to the attribute. Satisfaction ratings 

were (1) “very dissatisfied,” (2) “somewhat dissatisfied,” (3) 

“neither satisfied nor dissatisfied,” (4) “somewhat satisfied,” 

and (5) “very satisfied,” along with an option to specify 

“not applicable.” Importance ratings ranged from (1) “very 

unimportant” to (5) “very important.” For both scales, the 

value (3) represents a neutral answer.

The original set of attributes in the NVUM satisfaction 

survey are as follows, with abbreviations for easier 

referencing:

1. Adequacy of signage on this forest as a whole 

(TOTALSIGNS)

2. Condition of roads on this forest as a whole 

(TOTALROADS)

3. Scenery at this site/area (SCENERY)

4. Condition of the natural environment (NATENVR)

5. Available parking (PARKING)

6. Parking lot condition (PARKINGLOT)

7. Cleanliness of restrooms (CLEANLINESS)

8. Condition of developed recreation facilities 

(DEVFACILITIES)

9. Condition of forest roads (ROADS)

10. Condition of forest trails (TRAILS)

11. Feeling of safety (SAFETY)

12. Helpfulness of employees (not interviewer) 

(HELPFULNESS)

13. Availability of interpretative/educational displays, 

signs, and exhibits (DISPLAYS)

14. Value for fees paid (for fee sites only) (VALUE)

15. Availability of information on recreation about this site 

(RECINFO)

16. Adequacy of signage to this site (SIGNAGE).

A subset of 11 attributes was selected for IPA analysis: 

NATENVR, SCENERY, SAFETY, TRAILS, VALUE, 

ROADS, PARKING, SIGNAGE, DEVFACILITIES, 

CLEANLINESS, and PARKINGLOT. These are of primary 

interest, and the reduction in the number of attributes 

considered creates a workable set for presentation of results, 

with the procedure in place for further analysis as needed. 

As the analysis is linked to site type, attributes referring to 

the forest signs and roads as a whole were not considered. 

Moreover, each attribute does not necessarily apply to all 

four site types. For instance, how satisfied a recreationist was 

with the developed recreation facilities does not pertain to the 

WILD stratum, which is devoid of developed facilities. Thus, 

attributes analyzed for all four site types were NATENVR, 

SCENERY, SAFETY, TRAILS, and VALUE. Subsequently, 

ROADS, PARKING, and SIGNAGE were analyzed for all but 

the WILD stratum. Last, DEVFACILITIES, CLEANLINESS, 

and PARKINGLOT were evaluated only for the developed 

site types, DUDS and OUDS.

The NVUM survey also contained two measures potentially 

useful in evaluating user satisfaction levels. The basic module 

asked for respondents’ overall satisfaction levels with the visit 

to the NF, from (1) “very dissatisfied” to (5) “very satisfied.” 

The satisfaction module included a 10-point Likert scale for 

crowding, from (1) “hardly anyone” to (10) “overcrowded.” 

Intertemporal analyses were applied to overall satisfaction 

and perceived crowding levels. The results together with IPA 

provided an expanded perspective on recreation, specifically 

from a psychological perspective rather than a physically 

measurable quantity, such as accessible land acreage.

RESULTS

NVUM Sample Characteristics

Sample size should be adequate to reduce variance and 

yield potentially significant results. However, there is also 

a cost tradeoff with increased sampling for accuracy under 

budgetary constraints already in place. NVUM prework 

considers the budgets and the necessary sampling per expertly 

formulated methodologies. In terms of IPA, the minimum 

sample size needed is a function of the number of attributes 

asked in the questionnaire, with a 1:4 recommendation in 

the 1970s, a 1:10 ratio a decade later, and currently 1:20 (Lai 
10 A Temporal Importance-Performance Analysis of Recreation Attributes on National Forests

and Hitchcock 2015). With DUDS and OUDS, we evaluate 

11 attributes, compared to 8 for GFA, and 5 for WILD. Thus, 

the minimum sample by today’s standard would be 220 for 

the developed sites, 160 for GFA, and 100 for WILD. The 

only attribute that does not meet this on a regional and round 

basis for importance and satisfaction is that of VALUE 

in South WILD, with 75 and 91 observations in round 2, 

respectively, and 61 and 78 in round 3. Though the 1:20 ratio 

does not hold, the 1:10 ratio is satisfied. WILD does not have 

as much visitation as the developed sites, which is reflected 

in the sampling’s lower but adequate sample sizes for NVUM 

estimation. Additionally, because fees do not generally apply 

to WILD sites, VALUE is not as relevant to the wilderness 

experience, in contrast to natural aspects such as SCENERY 

and NATENVR. Sampling sizes are more than adequate 

across the site types and rounds when examined on a national 

basis.

Purpose of Visitation

The temporal aspect of sampling is also important in 

evaluating recreationists. To gauge recreationists’ perceptions 

of importance related to attributes, it is better to sample 

before the experience to understand the decisionmaking 

leading up to recreation (Oh 2001). The ratings are then based 

on the recreationists’ preconceived notions of what they are 

looking for from the experience, as well as the standards to 

which they will hold the subsequent trip for recreation. On 

the other hand, post-visit sampling provides insight into their 

satisfaction, namely their perceptions of performance. This 

allows for the respondents to compare their just-concluding 

visit to the framework of expectations with which they 

entered (Lai and Hitchcock 2015, Oh 2001). Due to the 

expense of sampling, conducting both pre- and post-visit 

interviews is not feasible for NVUM. Budget constraints 

already exist on distributing sampling days for onsite 

interviews. An onsite interview of first-entering recreationists 

would only evaluate importance from expectations 

unaffected by the subsequent visit; this setup would not 

measure satisfaction levels without some follow-up by mail, 

Internet, or phone. The post-trip components would require 

additional funds in survey collection, as well as more effort 

from respondents after their trip. Under the NVUM’s LERsampling protocol, the recent nature of the trip promotes more 

accurate recall of satisfaction. Though there may be some loss 

in the integrity of the importance assessment, this potential 

tradeoff is necessary to meet budgetary constraints.

The NVUM sampling prescreens so that respondents are 

only those who used the site/forest for the primary purpose 

of recreation. Respondents are able to further refine their 

responses according to four purposes for the recreation trip 

to the NF: primarily for recreation on this NF, which was the 

main destination; primarily for recreation but with a main 

destination other than this NF; primarily for business, family, 

or other reasons, with this NF visit a side trip; or some other 

reason. Figure 4(A and B) shows that most respondents by 

region and site type were visiting primarily for recreation at 

the NF. The smallest groups were from those who had other 

reasons, followed by those who visited as a side trip while 

traveling due to business, family, or other. Visitors who 

decided on the NF as a primary destination for recreation 

were the majority particularly among the OUDS, WILD, and 

GFA strata at both regional and national levels. 

IPA Results: Temporal Analysis

and Attribute Classifications

In keeping with the simplicity of application, NVUM data 

were analyzed for RPA regional and national comparisons by 

using data-centered traditional IPA and a modified approach 

using the iso-priority line. Due to the high means, the scalecentered approach with thresholds at 3 would not be very 

informative from a relative perspective. The target-driven 

method is also not well-suited to the high means. If the goal 

were to achieve at least some satisfaction on somewhat or 

very important items, then the thresholds for both measures 

would be set at 4. In this case, the relatively high means 

for importance and satisfaction would place all attributes 

in the “good work” (GW) quadrant. Another consideration 

with the target-driven method is that each attribute may 

not correspond to the same goal levels. Therefore, the 

data-centered approach was most appropriate, with grand 

mean importance and satisfaction ratings more effectively 

subdividing the attributes into one of four actions. 

Expression of the Likert-type variables as integers, such as 

a single attribute rating studied for IPA, permits testing but 

with some limitations. Multiple candidate tests for Likerttype variables each have their own statistical issues and 

assumption violations. In many research outlets ranging from 

social science to marketing, discussion arises about whether 

parametric methods may be validly applied to Likert data 

(de Winter and Dodou 2010). Likert-type items are individual 

ratings, whereas a Likert scale is a summation over a group 

of individual ratings (Clason and Dormody 1994). Likerttype items are ordinal, but coding them as numeric assumes 

a discrete distribution with uniform spacing between integer 

ratings. In reality, the distance between “somewhat satisfied” 

and “neither satisfied nor dissatisfied” may not be the same 

as between “somewhat satisfied” and “very satisfied.” 

Additionally, coding importance as 1 to 5 does not necessarily 

equate to the corresponding integer on the satisfaction 

scale (other than the neutral integer of 3). However, coding 

preserves the relative positioning along the scales.
11 A Technical Document Supporting the Update to the Forest Service 2010 RPA Assessment

(A)

0

10

20

30

40

50

60

70

80

90

100

Unweighted percentages

Region and site type

Purpose of Trip to NF: RPA Regions by Site Type for NVUM Round 2

Recreation at NF, main destination Recreation, NF not main destination

Business, family, or other reasons Some other reason

0

10

20

30

40

50

60

70

80

90

100

Unweighted percentages

Region and site type

Purpose of Trip to NF: RPA Regions by Site Type for NVUM Round 3

Recreation at NF, main destination Recreation, NF not main destination

Business, family, or other reasons Some other reason

(B)

Figure 4—Percentages of National Visitor Use Monitoring responses by purpose of the recreation 

visit to the national forest (NF) site for (A) round 2 and (B) round 3. RPA regions: N=North; S=South; 

RM=Rocky Mountain; PC=Pacific Coast. Site types: DUDS=day-use developed sites; GFA=general 

forest areas; OUDS=overnight-use developed sites; WILD=designated Wilderness.
12 A Temporal Importance-Performance Analysis of Recreation Attributes on National Forests

Leeworthy and others (2004) used a temporal comparison 

of IPA scores on 5-point Likert scales with respect to 

25 attributes on natural resources, facilities, and services 

through two-sample t-testing. We also used parametric twosample t-testing on 5-point Likert scales. Future research 

could compare the results obtained from nonparametric 

and parametric methods, but in any case, a precedent in 

parametric methodology would be necessary. Therefore, using 

two-sample t-testing, we integrated a temporal aspect into 

IPA on the NVUM satisfaction module data, constructing a 

framework illustrating IPA scores over two 5-year sampling 

lags.

Because only two complete NVUM rounds were available, 

we begin with a snapshot trend to observe how, or if, the 

attributes change going from round 2 to round 3. With 

two time points, we used the thresholds of round 2 values 

as baselines. This allowed for the observation of trends 

over time, noting possible downslides or improvements by 

attribute. If a trend showed an attribute gradually falling 

into the “concentrate here” (CH) classification, or movement 

from below the iso-priority line to above it, then the temporal 

evaluation allows for advance strategic planning.

The distributions of responses from the NVUM satisfaction 

and importance data were generally left skewed, as there was 

a predominance of ratings of 5, which skewed the weighted 

means upwards. This consistent phenomenon is illustrated in 

figure 5, which shows the frequency of weighted responses 

for DUDS in the North. Figures 6 to 9 represent the results 

for WILD, GFA, DUDS, and OUDS, respectively, by RPA 

region for rounds 2 to 3. It is important to note the scale of 

the axes, as the ranges are a relatively narrow segment of the 

5-point scale. Therefore, the overall NF recreation experience, 

as captured by the attributes, was rated very high in terms 

of performance and importance. Nevertheless, we computed 

relative comparisons to potentially inform managers in 

resource allocation, and to identify potential trend shifts from 

round 2 to round 3. Because managerial implications take 

place by site type, the analysis examined the implications of 

the attributes by site type and then scanned for commonalities 

across regions and the Nation.

An important note is that NVUM surveys are administered 

through a complex, multistage sampling design on strata. 

Prework allocates sampling days for each forest, depending 

not only on site type but also use level (i.e., low, medium, 

high, or very high). This work describes a basic application 

of IPA on respondents by site type, as if the surveys were 

obtained through simple random sampling, rather than 

addressing the underlying stratifications on both site type 

and use level. The more complex analysis would potentially 

be a future research direction, with this simplified analysis in 

place for comparison. 

We next present the results by site type for WILD, GFA, 

DUDS, and OUDS: first by the quadrants approach, then by 

use of the more stringent iso-priority line. Some attributes are 

Figure 5—Weighted frequencies (by inverse annual number of trips) of satisfaction and importance 

ratings on condition of the natural environment for North day-use developed sites (DUDS) in round 2.

0

100

200

300

400

500

600

700

800

1234 5

Weighted frequencies

Likert scale rang

Condition of the Natural Environment (at Site) Ratings Round 2,

North Region DUDS

Sasfacon Importance
13 A Technical Document Supporting the Update to the Forest Service 2010 RPA Assessment

classified differently with the latter approach compared to the 

quadrants approach. Though attributes below the iso-priority 

line do not change classifications, those above the diagonal 

line may be classified as “keep up the good work” (GW) 

under the quadrants approach but shift to “concentrate here” 

(CH). The iso-priority line operates under the assumption 

that the recreationists’ satisfaction should at least match 

their expectations (i.e., satisfaction should match or exceed 

preconceived importance levels).

Designated Wilderness 

For the WILD site types at the national and RPA regional 

levels, five site-specific attributes were analyzed: NATENVR 

(condition of the natural environment), SCENERY (scenery 

at this site/area), SAFETY (feeling of safety), TRAILS 

(condition of forest trails), and VALUE (value for fee paid, 

if applicable). The complete satisfaction module from the 

NVUM questionnaire is available on request from the 

authors.

Attribute classifications for WILD—Diagrammatic IPA 

results for both the quadrants approach and the iso-priority 

line approach are shown in figure 6. The quadrants approach 

indicated that none of the attributes fell squarely into the 

CH section in either time period. In fact, the two attributes 

reflecting SCENERY and NATENVR, perhaps most 

applicable to the character of wilderness, remained in the 

GW quadrant in both time periods, with apparent movement 

toward increased satisfaction over time, except for SCENERY 

in the South. Statistically significant gains (table 2) occurred 

in satisfaction alongside generally static importance in both 

attributes for the Rocky Mountain and Pacific Coast regions 

and the Nation. For the attribute of SCENERY, importance 

ratings increased significantly in the North region and for 

the Nation; North satisfaction levels remained static with 

increasing levels of importance between rounds.

The IPA scores for TRAILS remained in the LP quadrant 

through both time periods for the Nation and all regions 

except for the Rocky Mountain, despite increases in both 

importance and satisfaction for the Rocky Mountain region. 

A further increase in importance relative to satisfaction 

could conceivably push TRAILS into the CH quadrant in 

subsequent time periods. A statistically significant decline 

occurred in the satisfaction levels of TRAILS for the South 

with static importance, the only significant downturn 

observed for the WILD site type. Growth in both satisfaction 

and importance of TRAILS ratings occurred for the Rocky 

Mountain region, along with increasing importance and static 

satisfaction for the Pacific Coast and Nation.

Two attributes more closely associated with human use than 

nature are SAFETY and VALUE. In the case of VALUE, 

a quadrant shift from LP to PO occurred for the North and 

Rocky Mountain regions and Nation. Significant increases 

in both satisfaction and importance over time were observed 

for VALUE for the North and the Nation. These results 

suggest that visitors continued to be very pleased with their 

wilderness recreation experience relative to any site fees 

that were encountered. SAFETY, a very high priority in the 

Forest Service at all levels, appeared in the PO quadrant 

for both time periods, with the exception of the South. 

Both importance and satisfaction improved from round 2 

to round 3 for the Rocky Mountain region and the Nation; 

increasing satisfaction with static importance occurred for the 

attribute in the North. Should importance increase relative to 

satisfaction by the next round, this attribute could approach 

the GW quadrant in all regions and the Nation.

An alternative perspective at the national level using the 

iso-priority line approach indicated that four of the five 

attributes are not in need of attention, as their satisfaction 

ratings exceeded their importance ratings. For NATENVR, 

the rating importance exceeded satisfaction, indicating that 

some concern may be warranted. However, the temporal 

movement across periods showed an increase in satisfaction 

in NATENVR but no change in importance. Overall, for 

wilderness visitors on NFs, it is important to note that at the 

national level, all wilderness attributes showed increases in 

satisfaction ratings, and four of the five showed increases in 

importance ratings.

Intertemporal analysis for WILD—Attributes with 

increased satisfaction and importance in round 3 for WILD 

were SCENERY for the Pacific Coast and the Nation, 

SAFETY for the Rocky Mountain region and the Nation, 

TRAILS in the Rocky Mountain region, and VALUE in the 

North and the Nation. The largest growth in satisfaction for 

WILD occurred in VALUE for the North, shown in the shift 

from LP to the threshold between GW and PO; VALUE was 

also the attribute with the greatest growth in importance. 

Recreationists using North WILD were very satisfied with 

their experience relative to any fees they paid, and this 

attribute became increasingly important.

The only statistically significant downturn in satisfaction for 

WILD occurred in the South for TRAILS (table 2). Though 

this result did not necessarily signal an immediate concern 

for TRAILS, the trend could point to a future need for 

maintenance or improvement if importance increases.
14 A Temporal Importance-Performance Analysis of Recreation Attributes on National Forests

Figure 6—Importance-performance analysis for designated Wilderness (WILD) with weighting by inverse annual number of trips 

on five attributes across the four RPA regions and the Nation, from round 2 to round 3, with quadrants and hybrid iso-priority line 

approaches.

4.2 4.4 4.6 4.8

4.2 4.4 4.6 4.8

RPA Region: North

Satisfaction Rating

Importance Rating

2 3

2

3

2 3

2 3

2

3

Site Type WILD

4.2 4.4 4.6 4.8

4.2 4.4 4.6 4.8

RPA Region: South

Satisfaction Rating

Importance Rating

2 3 2 3

2 3

2

3 2 3

4.2 4.4 4.6 4.8

4.2 4.4 4.6 4.8

RPA Region: Rocky Mountain

Satisfaction Rating

Importance Rating

2 3 2 3

2

3

2

3

2

3

4.2 4.4 4.6 4.8

4.2 4.4 4.6 4.8

RPA Region: Pacific Coast

Satisfaction Rating

Importance Rating

2 3

2 3

23 2

3

2 3

4.2 4.4 4.6 4.8

4.2 4.4 4.6 4.8

Nation

Satisfaction Rating

Importance Rating

2 3 2 3

2 3

2

3

2

3

ISO−RATING LINE

MEAN

NATENVR

SCENERY

SAFETY

TRAILS

VALUE
15 A Technical Document Supporting the Update to the Forest Service 2010 RPA Assessment

Table 2—Rates of change between weighted means for satisfaction (satis) and importance (import) between rounds 

across Resources Planning Act regions and the Nation for designated Wilderness (WILD), with significance determined 

by parametric two-sample t-testing

WILD

North South Rocky Mountain Pacific Coast Nation

Attribute Satis Import Satis Import Satis Import Satis Import Satis Import

NATENVR 1.17% -0.37% 1.11% -0.82% 1.88%** 0.18% 2.28%** 0.09% 1.85%** -0.03%

SCENERY 0.57% 2.07%** -1.06% 0.71% 0.80%** 0.28% 1.33%** 0.82% 0.73%** 0.70%**

SAFETY 1.90%** -0.46% 1.36% -0.64% 1.33%** 2.44%** -0.21% -0.47% 0.94%** 0.76%

TRAILS -1.18% -0.56% -5.15%** -1.12% 2.18%** 1.55%** 0.81% 1.67%** 0.50% 0.93%**

VALUE 8.50%** 5.12%* -1.33% 0.90% 1.34% 2.13% 1.71% 0.47% 2.14%** 1.75%*

Attributes: NATENVR=Condition of the natural environment; SCENERY=Scenery at this site/area; SAFETY=Feeling of safety; 

TRAILS=Condition of forest trails; VALUE=Value for fees paid (for fee sites only).

*Statistically significant for α = 0.10

**Statistically significant for α = 0.05

Positive rate of change (significant)

Negative rate of change (significant)

Following are summarized managerial implications from the 

data-centered quadrants approach for the RPA regions and the 

Nation in the WILD site type. Borderline results are indicated 

by regions in italics.

Concentrate here (Relatively high importance, relatively 

low satisfaction)

• None

Keep up the good work (Relatively high importance, 

relatively high satisfaction)

• Scenery at this site/area (All)

• Condition of the natural environment (All)

• Feeling of safety (South and Rocky Mountain)

Low priority (Relatively low importance, relatively low 

satisfaction)

• Condition of forest trails (North, South, Pacific Coast, 

and Nation)

• Value for fee paid (Pacific Coast)

Possible overkill (Relatively low importance, relatively 

high satisfaction)

• Feeling of safety (North, Pacific Coast, and Nation)

• Value for fee paid (North, South, Rocky Mountain, and 

Nation)

• Trails (Rocky Mountain)

Potential shifts to area of concern per iso-priority line 

(i.e., satisfaction did not equal or exceed perceived 

importance)

• Scenery at this site/area (South)

• Condition of the natural environment (North, South, 

Pacific Coast, and Nation)

• Condition of forest trails (South)

General Forest Areas 

In addition to the five attributes analyzed for WILD, the 

analysis of GFA incorporated ROADS (condition of forest 

roads), PARKING (availability of parking), and SIGNAGE 

(adequacy of signage to this site) (fig. 7). These additional 

attributes explain how visitors using a vehicle perceived 

navigating and arriving at the site.

Attribute classifications for GFA—When the quadrants 

method was used, none of the attributes fell squarely into the 

CH classification. Some of the attributes produced results 

close to the thresholds; for example, TRAILS in the Rocky 

Mountain region and the Nation shifted from LP to just inside 

the CH classification. If importance were to increase relative 

to satisfaction for the next round, this attribute could be of 

concern, especially considering that trails are a part of the 

recreation experience for many activities on GFA.

For GFA, the NATENVR, SCENERY, and SAFETY 

attributes performed the best overall, appearing in the GW 

area. These results indicated that visitors felt safe overall 

and experienced fulfilling natural conditions for recreation. 

If importance declines significantly relative to satisfaction 

into round 4, the attributes for the South could shift to 

PO. SCENERY tended to surpass the other two attributes 

in satisfaction; NATENVR was associated with greater 

importance. Statistically significant gains in satisfaction 

along with static importance occurred in NATENVR and 

SCENERY for the Pacific Coast, SCENERY for the Nation, 

and SAFETY for the North (table 3). Visitors in these 

regions were more satisfied with the respective attributes in 

round 3 than in round 2, while attaching the same levels of 

importance temporally. SAFETY became a more important 

attribute with static satisfaction in the Rocky Mountain 

region. The only significant declines among these three 
16 A Temporal Importance-Performance Analysis of Recreation Attributes on National Forests

Figure 7—Importance-performance analysis for general forest areas (GFA) with weighting by inverse annual number of trips on eight 

attributes across the four RPA regions and the Nation, from round 2 to round 3, with quadrants and hybrid iso-priority line approaches.

4.2 4.4 4.6 4.8

4.2 4.4 4.6 4.8

RPA Region: North

Satisfaction Rating

Importance Rating

2

3

22 3

3

2 3

2

3

2

3 2

3

2

3

Site Type GFA

4.2 4.4 4.6 4.8

4.2 4.4 4.6 4.8

RPA Region: South

Satisfaction Rating

Importance Rating

2 3 2 3

2 3

23 2

3 2

3 2

3

2

3

4.2 4.4 4.6 4.8

4.2 4.4 4.6 4.8

RPA Region: Rocky Mountain

Satisfaction Rating

Importance Rating

3 2 2

3

2

3

2

3

2

3

2

3

2

3

2 3

4.2 4.4 4.6 4.8

4.2 4.4 4.6 4.8

RPA Region: Pacific Coast

Satisfaction Rating

Importance Rating

2

2 3 2 3

23

2 3

23

2 3

2

3

3

4.2 4.4 4.6 4.8

4.2 4.4 4.6 4.8

Nation

Satisfaction Rating

Importance Rating

32 23

23

2

3

2 3

2 3

2

3

2

3

VALUE

NATENVR

SCENERY

SAFETY

TRAILS

ISO−RATING LINE

MEAN

ROADS

PARKING

SIGNAGE
17 A Technical Document Supporting the Update to the Forest Service 2010 RPA Assessment

Table 3—Rates of change between weighted means for satisfaction (satis) and importance (import) between rounds 

across Resources Planning Act regions and the Nation for general forest areas (GFA), with significance determined by 

parametric two-sample t-testing

GFA

North South Rocky Mountain Pacific Coast Nation

Attribute Satis Import Satis Import Satis Import Satis Import Satis Import

NATENVR -0.30% 0.92% -0.90% -0.62% -0.92%** -0.34% 0.86%* -0.14% -0.38% -0.12%

SCENERY 0.69% 1.01% -0.72% -0.64% 0.07% 0.44% 0.94%** -0.51% 0.31%* 0.13%

SAFETY 0.98%* -0.69% 0.72% -0.83% -0.19% 0.79%* 0.54% 0.20% 0.26% 0.22%

TRAILS -1.46% -0.40% -0.08% -0.40% 0.35% 2.35%** 1.75%** 0.91%* 0.47% 1.06%**

VALUE -2.20% 2.24% -1.84% -2.02%* 0.69% 2.17%** -0.33% 0.39% -0.77% 0.62%

ROADS -1.69% 2.21%* -1.09% -1.25%* -0.43% 1.15%** -1.44%* 0.86% -0.93%** 0.88%**

PARKING -0.41% 1.63% 2.18%** -1.69%** -0.67% 2.15%** 0.24% 1.28%* -0.08% 1.35%**

SIGNAGE -1.91% -1.62% 1.81% -2.94%** 0.62% 0.44% -0.36% -0.53% 0.09% -0.66%*

Attributes: NATENVR=Condition of the natural environment; SCENERY=Scenery at this site/area; SAFETY=feeling of safety; 

TRAILS=Condition of forest trails; VALUE=Value for fees paid (for fee sites only); ROADS=Condition of forest roads; PARKING=Available 

parking; SIGNAGE=Adequacy of signage to this site.

*Statistically significant for α = 0.10

**Statistically significant for α = 0.05

Positive rate of change (significant)

Negative rate of change (significant)

attributes were in satisfaction with NATENVR for the Rocky 

Mountain region, alongside static importance, though this 

downturn was not sufficient to leave the GW quadrant.

ROADS, PARKING, and SIGNAGE are attributes that 

span the physical, informational, or directional aspects of 

visitation. These attributes fell into the LP quadrant across 

all regions. SIGNAGE tended to have the lowest satisfaction, 

while PARKING and ROADS showed the lowest levels of 

importance. Declining satisfaction levels coupled with static 

or growing importance occurred for ROADS in the Pacific 

Coast region and the Nation, but these changes were not 

yet problematic given the low means. Satisfaction levels for 

PARKING significantly increased with declining importance 

for the South. 

TRAILS could be in need of work (CH) for the North and 

Rocky Mountain regions, along with a borderline result 

for the Nation. TRAILS fell into the LP quadrant for the 

South through round 3. If importance increases relative to 

satisfaction into round 4 for these regions, then attention may 

be needed in improving the condition of forest trails. This 

attribute was classified as PO for the Pacific Coast in round 3; 

importance and satisfaction increased for that region as well 

as for the Nation. Importance became significantly greater 

(with static satisfaction) for TRAILS for the Rocky Mountain 

region between rounds, which shifted the attribute to the 

CH quadrant. 

The VALUE attribute was of low priority for the South 

and Pacific Coast regions and the Nation, and borderline 

for the North and Rocky Mountain regions. In the North, 

VALUE appeared on the threshold dividing the CH and LP 

quadrants. This finding may warrant attention should the 

trend persist to round 4. Visitors to the South attached less 

importance to VALUE in round 3 than in round 2. In contrast, 

recreationists in the Rocky Mountain region perceived greater 

importance. Overall, however, VALUE remained higher in 

both importance and satisfaction for the South than for the 

other regions.

Because the iso-priority line uses distances as a measure of 

relative attention needed, the results within the interior of the 

CH section could require more attention than the borderline 

results. However, budgetary considerations permitting, the 

borderline attributes could warrant monitoring. The isopriority line indicated that NATENVR and SIGNAGE may 

need attention across the Nation and all RPA regions, with 

a borderline result for the North in NATENVR, so that 

satisfaction at least matches the importance levels. Placement 

of attributes within the interior of the CH section indicated 

that TRAILS could need attention in the North and South, as 

well as VALUE in the South and ROADS in the South and 

Rocky Mountain regions and the Nation. Attributes appearing 

in the CH quadrant but near the iso-priority line indicated 

borderline results: SAFETY for the South, VALUE for the 

North and Pacific Coast regions, ROADS for the Pacific 

Coast, TRAILS for the Nation, and PARKING for the South 

(borderline). The iso-priority line had the greatest impact in 

reclassifying attributes for the South, which has less variation 

in importance than the other regions.

Intertemporal analysis for GFA—Satisfaction in 

GFA had the greatest growth in PARKING for visitors 

to the South, though in conjunction with declining 
18 A Temporal Importance-Performance Analysis of Recreation Attributes on National Forests

importance (table 3). TRAILS had the greatest growth in 

importance for the Rocky Mountain GFA, with unchanging 

satisfaction levels and a shift from LP to CH. In none of 

the regions or site types did satisfaction and importance 

simultaneously increase significantly for GFA.

Changes that may warrant concern occurred for VALUE 

in the North, and ROADS in the North and the Nation. The 

VALUE attribute in the North may be the most in need 

of consideration, given the shift from PO to the threshold 

close to the CH quadrant. NATENVR declined slightly in 

satisfaction for the South and Rocky Mountain regions and 

the Nation amid stable importance. This decrease is not yet 

problematic given the consistent placement of the means 

into the GW quadrant, but it could call for understanding 

why satisfaction declined between rounds. Also declining 

in satisfaction with stable importance were SCENERY 

in the South, VALUE in the Nation, and ROADS in the 

Pacific Coast region. VALUE in the South and SIGNAGE 

in the North declined in satisfaction, but did not warrant 

concern due to the co-occurring loss in importance. Visitors 

recreating in the South perceived safety and conditions 

of forest roads as less important amid sustained levels of 

satisfaction since round 2.

Following are summarized managerial implications for the 

RPA regions and the Nation in the GFA site type. Borderline 

results are indicated by regions in italics.

Concentrate here (Relatively high importance, relatively 

low satisfaction)

• Condition of forest trails (North and Rocky Mountain)

Keep up the good work (Relatively high importance, 

relatively high satisfaction)

• Condition of the natural environment (All)

• Scenery at this site/area (All)

• Feeling of safety (All)

Low priority (Relatively low importance, relatively low 

satisfaction)

• Adequacy of signage to site (All)

• Value for fee paid (South, Rocky Mountain, Pacific Coast, 

and Nation)

• Condition of forest trails (South and Nation)

• Available parking (All)

• Condition of forest roads (All)

Possible overkill (Relatively low importance, relatively 

high satisfaction)

• Condition of forest trails (Pacific Coast)

Indeterminate due to proximity to thresholds

• Value for fee paid (North)

• Condition of forest trails (North and Nation)

Potential shifts to area of concern per iso-priority line 

(i.e., satisfaction did not equal or exceed perceived 

importance)

• Condition of the natural environment (North, South, 

Rocky Mountain, Pacific Coast, and Nation)

• Adequacy of signage to site (All)

• Condition of forest trails (North and South)

• Value for fee paid (North, South, and Pacific Coast)

• Condition of forest roads (South, Rocky Mountain, 

Pacific Coast, and Nation)

• Feeling of safety (South)

Day-Use Developed Sites (DUDS)

Analyses of DUDS covered all 11 attributes. In addition 

to those for GFA, developed facilities (DEVFACILITIES), 

cleanliness of restrooms (CLEANLINESS), and condition of 

parking lot (PARKINGLOT) were analyzed (fig. 8). These 

additional attributes explain how visitors perceived elements 

that are relevant to developed sites but not applicable to 

WILD or GFA.

Attribute classifications for DUDS—NATENVR, 

SCENERY, and SAFETY all appeared in the GW quadrant 

for both rounds. Visitors to DUDS for all regions and the 

Nation perceived a safe and fulfilling experience in nature. 

The Pacific Coast performed well temporally, achieving gains 

in satisfaction on all three attributes. The perception of safety 

also had gains in satisfaction for all but the South. Declines 

in satisfaction occurred for NATENVR in the South and 

Rocky Mountain regions, but the attribute remained in the 

GW quadrant.

SIGNAGE may need improvement (CH) for all regions, 

with borderline results for the Rocky Mountain and Pacific 

Coast regions and the Nation. Visitors in the North and 

Nation were more satisfied with SIGNAGE in round 3, while 

viewing it with the same levels of importance; in the South, 

importance increased with unchanged satisfaction (table 4). 

These changes either maintained or shifted placement of the 

attribute into the CH interior. 

Likewise, CLEANLINESS may need attention (CH) across 

all regions and the Nation, demonstrating the overall lowest 

satisfaction levels, especially in the North, Rocky Mountain, 

and Pacific Coast regions and the Nation. The attribute 

became more important to visitors in round 3 in the South 

and Pacific Coast, with satisfaction levels unchanged in the 

former and greater in the latter. The progression in both 
19 A Technical Document Supporting the Update to the Forest Service 2010 RPA Assessment

Figure 8—Importance-performance analysis for day-use developed sites (DUDS) with weighting by inverse annual number of trips on 

11 attributes across the 4 RPA regions and the Nation, from round 2 to round 3, with quadrants and hybrid iso-priority line approaches.

Site Type DUDS

VALUE

NATENVR

SCENERY

SAFETY

TRAILS

ROADS

PARKING

ISO−RATING LINE

MEAN

SIGNAGE

DEVFACILITIES

CLEANLINESS

PARKINGLOT

4.2 4.4 4.6 4.8

4.2 4.4 4.6 4.8

RPA Region: North

Satisfaction Rating

Importance Rating

23

2 32

3

2

3

2

3

2

3 2

3

2 3

23

32

2

3

4.2 4.4 4.6 4.8

4.2 4.4 4.6 4.8

RPA Region: South

Satisfaction Rating

Importance Rating

3 2

322

3

2

3 2

3

2

3 3

2

3

3 2

2

3

2

3

4.2 4.4 4.6 4.8

4.2 4.4 4.6 4.8

RPA Region: Rocky Mountain

Satisfaction Rating

Importance Rating

32 32

2 3

2 2 3

3

2 3

2 3

23

2

3

2 3

2

3

4.2 4.4 4.6 4.8

4.2 4.4 4.6 4.8

RPA Region: Pacific Coast

Satisfaction Rating

Importance Rating

2 3

23 2 3

2

3 2

3

2

3

2

3

2 3

2

3

2 3

2

3

4.2 4.4 4.6 4.8

4.2 4.4 4.6 4.8

Nation

Satisfaction Rating

Importance Rating

32 32

2 3

2

3 2

3

2

3

2

3

2 3

2

3

2 3

2

3
20 A Temporal Importance-Performance Analysis of Recreation Attributes on National Forests

satisfaction and importance in the Pacific Coast region may 

indicate a positive trend in CLEANLINESS, but with the low 

placement of the means in round 2, the increase in satisfaction 

was insufficient to declassify this attribute as CH. Gains 

in satisfaction occurred in the Rocky Mountain and Pacific 

Coast regions and the Nation on static importance, but the 

attribute remained in the CH quadrant even to round 3 due to 

the low round 2 means.

Additionally, VALUE belonged to the CH quadrant for 

the North and Rocky Mountain regions and the Nation, 

while performing well (GW) in the South and just into the 

LP quadrant for the Pacific Coast region. This attribute 

had less importance for visitors in the Pacific Coast in 

round 3 than in round 2, while sustaining the same levels of 

satisfaction. Because of the proximity of the round 2 mean 

to the thresholds for the Pacific Coast region, the decline of 

importance shifted the attribute from CH to LP. All other 

regions and the Nation reported greater perceptions of 

importance in round 3; satisfaction was unchanged except for 

the South, where VALUE moved into the GW quadrant. 

TRAILS could need improvement (CH) in the North and 

South; the attribute performed well (GW) in the Pacific 

Coast region and produced indeterminate results for the 

Rocky Mountain region and the Nation. Visitors to the South 

perceived a decline in the condition of forest trails between 

rounds, while rating the attribute as more important than in 

round 2. This problematic trend of declining satisfaction on 

an increasingly important attribute could signal managerial 

implications, especially for the South. Recreationists 

in the Pacific Coast region and visitors nationally saw 

improvements in both satisfaction and importance for 

trail conditions in round 3, but the round 2 means and the 

magnitudes of increases meant that classifications changed 

for both regions. Because of the proximity to the threshold in 

round 2, this intertemporal shift for the Pacific Coast meant 

that the attribute was in the GW quadrant by the next round. 

The national gains were not as strong, placing the attribute 

near the threshold bordering the GW quadrant. Visitors to 

Rocky Mountain DUDS perceived greater satisfaction while 

importance stayed at or near round 2 levels; this gain shifted 

the attribute closer to the thresholds, potentially classifying 

it as GW or PO depending on progressions to round 4. The 

shift into CH for the North occurred due to the increase in 

importance (over unchanging satisfaction) on a mean that was 

close to the thresholds in round 2.

In the PO section across all areas were PARKING (with a 

borderline result for South) and PARKINGLOT. Visitors 

generally attached the lowest importance to PARKINGLOT, 

although they were generally highly satisfied. In the Rocky 

Mountain and Pacific Coast regions and the Nation, the 

attribute had gains not only in satisfaction but also in 

importance. However, the significant increases in satisfaction 

observed in round 3 failed to offset the low starting mean 

in round 2. Visitors to North DUDS attached greater 

importance to parking lot conditions while satisfaction 

remained the same. Again, the round 2 placement deep into 

the PO quadrant meant that a shift into another quadrant 

Table 4—Rates of change between weighted means for satisfaction (satis) and importance (import) between rounds across 

Resources Planning Act regions and the Nation for day-use developed sites (DUDS), with significance determined by 

parametric two-sample t-testing

DUDS

North South Rocky Mountain Pacific Coast Nation

Attribute Satis Import Satis Import Satis Import Satis Import Satis Import

NATENVR 0.15% 0.01% -1.54%** 0.06% -0.48* -0.04% 1.00%** 0.17% -0.09% 0.07%

SCENERY 0.00% 0.00% -0.67% -0.18% -0.08% -0.17% 0.38%* 0.03% -0.02% -0.10%

SAFETY 0.76%** -0.65% 0.33% 0.58% 0.92%** -0.33% 0.79%** 0.18% 0.77%** -0.10%

TRAILS 0.63% 1.26%* -2.39%** 8.63%** 1.20%** 0.44% 2.71%** 2.22%** 1.14%** 2.18%**

VALUE -0.94% 3.90%** 1.20%* 4.33%** 0.42% 2.24%** -0.54% -1.29%** -0.20% 1.45%**

ROADS 1.58%** 1.30% -0.06% 6.61%** 0.69% 0.54% 0.98%* 2.29%** 0.74%** 1.98%**

PARKING 0.32% 0.74% -0.25% 0.05% 0.98%** 0.88%** 0.02% 1.27%** 0.37%* 0.84%**

SIGNAGE 3.08%** -0.31% 0.17% 2.60%** 0.18% 0.13% 0.67% -0.43% 0.65%** 0.22%

DEVFACILITIES 0.43% -0.34% -0.70% -0.03% 1.96%** 1.28%** 2.49%** 2.50%** 1.38%** 1.10%**

CLEANLINESS -0.08% -0.30% -0.73% 2.15%** 2.10%** -0.22% 3.88%** 0.83%* 1.63%** 0.33%

PARKINGLOT 0.25% 3.09%** 1.03%** -1.26%* 1.15%** 2.23%** 1.05%** 1.27%** 0.92%** 1.41%**

Attributes: NATENVR=Condition of the natural environment; SCENERY=Scenery at this site/area; SAFETY=Feeling of safety; TRAILS=Condition of 

forest trails; VALUE=Value for fees paid (for fee sites only); ROADS=Condition of forest roads; PARKING=Available parking; SIGNAGE=Adequacy 

of signage to this site; DEVFACILITIES=Condition of developed recreation facilities; CLEANLINESS=Cleanliness of restrooms; 

PARKINGLOT=Parking lot condition.

*Statistically significant for α = 0.10

**Statistically significant for α = 0.05

Positive rate of change (significant)

Negative rate of change (significant)
21 A Technical Document Supporting the Update to the Forest Service 2010 RPA Assessment

did not occur. However, should the increases in importance 

continue, then the attribute may belong to the GW quadrant 

by round 4 or beyond. The attribute moved even deeper into 

the PO quadrant in the South due to declining importance and 

rising satisfaction.

DEVFACILITIES was classified as PO for the Rocky 

Mountain and Pacific Coast regions and the Nation, and 

as LP elsewhere. Only the Rocky Mountain and Pacific 

Coast regions and the Nation had significant increases in 

visitor satisfaction and importance for developed facilities. 

Coupled with the low means in round 2, these gains in the 

two regions and the Nation meant a progression from LP to 

PO. Should importance continue to improve (with increasing 

or static satisfaction), this attribute could perform well (GW) 

to round 4. Visitors to North and South DUDS did not vary 

significantly in their ratings from round 2 to round 3, so the 

proximity to the thresholds contributes to the uncertainty of 

the classification in the next round.

ROADS appeared as LP for all but the South, where it could 

need attention. The transition from LP to CH for the South 

stemmed from significant increases in importance on static 

satisfaction levels. Pacific Coast and national DUDS visitors 

perceived greater satisfaction and importance levels for 

ROADS by round 3, but the gains failed to offset the impact 

of round 2 means being squarely in the LP quadrant. Visitors 

to the North DUDS were more satisfied with conditions of the 

roads by round 3 without significant changes in importance. 

In the South, visitors saw the attribute as significantly more 

important, though with unchanging satisfaction levels from 

round 2.

Factoring in the iso-priority line meant that NATENVR could 

be in need of attention (CH) for all areas, with borderline 

results for the North and Rocky Mountain regions and the 

Nation. This indicates that even in the developed areas, 

the quality of the surrounding natural environment is 

increasingly important to recreation visitors. The diagonal 

also indicated SIGNAGE and CLEANLINESS as needing 

attention (CH) for all regions and the Nation. VALUE 

could also need work for the Nation and all regions but the 

South, with borderline results for the North and Nation. The 

iso-priority line also affirmed that TRAILS and ROADS 

(borderline) in the South could merit attention as importance 

equaled or exceeded satisfaction. 

Intertemporal analysis for DUDS—Overall, satisfaction 

increased for many attributes for DUDS in the Rocky 

Mountain and Pacific Coast regions and the Nation. The 

greatest gains in satisfaction occurred for CLEANLINESS in 

the Pacific Coast; while still in the CH quadrant, the attribute 

shifted in a positive direction closer to the GW section.

For DUDS, declines in satisfaction occurred for NATNEVR 

in the South (where it was most pronounced) and Rocky 

Mountain regions (table 4). If the satisfaction continues to 

decline with static or increasing importance in the South, 

then management may need to evaluate the loss in fulfillment 

from the natural environment. TRAILS in the South could 

also warrant attention because of declining satisfaction and 

increasing importance. VALUE became less important in 

round 3 for the Pacific Coast. This result was also observed 

for PARKINGLOT in the South. 

Following are summarized managerial implications for 

the RPA regions and the Nation in the DUDS site type. 

Borderline results are indicated by regions in italics.

Concentrate here (Relatively high importance, relatively 

low satisfaction)

• Cleanliness of restrooms (All)

• Adequacy of signage to this site (North, South, Rocky 

Mountain, Pacific Coast, and Nation)

• Value for fees paid (North, Rocky Mountain, and Nation)

• Condition of forest roads (South)

• Condition of forest trails (North and South)

Keep up the good work (Relatively high importance, 

relatively high satisfaction)

• Condition of the natural environment (All)

• Scenery at this site/area (All)

• Feeling of safety (All)

• Value for fee paid (South)

• Condition of forest trails (Pacific Coast)

Low priority (Relatively low importance, relatively low 

satisfaction)

• Condition of forest roads (North, Rocky Mountain, 

Pacific Coast, and Nation)

• Condition of developed recreation facilities (North and 

South)

• Value for fee paid (Pacific Coast)

Possible overkill (Relatively low importance, relatively 

high satisfaction)

• Parking lot condition (All)

• Available parking (North, South, Rocky Mountain, 

Pacific Coast, and Nation)

• Condition of developed recreation facilities (Rocky 

Mountain, Pacific Coast, and Nation)
22 A Temporal Importance-Performance Analysis of Recreation Attributes on National Forests

Indeterminate due to proximity to thresholds

• Condition of forest trails (Rocky Mountain and Nation)

Potential shifts to area of concern per iso-priority line 

(i.e., satisfaction did not equal or exceed perceived 

importance)

• Cleanliness of restrooms (All)

• Adequacy of signage to site (All)

• Value for fee paid (North, Rocky Mountain, Pacific 

Coast, and Nation)

• Condition of forest trails (South)

• Condition of forest roads (South)

• Condition of the natural environment (North, South, 

Rocky Mountain, Pacific Coast, and Nation)

Overnight-Use Developed Sites

The 11 attributes assessed for DUDS were also examined for 

the second developed site type, OUDS. Users of this site type 

factored in their experience with the developed sites, although 

in this case “developed facilities” includes facilities related to 

overnight accommodations, whether by lodging, camping, or 

recreational vehicle (fig. 9).

Attribute classifications for OUDS—Performing well 

(GW) were the NATENVR, SCENERY, and SAFETY 

attributes, which are integral to a secure and satisfying 

recreation experience. These results were consistent with 

findings for the other three site types examined. Satisfaction 

improved with static importance for all three attributes in 

the Pacific Coast region, as well as for NATENVR in the 

North and SAFETY in the Nation (table 5), preserving the 

GW classification to round 3. Visitors perceived a greater 

importance in round 3 with unchanging satisfaction levels 

for NATENVR in the South and Nation and SAFETY in 

the South, pushing the results farther into the GW interior. 

Positive trends for both satisfaction and importance occurred 

for SCENERY for visitors in the South. Satisfaction declined 

for visitors between rounds 2 and 3 in the Rocky Mountain 

region for NATENVR and SCENERY, and NATENVR 

approached the threshold dividing the GW and CH quadrants. 

If this trend continues to round 4, the conditions contributing 

to declining satisfaction may warrant a closer look and 

possible reallocation of resources. 

The quadrants approach indicated that CLEANLINESS needs 

improvement across all regions and the Nation, often having 

the lowest satisfaction means but relatively high importance 

scores, especially in the South and Pacific Coast regions. 

Nationally and in the South, visitors viewed cleanliness 

of restrooms with greater importance by round 3, while 

maintaining satisfaction levels; the South reported the largest 

increase in importance. In the Pacific Coast region, visitors 

were more satisfied with the attribute by round 3, though 

importance was static. Regardless, the attribute remained in 

the CH quadrant due to the starting point in round 2. 

SIGNAGE appeared in the CH section for the North 

(borderline), South and Rocky Mountain regions, and 

the Nation (borderline), while being low priority for the 

Pacific Coast. Adequacy of signage was one of the lowerrated attributes for satisfaction; the importance by region 

determined whether this attribute was classified as CH or LP. 

Visitors in round 3 found SIGNAGE more important with 

unchanged satisfaction in the South and Rocky Mountain 

regions and the Nation, resulting in a shift to the CH 

quadrant. Recreationists in the North viewed SIGNAGE with 

improved satisfaction levels alongside static importance. 

The shift from LP to CH for the North was largely due to the 

proximity to the thresholds in round 2 and the slight (but not 

statistically significant) increase in importance.

TRAILS could require maintenance (CH) for the South and 

possibly for the North, while having low priority or being 

overkill in other regions. The shift of greatest concern for 

TRAILS occurred in the South, with declining satisfaction 

and increasing importance to round 3. The attribute improved 

in both satisfaction and importance for the Rocky Mountain 

region and the Nation, with a classification of LP, though 

shifts could occur depending on the trends to round 4. 

Visitors to the Pacific Coast were more satisfied with trail 

conditions in round 3 coupled with unchanged importance, 

thereby yielding the PO classification for TRAILS.

VALUE fulfilled expectations (GW) for the North, South, 

and Pacific Coast (borderline) regions and the Nation, with 

attention possibly needed (CH) in the Rocky Mountain region. 

The visitors in the North and South in round 3 found greater 

satisfaction on VALUE, which likewise gained importance. 

Nationally, by round 3, increasing satisfaction on VALUE 

along with static importance shifted the means from near the 

threshold in CH into the interior of GW. The Pacific Coast 

trend for VALUE could be indeterminate given the shift from 

one threshold to another; though visitors were more satisfied 

on this attribute, they judged it as less important.

ROADS were deemed low priority for all but the South, 

where the attribute fell into the CH quadrant. The conditions 

of roads on the site became more important in round 3 for 

visitors in the South and Rocky Mountain regions and the 

Nation, while sustaining round 2 importance levels. The 

growth in importance for the South was sufficient to exceed 

the threshold for the CH quadrant. 
23 A Technical Document Supporting the Update to the Forest Service 2010 RPA Assessment

Site Type OUDS

4.2 4.4 4.6 4.8

4.2 4.4 4.6 4.8

RPA Region: North

Satisfaction Rating

Importance Rating

2 3

2

32

3

2

32

3

2 3

2

3

2

3 2 3

2

3

2

3

4.2 4.4 4.6 4.8

4.2 4.4 4.6 4.8

RPA Region: South

Satisfaction Rating

Importance Rating

2

3

2

2 3

3

2

3

2

3

2

3

2

3

2

3

2

3 2

3

2

3

VALUE

NATENVR

SCENERY

SAFETY

TRAILS

ROADS

PARKING

ISO−RATING LINE

MEAN

SIGNAGE

DEVFACILITIES

CLEANLINESS

PARKINGLOT

4.2 4.4 4.6 4.8

4.2 4.4 4.6 4.8

RPA Region: Rocky Mountain

Satisfaction Rating

Importance Rating

3 2

3 22

3

2

3

2

3

2

3

2 3

2

3

2 3

2

3

2

3

4.2 4.4 4.6 4.8

4.2 4.4 4.6 4.8

RPA Region: Pacific Coast

Satisfaction Rating

Importance Rating

2 3

2 2 3

3

2 3

2

3

2 3 2 3

2 3 2 3

2 3

2 3

4.2 4.4 4.6 4.8

4.2 4.4 4.6 4.8

Nation

Satisfaction Rating

Importance Rating

2

3

2 23

3

2

3

2 3

2

3

2

3

2

3

2 3

2 3

2

3

Figure 9—Importance-performance analysis for overnight-use developed sites (OUDS) with weighting by inverse annual number of 

trips on 11 attributes across the 4 RPA regions and the Nation, from round 2 to round 3, with quadrants and hybrid iso-priority line 

approaches.
24 A Temporal Importance-Performance Analysis of Recreation Attributes on National Forests

Table 5—Rates of change between weighted means for satisfaction (satis) and importance (import) between rounds across 

Resources Planning Act regions and the Nation for overnight-use developed sites (OUDS), with significance determined by 

parametric two-sample t-testing

OUDS

North South Rocky Mountain Pacific Coast Nation

Attribute Satis Import Satis Import Satis Import Satis Import Satis Import

NATENVR 1.71%** -0.01% -0.21% 3.47%** -2.00%** -0.03% 1.76%** 0.17% -0.06% 0.58%**

SCENERY 0.37% -0.66% 1.20%** 2.33%** -0.90%** -0.17% 0.74%* -0.46% 0.12% 0.10%

SAFETY -0.77% 1.81% 0.61% 2.10%** 0.51% 0.82% 2.33%** -0.95% 1.00%** 0.51%

TRAILS 0.86% 1.40% -3.29%** 6.06%** 2.18%** 2.12%** 2.06%** 0.64% 1.25%** 2.13%**

VALUE 2.88%* 2.70%** 2.84%** 2.71%** 0.06% 0.38% 1.69%** -2.14%** 1.30%** 0.08%

ROADS -1.89% 0.80% -0.49% 7.83%** 0.41% 2.11%** 0.81% -0.04% 0.13% 2.11%**

PARKING -1.08% 1.00% 0.18% 7.44%** 1.80%** 0.05% 1.42%** -0.30% 1.05%** 1.09%**

SIGNAGE 4.03%** 0.99% -0.81% 4.41%** -0.50% 1.48%** 1.14% 0.63% 0.49% 1.53%**

DEVFACILITIES 0.71% 0.91% -2.74%** 5.40%** 0.78% 0.32% 2.28%** -0.45% 0.72%* 0.82%*

CLEANLINESS 0.01% 1.02% 0.46% 4.14%** 0.33% 0.70% 2.50%** -0.65% 0.97% 0.78%**

PARKINGLOT -1.71%* 1.88% 1.47%* 8.56%** 2.03%** 1.20% 2.49%** 0.07% 1.65%** 1.93%**

Attributes: NATENVR=Condition of the natural environment; SCENERY=Scenery at this site/area; SAFETY=Feeling of safety; TRAILS=Condition 

of forest trails; VALUE=Value for fees paid (for fee sites only); ROADS=Condition of forest roads; PARKING=Available parking; 

SIGNAGE=Adequacy of signage to this site; DEVFACILITIES=Condition of developed recreation facilities; CLEANLINESS=Cleanliness of 

restrooms; PARKINGLOT=Parking lot condition.

*Statistically significant for α = 0.10

**Statistically significant for α = 0.05

Positive rate of change (significant)

Negative rate of change (significant)

PARKINGLOT and PARKING could be possible overkill 

for the Nation and all regions except the South, where they 

were performing relatively well (GW). PARKINGLOT, 

followed by PARKING, tended to be the attributes (relatively) 

lowest in importance. Nevertheless, both attributes gained 

substantially in importance, with satisfaction increasing for 

PARKINGLOT and remaining stable for PARKING in the 

South. These changes propelled the attributes into the GW 

quadrant from the round 2 PO results. For both parkingrelated attributes, the Rocky Mountain and Pacific Coast 

regional visitors were more satisfied in round 3 than in 

round 2, though with importance at round 2 levels. Visitors 

nationally were more satisfied with both attributes and 

viewed them as more important than in the previous round. 

Because the round 2 means were squarely in the PO quadrant, 

the gains for both attributes in satisfaction and importance 

were not sufficient for the GW quadrant. Satisfaction declined 

for PARKINGLOT in the North, where visitors perceived 

importance at round 2 values, so the attribute remained in the 

PO section. 

DEVFACILITIES performed well (GW) in the North, with 

attention needed (CH) in the South by round 3 and possible 

overkill for the Rocky Mountain and Pacific Coast regions 

and the Nation. The national mean for DEVFACILITIES 

tended toward the threshold bordering GW, so this may signal 

a change if the trend persists to round 4. Visitors’ perceptions 

of developed facilities in the South showed significantly 

declining satisfaction on increasing importance, driving the 

attribute to the CH classification. In the Pacific Coast region 

and the Nation, visitors’ satisfaction levels increased, with 

static importance in the former and increasing importance in 

the latter. Though the changes in DEVFACILITIES for the 

North were not intertemporally significant, the proximity to 

the threshold and slight changes motivated a marginal shift to 

the GW quadrant. 

The iso-priority line confirmed CLEANLINESS as needing 

attention in the Nation and all regions. SIGNAGE could 

warrant attention in the South, Rocky Mountain, and Pacific 

Coast regions and the Nation. VALUE needed work in 

round 2 for the Pacific Coast and the Nation, but satisfaction 

rose by round 3. However, this attribute could be in need 

of attention for the Rocky Mountain region. In the South, 

SAFETY and PARKING tended toward the diagonal, so 

this could signal potential changes toward the CH section 

depending on future patterns. In the South, attributes in the 

CH quadrant had a relatively large increase in importance 

relative to the changes in satisfaction (generally declining) 

to round 3. The iso-priority line confirms these attributes 

as potentially problematic: DEVFACILITIES, ROADS, and 

TRAILS, along with the aforementioned SIGNAGE and 

CLEANLINESS.
25 A Technical Document Supporting the Update to the Forest Service 2010 RPA Assessment

Intertemporal analysis for OUDS—OUDS visitors 

generally maintained or gained satisfaction across attributes 

between rounds, with few exceptions (table 5). Adequacy of 

signage in the North showed the greatest gain in satisfaction, 

though at static importance levels. The importance of 

PARKINGLOT increased in the South by almost 8.6 percent 

between rounds, the largest jump in any measure for OUDS. 

Parking lot conditions become especially important to those 

using overnight facilities, as reflected in this gain, and users 

found the experience increasingly satisfactory in that respect. 

A general pattern of increased importance with relatively 

static satisfaction across many attributes was reported for 

OUDS in the South.

Satisfaction declined while importance remained unchanged 

for NATENVR and SCENERY in the Rocky Mountain 

region, as did PARKINGLOT in the North. The potentially 

problematic trend of increasing importance and a downturn in 

satisfaction was observed for TRAILS and DEVFACILITIES 

for the South. These shifts into the CH quadrant can be seen 

in figure 9. TRAILS displayed the greatest loss in satisfaction 

between rounds, along with a relatively large increase in 

importance.

Following are summarized managerial implications for 

the RPA regions and the Nation in the OUDS site type. 

Borderline results are indicated by regions in italics.

Concentrate here (Relatively high importance, relatively 

low satisfaction)

• Cleanliness of restrooms (All)

• Adequacy of signage to site (North, South, Rocky 

Mountain, and Nation)

• Condition of forest trails (South)

• Condition of forest roads (South)

• Condition of developed recreation facilities (South)

• Value for fee paid (Rocky Mountain)

Keep up the good work (Relatively high importance, 

relatively high satisfaction)

• Condition of the natural environment (All)

• Scenery at this site/area (All)

• Feeling of safety (All)

• Value for fee paid (North, South, Pacific Coast, and 

Nation)

• Condition of developed recreation facilities (North)

• Available parking (South)

• Parking lot condition (South)

Low priority (Relatively low importance, relatively low 

satisfaction)

• Condition of forest roads (North, Rocky Mountain, 

Pacific Coast, and Nation)

• Condition of forest trails (Rocky Mountain and Nation)

• Adequacy of signage to this site (Pacific Coast)

Possible overkill (Relatively low importance, relatively 

high satisfaction)

• Available parking (North, Rocky Mountain, Pacific 

Coast, and Nation)

• Parking lot condition (North, Rocky Mountain, Pacific 

Coast, and Nation)

• Condition of developed recreation facilities (Rocky 

Mountain, Pacific Coast, and Nation)

• Condition of forest trails (Pacific Coast)

Indeterminate due to proximity to thresholds

• Condition of forest trails (North)

Potential shifts to area of concern per iso-priority line 

(i.e., satisfaction did not equal or exceed perceived 

importance)

• Cleanliness of restrooms (All)

• Condition of forest trails (South)

• Condition of forest roads (South and Rocky Mountain) 

• Adequacy of signage to site/area (South, Rocky 

Mountain, Pacific Coast, and Nation)

• Condition of developed recreation facilities (South)

• Available parking (South)

• Condition of the natural environment (South, Rocky 

Mountain, Pacific Coast, and Nation)

• Value for fee paid (Rocky Mountain)

Overall Satisfaction

The NVUM satisfaction module contains two additional 

measures by which visitors may rank their overall satisfaction 

and perceptions of crowding during their exit interview. 

The overall satisfaction pertaining to their visit is ranked on 

a 5-point Likert scale, ranging across 1 (very dissatisfied), 

2 (somewhat dissatisfied), 3 (neither satisfied nor dissatisfied), 

4 (somewhat satisfied), or 5 (very satisfied). No significant 

declines in overall satisfaction occurred in responses by site 

type and region (table 6) from round 2 to round 3, implying 

that recreationists were either sustaining or improving their 

satisfaction levels. The greatest growth in overall satisfaction 

occurred for North WILD and South OUDS, both being 

approximately 1.5 percent, a slight but statistically significant 
26 A Temporal Importance-Performance Analysis of Recreation Attributes on National Forests

Table 6—Results of t-testing on overall satisfaction and crowding 

ratings between rounds 2 and 3, with weighting for each Resources 

Planning Act region and the Nation

Region, Site 

type

Overall 

satisfaction,

round 2

Overall 

satisfaction,

% change to 

round 3

Crowding,

round 2

Crowding, 

% change 

to round 3

N, DUDS 4.80 0.17% 4.18 5.81%**

N, GFA 4.69 0.09% 4.21 3.99%*

N, OUDS 4.75 0.04% 4.54 9.44%**

N, WILD 4.76 1.48%** 4.12 2.60%

S, DUDS 4.76 0.55%** 4.78 -4.64%**

S, GFA 4.76 0.86%** 4.65 1.67%

S, OUDS 4.69 1.46%** 5.13 1.00%

S, WILD 4.78 0.38% 4.48 -6.98%

RM, DUDS 4.78 0.69%** 4.46 2.99%**

RM, GFA 4.73 0.25% 4.40 5.84%**

RM, OUDS 4.73 0.70%** 4.98 -0.30%

RM, WILD 4.85 0.00% 4.57 4.74%**

PC, DUDS 4.77 0.67%** 4.38 5.84%**

PC, GFA 4.72 0.76%** 4.35 6.79%**

PC, OUDS 4.72 0.62%** 4.93 2.54%

PC, WILD 4.80 0.51%** 4.25 10.60%**

Nation, DUDS 4.78 0.58%** 4.45 2.96%**

Nation, GFA 4.73 0.50%** 4.39 5.38%**

Nation, OUDS 4.72 0.71%** 4.93 1.92%*

Nation, WILD 4.81 0.34%** 4.41 4.61%**

Regions: N=North; S=South; RM=Rocky Mountain; PC=Pacific Coast.

Site types: DUDS=day-use developed sites; GFA=general forest areas; 

OUDS=overnight-use developed sites; WILD=designated Wilderness.

*Statistically significant for α = 0.10

**Statistically significant for α = 0.05

Positive rate of change (significant)

Negative rate of change (significant)

change given the relatively low variability. Figure 10 shows 

the slightly greater proportion of responses for “very 

satisfied” by North WILD visitors in round 3. 

The 5-point scale does not provide as much spread as a 

7-point scale, and the ratings are clustered at 4 and 5 for 

overall satisfaction, a tendency also observed in individual 

satisfaction measures from the 11 attributes. The low 

variations may be attributed to this clustering; though the 

gains in satisfaction were not large, they were statistically 

significant with respect to the variance. As a whole, none of 

the means for satisfaction was below 4.6 on a 5-point scale, 

indicating high performance overall.

Crowding

Another aspect of sustaining satisfaction levels over the 

long term is monitoring increases in crowding level, which 

could translate to reduced recreation resources available per 

participant. The density of users on a site at a given time 

can vary by the popularity of the activities as well as visitor 

expectations, and thus visitor attitudes toward crowding can 

vary by site type. Crowding, often called perceived crowding, 

is an assessment made by visitors, based on how many others 

they encountered during their visit (Vaske and Shelby 2008). 

In the NVUM survey satisfaction module, respondents 

were asked about their perception of crowding on their 

just-concluded site visit. The crowding scale ranged from 

1 (hardly anyone) to 10 (overcrowded). 

Crowding does not follow a linear pattern for all activities. 

Activities that thrive on group or social engagement may 

benefit from some crowding. Additionally, it may not be 

practical to allocate resources and personnel to staff sites with 

very low levels of use, such as a GFA that has little traffic 

during a particular time of year. Activities that depend on 

solitude, such as backpacking in designated Wilderness, will 

have a lower threshold of overcrowding, at which point the 

quality of the recreation experience is eroded by the presence 

of others. There is no one-size-fits-all approach in managing 

crowding, as variations occur by activity and site type, but 

planners can use overall satisfaction and crowding scores as 

indicators of areas for improvement or maintenance, and as 

indicators that the quality of recreation experience is, or is 

not, being sustained.
27 A Technical Document Supporting the Update to the Forest Service 2010 RPA Assessment

From round 2 to round 3, the means across regions and 

site types for crowding tended to range between 4 and just 

over 5. Statistically significant declines in crowding ratings 

occurred only for South DUDS and South WILD. These 

downturns in crowding ratings coincided with declining 

satisfaction in natural elements and trails; South DUDS and 

South WILD experienced declining satisfaction in SCENERY 

and TRAILS, and visitors to South DUDS further reported 

decreased satisfaction in NATENVR. No significant changes 

in crowding could be detected for the North WILD, South 

GFA, and OUDS for the South, Rocky Mountain, and Pacific 

Coast regions. In all other site types and regions, statistically 

significant crowding increases were reported between rounds 

2 and 3.

In the Rocky Mountain and Pacific Coast regions and the 

Nation, nearly all site types had crowding increases. The 

increase was especially pronounced in the Pacific Coast 

WILD at 10.60 percent between rounds (table 6). This result 

is generally indicative of the upward trend in national forest 

visitation during this time. Figure 11 reveals a bimodal 

distribution of responses about crowding in the Pacific 

Coast WILD: one for the low-level crowding centered at 2 

and another for higher crowding levels at 6. Few perceived 

overcrowding at levels of 7 and above. Such a pattern 

may reflect the fact that visits to designated Wilderness 

often involve day visitors who do not venture deep into 

the backcountry and multiday visitors who more often 

get to more-remote locations. A contributing factor to the 

temporal changes was the decrease in ratings of 1 and the 

increase in scores of 6, which could be problematic for 

WILD. Wilderness managers in the Pacific Coast and Rocky 

Mountain regions, and the Nation should be cognizant of the 

potential issues associated with two distinctly different types 

of users, particularly given the statutory requirements of 

designated Wilderness.

CONCLUSIONS 

The methods described here build on the basic technique of 

IPA developed by Martilla and James (1977), a procedure that 

has endured due to its ease of application and interpretation. 

IPA has been used in multiple publications over the last three 

decades: O’Leary and Adams (1982), Crompton and Duray 

(1985), Ziegler and others (2012), Albayrak (2015), and Lee 

(2015). Sörensson and von Friedrichs (2013) applied IPA to 

evaluate sustainable tourism for two groups (international 

and national tourists). They compared the managerial actions 

across the two groups, but their analysis did not include a 

temporal dimension. The NVUM sampling data that we 

analyzed introduced additional complexities and richness 

not recognized in Martilla and James (1977). Not only were 

the data collected over multiple spatial levels, the rounds 

entailed a temporal aspect that provided a framework for 

trend evaluations. The intertemporal comparisons were 

achieved through parametric two-sample t-testing, as 

described in Leeworthy and others (2004); they used this 

0

10

20

30

40

50

60

70

80

90

100

Very Dissatisfied Somewhat

Dissatisfied

Neither Satisfied

Nor Dissatisfied

Somewhat

Satisfied

Very Satisfied

Weighted percentages

Likert scale: 1 (Very Dissatisfied) to 5 (Very Satisfied)

Overall Visit Satisfaction for Site, North WILD, Rounds 2 to 3

Round 2 Round 3

Figure 10—Distribution of National Visitor Use Monitoring responses (percentages based on 

weighting with inverse annual number of trips) for overall satisfaction on North designated Wilderness 

(WILD) for rounds 2 and 3.
28 A Temporal Importance-Performance Analysis of Recreation Attributes on National Forests

method to compare importance and satisfaction for visitors 

and residents across two time periods. A further complication 

in our analysis was avidity bias from frequent users’ 

increased likelihood of being sampled. Similar to Gill and 

others (2010), this work incorporated a weighting scheme 

based on the inverse of the annual number of trips to address 

endogenous stratification. We also added the iso-priority 

line methodology as discussed in Abalo and others (2007), 

Ziegler and others (2012), Azzopardi and Nash (2013), Lai and 

Hitchcock (2015), and Lee (2015). This approach has been 

used in a variety of settings, such as whale shark tourism 

in Mexico (Ziegler and others 2012). Our research presents 

the first applications of IPA with intertemporal testing since 

Leeworthy and others (2004). These results are also the first 

applications of this methodology to national forests. 

We presented the results from the traditional IPA threshold 

framework first, with a secondary discussion on the attributes 

affected by adding the diagonal line. The diagonal line has the 

advantage of being data-independent, as the sole restriction 

is that satisfaction ratings at least match the attached 

importance. For example, if a respondent attaches a neutral 

importance rating to an attribute, then the outcome should be 

neutral or concurrent with at least some level of satisfaction. 

Attributes can therefore fall into the “good work” (GW) 

quadrant of conventional IPA and be in need of improvement 

if falling above the diagonal. Per the more stringent criterion 

for the iso-priority line, a user who rates an attribute as 

highly important should be highly satisfied, not simply 

somewhat satisfied (even if the attribute is in the conventional 

GW quadrant). Additionally, the farther into the space 

above the diagonal line, the greater the mismatch is between 

expectations and outcomes. This may be useful in prioritizing 

areas of need, as the distance metric is meaningful. 

Evaluating participants from a consumer standpoint can 

provide additional insight into the long-term outlook for 

outdoor recreation. The 2010 RPA Assessment and related 

works gauged recreation in terms of participation rates, total 

participants, participant intensity, and total days of activity for 

multiple activities (Bowker and Askew 2012, 2013; Bowker 

and others 1999, 2012, and 2013; Cordell 2012; Cordell and 

others 2013). These projections would be most helpful in 

managing resources and access, specifically concerning 

those activities that could have the most growth or decline. 

Looking at individuals and their decisionmaking mechanisms 

can provide additional insight for management. IPA results 

for visitors’ responses about specific attributes can guide 

managers as they develop focused directives for improvement 

or maintenance and reallocate resources accordingly. With 

cooperative resource and attribute management, the goal 

outcome of an enduring satisfied user base can provide a 

long-term healthy outlook for outdoor recreation. Monitoring 

changes temporally and spatially can identify areas of need 

in time for improvement. For example, in a future where 

swimming could become increasingly popular (as indicated 

0

5

10

15

20

25

30

12345678 9 10

Weighted percentages

1 (Hardly Anyone) to 10 (Overcrowded)

Crowding Ratings for Pacific Coast WILD

Round 2 Round 3

Figure 11—Distribution of National Visitor Use Monitoring responses (percentages based on weighting with 

inverse annual number of trips) for crowding on Pacific Coast designated Wilderness (WILD) for rounds 2 and 3.
29 A Technical Document Supporting the Update to the Forest Service 2010 RPA Assessment

by participation rates and total participants at the national 

level in the 2010 RPA Assessment), resources may be needed 

to provide more access or mitigate crowding in order to 

sustain the quality of users’ experiences. Perhaps a waterside 

should be prioritized for preservation or cleanup, or beach 

access should be designated in response to crowding. IPA 

could then be used in a future study to focus on those who 

used the national forest for swimming activities. As an 

example, if users do not find parking to be convenient, then 

they may visit another site with more suitable parking or 

embark on another activity. The application of IPA means that 

the user base can be specified or broadened to the appropriate 

context.

In addition to presenting an updated IPA methodology, 

the major takeaway from this study is that at the regional 

level, all measured national forest recreation attributes have 

performed well (i.e., mean attribute ratings above 4) on both 

satisfaction and importance, meaning that recreationists 

were generally satisfied with factors deemed important to 

their recreation visits. However, because of budgetary and 

resource limitations affecting long-term management, there 

is a need to identify areas of improvement and those of 

“possible overkill” or “low priority.” The static and temporal 

IPA analyses are one way to give managers insight into when 

resources may be shifted from the relatively less important 

attributes toward maintaining others to “keep up the good 

work” or improving those in the “concentrate here” area. 

Even when all attribute means are 4 or greater across the 

various site types and regions, identifying relative differences 

can inform management. 

Generally, NATENVR, SCENERY, and SAFETY performed 

best relative to the other attributes, and were strong points 

in the national forest recreation experience. Though humans 

may not always be able to manage the conditions of the 

natural environment (i.e., drought, fire, disease) directly, 

understanding what has caused a downturn in satisfaction 

may help with strategic improvements. Depending on the 

cause, counteractive steps may be taken to mitigate decline in 

satisfaction for natural environment conditions. For example, 

in wildlife viewing, if a species has declined due to habitat 

loss, then preserving remaining habitats for the species 

may encourage a return of both the species and the viewers. 

Overall, the surveys revealed that people tended to seek 

recreation where the natural environment is enjoyable, with 

high quality scenery and a feeling of safety. The perception 

of safety became less important with the WILD stratum, as 

expected, where people are accustomed to, and in fact seek 

out, a sense of solitude and attached emphasis on NATENVR 

and SCENERY. In fact, none of the attributes could be 

considered in need of work for the WILD stratum, with the 

exception of the borderline TRAILS in the Pacific Coast.

For the developed site types only, the cleanliness of restrooms 

could warrant improvement, as the importance ratings 

consistently outstripped the satisfaction levels. The adequacy 

of signage also could benefit from improvement for DUDS 

and OUDS, nationally and for all regions but the Pacific 

Coast. Generally, the availability of parking, developed 

facilities, and parking lot conditions were in the "possible 

overkill" or "low priority" quadrant. However, OUDS 

developed facilities in the South fell into the "concentrate 

here" quadrant, while parking and parking lot conditions were 

in the "keep up the good work" quadrant. Developed facilities 

for OUDs in the North were also in the "keep up the good 

work" quadrant.

Regionally, the South overall tended to demonstrate more 

attributes above the iso-priority line, indicating potential need 

for attention under the more stringent criterion. The patterns 

of attributes for the other three regions and the Nation were 

mostly similar, with a few minor individual variations. 

The results were unique for the South under OUDS, where 

satisfaction changed little in conjunction with a rise in 

importance. Future work could be undertaken to identify the 

driver(s) of this growth in importance for South OUDS, by 

using modeling approaches on relevant explanatory variables. 

Increasing population growth in the South may be leading 

to a class of visitors different from the past. In some cases 

such as Rocky Mountain GFA, there were miniscule changes 

between rounds 2 and 3. Much larger changes occurred 

elsewhere, particularly in the South. 

Short-term planning with long-term satisfaction in mind 

should acknowledge the room for improvement, as indicated 

by this multifaceted analysis. Though all the means are 

dominated by a large group of highly satisfied users with 

highly important items, some users were not satisfied, rating 

their experiences at 1 or 2. Finding out more about these 

users would be a potential direction for future research. 

Their dissatisfaction may derive from a particular setting at 

a specific national forest. Due to the prevalence of ratings 

of 4 or 5, examining the unfulfilled users more closely 

could perhaps reveal some patterns or systematic need for 

improvement. A potential next step could be to investigate 

what these low-rated responses have in common, if anything, 

but perhaps some users of a resource or experience will never 

be satisfied. That a substantial proportion of satisfied users 

appeared in the survey data indicates the health of recreation 

regionally and nationally. IPA provides a scientific instrument 

that identifies the relative weak points, and therefore indicates 

where resources could be allocated or shifted toward higher 

priorities. The simplicity of IPA’s graphical story offers a way 

to examine 175 different mean pairs simultaneously within 

a site type for five geographic entities. The incorporation 

of a temporal aspect into IPA adds another dimension to 
30 A Temporal Importance-Performance Analysis of Recreation Attributes on National Forests

the analysis, with two time points currently in place as 

two complete rounds of NVUM sampling. As more rounds 

are added, the trends may be expanded and subsequently 

analyzed by using appropriate statistical methods. For this 

work, the two points represented a snapshot of recreation 

spanning the two rounds. This snapshot revealed multiple 

avenues of insight into visitors with respect to satisfaction 

and thereby provided managers with an opportunity to assess 

potential trends. This graphical presentation of the temporal 

aspect is also novel, incorporating the trend lines and 

contributing to an assessment of sustainability.

With the application of parametric t-testing, we compared 

the mean scores from round 2 to round 3 for satisfaction 

and importance ratings. The application of a parametric 

test and quantification of the Likert-type items have been 

done in practice across many studies, and in some cases, the 

parametric test may perform well, depending on the setting 

(i.e., scope of the Likert scale, sample sizes). Relatively 

speaking, many of the attributes did not change significantly 

temporally, perhaps because of the prevalence of high ratings. 

These left-skewed distributions most likely dominated the 

mean computations, creating less variability than with a 

7-point or higher interval scale. The distributions were thus 

similar enough for little change to occur between rounds. 

However, some of the attributes did significantly decline 

or increase. A decline in satisfaction with an increase in 

importance or a large decline in satisfaction with static 

importance is a pattern of concern. The parametric testing 

was also applied to the overall satisfaction and crowding 

ratings, revealing overall high levels of satisfaction with 

increases to round 3. With few exceptions, crowding generally 

increased to round 3, indicating growth in encounters of other 

recreationists on trips consistent with the trend of increased 

visitation across all national forest setting types. As crowding 

has ultimately been shown to have a negative effect on visitor 

satisfaction, it may be only a matter of time before satisfaction 

ratings for crowding begin to fall. This outcome could be 

averted through carefully considered limits to carrying 

capacity, where applicable, or increasing infrastructure 

to accommodate the increased demand. The interactions 

between satisfaction and crowding in the future will be of 

interest, and are likely to cause concern about management of 

supply and demand for managers facing limited resources. 

Limitations and Future Directions

Sampling—An inherent challenge is quantifying 

importance and performance, as there are multiple 

perspectives and ways of gleaning responses. Direct and 

indirect measures of importance each have their own set 

of issues. An alternative formulation that may address the 

problems of both methods is that of partial ranking (Abalo 

and others 2007, Azzopardi and Nash 2013). The rater is 

asked to evaluate the set of attributes all at once, ranking in 

order the importance of each with no ties permitted. This 

has the advantage of removing the redundancy of using the 

same scale not only for performance but also for importance; 

the rater must be more engaged in evaluation to express his 

or her decisionmaking mindset accurately. The importance 

scores are necessarily dispersed, as respondents cannot rate 

attributes using the same range (Abalo and others 2007). 

Additionally, another potential improvement could be 

expanding the existing 5-point scale to 7 points for greater 

spread of responses but at the cost of the ability to compare 

results with rounds 2 and 3.

Users of IPA may encounter the “ceiling effects” problem, 

where importance ratings are scored highly, or inflated, for 

most attributes (Lai and Hitchcock 2015). One underlying 

reason is that the attributes have been prescreened to be 

important in determining consumer satisfaction, which 

can be affirmed by high mean importance ratings. Another 

potential source of bias is rater fatigue or common method 

bias (Podsakoff and others 2003). Raters may provide 

inaccurate answers if the survey is deemed arduous because 

the questionnaire is too long or intrusive. Additionally, 

if the sets of importance and satisfaction questions are 

presented simultaneously, then raters may find the task 

repetitive, and responses to the two scales may be dependent. 

A design strategy is to eliminate the confounding caused 

by simultaneous presentation by asking for ratings of all 

attributes on one aspect (importance or performance) before 

the other. This separation of evaluations by importance and 

satisfaction encourages raters to contemplate the entire set of 

attributes in one sense before moving to the next (Podsakoff 

and others 2003). A type of common method bias known as 

social desirability occurs when respondents provide answers 

that are socially acceptable or agreeable rather than honest 

(Podsakoff and others 2003). This is a realization of the 

Hawthorne effect, where subjects exhibit different behavior 

than if unaware of being observed (Lai and Hitchcock 2015, 

Schwartz and others 2013). NVUM surveys are administered 

with a trained interviewer; though the responses are kept 

anonymous, there is a face-to-face interaction between 

interviewer and respondents. Another type of bias seen 

with survey data is that of common scale formats, where 

covariation occurs because of the same scale being used for 

rating of multiple items (Podsakoff and others 2003).

Analysis—Like most analytical approaches, IPA has 

some shortcomings, but it is extensively used in multiple 

applications due to its simplicity and applicability. The 

analysis starts with a simplification of complex human 

reasoning and the challenges of surveys, but the output 

is conveniently interpreted, acknowledging the inherent 
31 A Technical Document Supporting the Update to the Forest Service 2010 RPA Assessment

limitations. The analyses of Likert-type items present 

challenges in representing an ordinal variable as numeric, 

with an underlying continuous distribution. An approach to 

consider is satisfying the normality assumption of t-testing 

by using summated scales of at least five attributes (Clason 

and Dormody 1994). Because we proceeded with analysis 

on Likert-type items, we must consider whether temporal 

comparisons are based on parametric or nonparametric 

methods. In this paper, we computed t-tests on means 

between rounds 2 and 3. Using a t-test assumes a continuous 

variable and is powerful under the assumptions of normality 

and population homoscedasticity. Though generally more 

powerful than nonparametric tests, parametric testing may 

not retain validity under significant departures from the 

underlying assumptions. Therefore, a nonparametric method 

such as the Mann-Whitney-Wilcoxon test may be more 

suitable (Clason and Dormody 1994, de Winter and Dodou 

2010). On a 5-point Likert scale, de Winter and Dodou 

(2010) found through simulation that power is generally 

equivalent for t-testing and Mann-Whitney-Wilcoxon testing. 

This approach does not necessarily guarantee similar 

properties on a 7-point scale, as the wider range can lead to 

greater skewness. The Mann-Whitney-Wilcoxon test can 

be preferable in some cases, with t-testing used in others. 

Overall, the methodology can be selected on a case-by-case 

basis, depending on the underlying distribution and number of 

points on the Likert scale. A future methodological extension 

would be to compare the results under Mann-WhitneyWilcoxon methods and the t-testing in this study. This work 

used the parametric approach described by Leeworthy and 

others (2004). 

An underlying element of complexity to NVUM data analysis 

is the stratification of sampling, namely on multiple-use 

levels within site types (e.g., low-, medium-, and high-use 

DUDS). Use levels at a given site type may vary temporally, 

depending on the day of week, season, conditions, and 

popularity shown in LER traffic. The NVUM sampling 

protocol acknowledges budgetary limitations in the sampling 

design by factoring in use levels. Fewer sampling days may 

be allocated to a high-use site type than a low-use one due 

to the higher traffic expected in a shorter timeframe. Within 

each sampling day, per NVUM methodology, a 24-hour 

count of exiting traffic is computed, along with 6 hours of 

onsite exit interviews. In the current weighting scheme, the 

only adjustments made were based on the reported trips 

taken by a visitor annually. By not stratifying further, the 

current analysis looked at the site type level only, ignoring 

the additional information of use level and acknowledging 

a known source of variation. It may be more efficient to 

assure high quality of experiences at sites with high use 

levels within a site type. This approach could present some 

analytical issues in computation of standard errors, biasing 

them downward. However, already inherent in basic IPA is 

the technical matter that assumes an underlying continuous 

distribution for a set of qualities treated as integers. This 

analysis was intended to establish a foundation in application 

of IPA to NVUM data to examine short- and long-term 

recreation sustainability. Now that a simple application has 

been established, future work can address the complexity 

of site type use levels, factoring in an additional layer of 

known variation due to stratification. The statistical software 

SAS® (SAS Institute Inc., Cary, NC) includes powerful 

tools that may supplement future work, such as PROC 

SURVEYSELECT, which factors in intricate probabilitybased sampling designs (An and Watts 1998).

There are elements in the IPA methodology that may be 

improved, but efforts to improve them can bring about 

another set of issues or detract from the appeal of the original 

version. Likewise, any analyses using Likert-type items also 

have the underlying issue of scale, but analyses are computed 

with this understanding and acknowledgment. Further 

research should investigate these more complex analyses 

and compare them against the results of basic IPA. Research 

continues into IPA, with more complex formulations being 

discovered while the basic methodology remains popular. 

Now that we have a simple representation for the sampling 

scheme in place, the next level of research will be to factor 

in the complex sampling design over use levels. It will be of 

interest to compare results computed from subsequent IPA 

designs against this simpler foundational application. 

An important component of management strategy is to 

monitor recreationists’ satisfaction over time, as users can 

indicate through the NVUM survey process those areas 

performing highly and those in need of remediation. With the 

NVUM Program’s network of sampling, this work presents 

a systematic and convenient use of those responses toward 

better understanding visitor satisfaction. The methodology 

for this report presented both a fundamental IPA and a hybrid 

approach using the thresholds of basic IPA and a diagonal 

line for increased sensitivity, combined with an intertemporal 

framework that can be extended to future time points. 

Additionally, understanding overall satisfaction levels and 

crowding perception in time may be valuable supplements. 

Overall satisfaction can be an indicator of how fulfilling 

each recreationist finds the overall experience, rather than 

rating by attribute. Levels of crowding can serve as a gauge 

of users’ space perceptions, and suitable levels may vary by 

site type. An acceptable level of crowding for developed sites 

will not match that for designated Wilderness. Therefore, by 

monitoring crowding and general satisfaction perceptions 

over time in conjunction with IPA, we have established a 

multifaceted perspective on the recreation visitor base. 
32 A Temporal Importance-Performance Analysis of Recreation Attributes on National Forests

ACKNOWLEDGMENTS

We are grateful to Ross Arnold, Linda Langner, 

Lincoln Larson, and Vernon R. Leeworthy for suggesting 

improvements to earlier versions of this manuscript.

LITERATURE CITED

Abalo, J.; Varela, J.; Manzano, V. 2007. Importance values for 

importance-performance analysis: a formula for spreading 

out values derived from preference rankings. Journal of 

Business Research. 60(2): 115–121.

Albayrak, T. 2015. Importance performance competitor 

analysis (IPCA): a study of hospitality companies. 

International Journal of Hospitality Management. 

48: 135–142.

An, A.; Watts, D. 1998. New SAS® procedures for analysis of 

sample survey data. Cary, NC: SAS Institute Inc. 11 p. 

Anderson, E.W.; Fornell, E.; Lehmann, D.R. 1994. Consumer 

satisfaction, market share, and profitability: findings from 

Sweden. Journal of Marketing. 58(3): 53–66.

Askew, A.E.; English, D.B.K.; Zarnoch, S.J. [and others]. 

2014. Mitigating budget constraints on visitation volume 

surveys: the case of U.S. National forests. Gen. Tech. Rep. 

SRS–192. Asheville, NC: U.S. Department of Agriculture 

Forest Service, Southern Research Station. 21 p. http://

www.srs.fs.usda.gov/pubs/gtr/gtr_srs192.pdf. [Date 

accessed: March 29, 2016].

Azzopardi, E.; Nash, R. 2013. A critical evaluation of 

importance-performance analysis. Tourism Management. 

35: 222–233.

Bowker, J.M.; Askew, A. 2012. U.S. outdoor recreation 

participation projections to 2060. In: Cordell, H.K., ed. 

Outdoor recreation trends and futures: a technical document 

supporting the Forest Service 2010 Resources Planning 

Act Assessment. Gen. Tech. Rep. SRS–150. Asheville, NC: 

U.S. Department of Agriculture Forest Service, Southern 

Research Station: 105–124. http://www.srs.fs.fed.us/pubs/

gtr/gtr_srs150/gtr_srs150_105.pdf. [Date accessed: March 

29, 2016].

Bowker, J.M.; Askew, A.E. 2013. Outlook for outdoor 

recreation in the Northern United States: a technical 

document supporting the Northern Forest Futures Project 

with projections through 2060. Gen. Tech. Rep. NRS–120. 

Newtown Square, PA: U.S. Department of Agriculture 

Forest Service, Northern Research Station. 62 p.

Bowker, J.M.; Askew, A.; Cordell, H.K.; Bergstrom, J.C. 

2013. Outdoor recreation. In: Wear, D.N.; Greis, J.G., eds. 

The Southern Forest Futures Project: technical report. Gen. 

Tech. Rep. SRS–178. Asheville, NC: U.S. Department of 

Agriculture Forest Service, Southern Research Station: 

161–182. http://www.srs.fs.fed.us/pubs/gtr/gtr_srs178/gtr_

srs178_161.pdf. [Date accessed: March 29, 2016].

Bowker, J.M.; Askew, A.E.; Cordell, H.K. [and others]. 2012. 

Outdoor recreation participation in the United States—

projections to 2060: a technical document supporting the 

Forest Service 2010 RPA Assessment. Gen. Tech. Rep. 

SRS–160. Asheville, NC: U.S. Department of Agriculture 

Forest Service, Southern Research Station. 34 p. http://

www.srs.fs.fed.us/pubs/gtr/gtr_srs160.pdf. [Date accessed: 

March 29, 2016].

Bowker, J.M.; Askew, A.E.; Poudyal, N. [and others]. 2014. 

Climate change and outdoor recreation participation in the 

Southern United States. In: Vose, J.M.; Klepzig, K.D., eds. 

Climate change adaptation and mitigation management 

options. A guide for natural resource managers in southern 

forest ecosystems. New York: CRC Press-Taylor and 

Francis: 421–450. http://www.srs.fs.usda.gov/pubs/ja/2014/

ja_2014_bowker_001.pdf. [Date accessed: March 29, 2016].

Bowker, J.M.; English, D.B.K.; Cordell, H.K. 1999. 

Projections of outdoor recreation participation to 2050. 

In: Cordell, H.K.; Betz, C.; Bowker, J.M. [and others]. 

Outdoor recreation in American life: a national assessment 

of demand and supply trends. Champaign, IL: Sagamore 

Publishing: 323–351. http://www.srs.fs.usda.gov/pubs/ja/

ja_bowker005.pdf. [Date accessed: March 29, 2016].

Clason, D.L.; Dormody, T.J. 1994. Analyzing data measured 

by individual Likert-type items. Journal of Agricultural 

Education. 35(4): 31–35.

Cochran, W.G. 1977. Stratified random sampling. Sampling 

Techniques. 3: 89–110.

Cordell, H.K. 2012. Outdoor recreation trends and futures: 

a technical document supporting the Forest Service 2010 

RPA Assessment. Gen. Tech. Rep. SRS–150. Asheville, NC: 

U.S. Department of Agriculture Forest Service, Southern 

Research Station. 167 p. http://www.srs.fs.usda.gov/pubs/

gtr/gtr_srs150.pdf. [Date accessed: March 29, 2016].

Cordell, H.K.; Bergstrom, J.C.; Hartmann, L.A.; English, 

D.B.K. 1990. An analysis of the outdoor recreation and 

wilderness situation in the United States, 1989–2040: a 

technical document supporting the 1989 USDA Forest 

Service RPA Assessment. Gen. Tech. Rep. RM–GTR–189. 

Fort Collins, CO: U.S. Department of Agriculture Forest 
33 A Technical Document Supporting the Update to the Forest Service 2010 RPA Assessment

Service, Rocky Mountain Forest and Range Experiment 

Station. 112 p. http://www.fs.fed.us/rm/pubs_rm/rm_gtr189.

pdf. [Date accessed: March 29, 2016].

Cordell, H.K.; Betz, C.J.; Bowker, J.M.; English, D.B.K. 

[and others]. 1999. Outdoor recreation in American life: 

a national assessment of demand and supply trends. 

Champaign, IL: Sagamore Publishing. xii, 449 p.

Cordell, H.K.; Betz, C.J.; Zarnoch, S.J. 2013. Recreation and 

protected land resources in the United States: a technical 

document supporting the Forest Service 2010 RPA 

Assessment. Gen. Tech. Rep. SRS–169. Asheville, NC: 

U.S. Department of Agriculture Forest Service, Southern 

Research Station. 198 p. http://www.srs.fs.fed.us/pubs/gtr/

gtr_srs169.pdf. [Date accessed: March 29, 2016].

Crompton, J.L.; Duray, N.A. 1985. An investigation of 

the relative efficacy of four alternative approaches to 

importance-performance analysis. Journal of the Academy 

of Marketing Science. 13(4): 69–80.

de Winter, J.C.F.; Dodou, D. 2010. Five-point Likert 

items: t-test versus Mann-Whitney-Wilcoxon. Practical 

Assessment, Research & Evaluation. 15(11): 1–12.

English, D.B.K.; Kocis, S.M.; Zarnoch, S.J.; Arnold, J.R. 

2002. Forest Service National Visitor Use Monitoring 

process: research method documentation. Gen. Tech. Rep. 

SRS–57. Asheville, NC: U.S. Department of Agriculture 

Forest Service, Southern Research Station. 14 p. http://

www.srs.fs.usda.gov/pubs/gtr/gtr_srs057.pdf. [Date 

accessed: March 29, 2016].

Gill, J.K.; Bowker, J.M.; Bergstrom, J.C.; Zarnoch, S.J. 2010. 

Accounting for trip frequency in importance-performance 

analysis. Journal of Park and Recreation Administration. 

28(1): 16–35. http://www.srs.fs.usda.gov/pubs/ja/ja_gill002.

pdf. [Date accessed: March 29, 2016].

Gourville, J.T.; Soman, D. 2005. Overchoice and assortment 

type: when and why variety backfires. Marketing Science. 

24(3): 382–395.

Huang, F.-H.; Ye, Y.-J.; Kao, C.-H. 2015. Developing a novel 

intuitionistic fuzzy importance-performance analysis for 

evaluating corporate social responsibility in sport tourism 

event. Expert Systems with Applications. 42(19): 6530–

6538.

Jaccard, J.; Brinberg, D.; Ackerman, L.J. 1986. Assessing 

attribute importance: a comparison of six methods. Journal 

of Consumer Research. 12(4): 463–468.

Janes, P.L.; Wisnom, M.S. 2003. The use of importanceperformance analysis in the hospitality industry. Journal of 

Quality Assurance in Hospitality & Tourism. 4(1/2): 23–45.

Lai, I.K.W.; Hitchcock, M. 2015. Importance-performance 

analysis in tourism: a framework for researchers. Tourism 

Management. 48: 242–267.

Lee, H.-S. 2015. Measurement of visitors’ satisfaction with 

public zoos in Korea using importance-performance 

analysis. Tourism Management. 47: 251–260.

Leeworthy, V.R.; Wiley, P.C.; Hospital, J. 2004. Importancesatisfaction ratings five-year comparison, SPA & ER use, 

and socioeconomic and ecological monitoring comparison 

of results 1995-96 to 2000-01. Silver Spring, MD: 

U.S. Department of Commerce, National Oceanic and 

Atmospheric Administration, National Ocean Service, 

Special Projects Division. 59 p. http://sanctuaries.noaa.gov/

science/socioeconomic/floridakeys/pdfs/impsat.pdf. [Date 

accessed: March 29, 2016].

Lego, R.; Shaw, R. 1992. Convergent validity in tourism 

research: an empirical analysis. Tourism Management. 

13(4): 387–393. 

Martilla, J.A.; James, J.C. 1977. Importance-performance 

analysis. Journal of Marketing. 41(1): 77–79.

Matzler, K.; Bailom, F.; Hinterhuber, H.H. [and others]. 

2004. The asymmetric relationship between attributelevel performance and overall customer satisfaction: a 

reconsideration of the importance-performance analysis. 

Industrial Marketing Management. 33(4): 271–277.

Oh, H. 2001. Revisiting importance-performance analysis. 

Tourism Management. 22(6): 617–627.

O’Leary, J.T.; Adams, M.B. 1982. Community views 

concerning urban forest recreation resources, facilities, 

and services. Final report, Cooperative Research Project. 

Chicago, IL: U.S. Department of Agriculture Forest Service, 

North Central Forest Experiment Station. 91 p.

Pan, F.C. 2015. Practical application of importanceperformance analysis in determining critical job satisfaction 

factors of a tourist hotel. Tourism Management. 46: 84–91.

Podsakoff, P.M.; MacKenzie, S.B.; Lee, J.-Y.; Podsakoff, N.P. 

2003. Common method biases in behavioral research: a 

critical review of the literature and recommended remedies. 

Journal of Applied Psychology. 88(5): 879–903.
34 A Temporal Importance-Performance Analysis of Recreation Attributes on National Forests

Schwartz, D.; Fischoff, B.; Krishnamurti, T.; Sowell, F. 2013. 

The Hawthorne effect and energy awareness. Proceedings 

of the National Academy of Sciences of the United States of 

America. 110(38): 15242–15246.

Sever, I. 2015. Importance-performance analysis: A valid 

management tool? Tourism Management. 48: 43–53.

Shaw, D. 1988. On-site samples’ regression: problems 

of non-negative integers, truncation, and endogenous 

stratification. Journal of Econometrics. 37(2): 211–223.

Sheng, X.; Simpson, P.; Siguaw, J.A. 2014. U.S. winter 

migrants’ park community attributes: an importanceperformance analysis. Tourism Management. 43: 55–67.

Sörensson, A.; von Friedrichs, Y. 2013. An importanceperformance analysis of sustainable tourism: a comparison 

between international and national tourists. Journal of 

Destination Marketing & Management. 2(1): 14–21.

To, W.M.; Lam, K.H.; Lai, T.M. 2015. Importanceperformance ratings for environmental practices among 

Hong Kong professional-level employees. Journal of 

Cleaner Production. 108: 699–706.

USDA Forest Service. 2016. National Visitor Use Monitoring 

Program. http://www.fs.fed.us/recreation/programs/nvum/. 

[Date accessed: February 29, 2016]. 

Vaske, J.J.; Shelby, L.B. 2008. Crowding as a descriptive 

indicator and an evaluative standard: results from 30 years 

of research. Leisure Sciences. 30(2): 111–126.

Zarnoch, S.J.; White, E.M.; English, D.B.K. [and others]. 

2011. The National Visitor Use Monitoring methodology 

and final results for round 1. Gen. Tech. Rep. SRS–144. 

Asheville, NC: U.S. Department of Agriculture Forest 

Service, Southern Research Station. 74 p. http://www.srs.

fs.usda.gov/pubs/gtr/gtr_srs144.pdf. [Date accessed: March 

29, 2016].

Ziegler, J.; Dearden, P.; Rollins, R. 2012. But are tourists 

satisfied? Importance-performance analysis of the whale 

shark tourism industry on Isla Holbox, Mexico. Tourism 

Management. 33(3): 692–701.


Askew, Ashley E.; Bowker, J.M.; English, Donald B.K.; Zarnoch, Stanley J.; and Green, Gary T. 2017. A temporal 

importance-performance analysis of recreation attributes on national forests: a technical document supporting the Forest 

Service update of the 2010 RPA Assessment. Gen. Tech. Rep. SRS–223. Asheville, NC: U.S. Department of Agriculture 

Forest Service, Southern Research Station. 34 p.

The outdoor recreation component of the 2010 Resources Planning Act (RPA) Assessment provided projections and 

modeling of participation and intensity by activity. Results provided insight into the future of multiple outdoor recreation 

activities through projections of participation rates, number of participants, days per participant, and total activity days. 

These projections can be considered in managing potential shifts in outdoor recreation. In anticipation of projected trends, 

resources can be allocated appropriately by each outdoor recreation activity, or by the settings in which these activities take 

place. Decades of marketing and consumer research have established that an industry depends on and benefits from satisfied 

customers. In this study, we apply a widely used tool with origins in marketing research known as importance-performance 

analysis (IPA) to better understand visitor satisfaction with recreation settings on national forests. The results from IPA 

can supplement planning for improving efficient provision of recreation opportunities through management of setting 

attributes, which in turn could reinforce activity popularity and intensity. With the systematic and consistent data collection 

methodology in place from the National Visitor Use Monitoring Program, we assessed visitor ratings of importance and 

satisfaction on multiple attributes related to national forest recreation, as well as overall satisfaction and crowding ratings, 

over two recent survey periods (rounds 2 and 3) spanning 10 years. These attributes ranged from those in the natural 

domain (e.g., condition of natural environment and quality of scenery) to the developed setting (e.g., availability of parking 

and condition of parking lots). We classified the attributes into one of four IPA management actions and also conducted 

an intertemporal analysis to assess sustainability. The significance of changes in importance and satisfaction between 

rounds can provide insight into potentially problematic shifts, such as declining satisfaction in conjunction with increasing 

importance. Overall, users of the national forests were satisfied and found the attributes important. We identify relative 

overperformers and underperformers to better inform the allocation of resources within site type per attribute for the RPA 

regions and the Nation.

Keywords: Importance-performance analysis, National Visitor Use Monitoring (NVUM) Program, outdoor recreation, 

recreation setting attributes, Resources Planning Act (RPA) Assessment, visitor satisfaction.

How do you rate this publication?

Scan this code to submit your feedback 

or go to www.srs.fs.usda.gov/pubeval.

A copy of this general technical report is 

available for download at www.srs.fs.usda.

gov/pubs/.

In accordance with Federal civil rights law and U.S. Department of Agriculture (USDA) civil rights regulations 

and policies, the USDA, its Agencies, offices, and employees, and institutions participating in or administering 

USDA programs are prohibited from discriminating based on race, color, national origin, religion, sex, gender 

identity (including gender expression), sexual orientation, disability, age, marital status, family/parental status, 

income derived from a public assistance program, political beliefs, or reprisal or retaliation for prior civil rights activity, in 

any program or activity conducted or funded by USDA (not all bases apply to all programs). Remedies and complaint filing 

deadlines vary by program or incident. 

Persons with disabilities who require alternative means of communication for program information (e.g., Braille, large 

print, audiotape, American Sign Language, etc.) should contact the responsible Agency or USDA’s TARGET Center at 

(202) 720-2600 (voice and TTY) or contact USDA through the Federal Relay Service at (800) 877-8339. Additionally, 

program information may be made available in languages other than English. 

To file a program discrimination complaint, complete the USDA Program Discrimination Complaint Form, AD-3027, 

found online at http://www.ascr.usda.gov/complaint_filing_cust.html and at any USDA office or write a letter addressed 

to USDA and provide in the letter all of the information requested in the form. To request a copy of the complaint form, 

call (866) 632-9992. Submit your completed form or letter to USDA by: (1) mail: U.S. Department of Agriculture, 

Office of the Assistant Secretary for Civil Rights, 1400 Independence Avenue, SW, Washington, D.C. 20250-9410; 

(2) fax: (202) 690-7442; or (3) email: program.intake@usda.gov.

USDA is an equal opportunity provider, employer, and lender.

General Technical Report SRS–223 August 2017