September 2010

Regional Technology Strategies, Inc. with

Mt. Auburn Associates and

Appalachian State University

www.rtsinc.org

Program Evaluation of ARC’s 

Tourism, Cultural Heritage and 

Natural Asset-Related Projects 

Prepared for:

The Appalachian Regional Commission
Program Evaluation of ARC’s Tourism, i 

Cultural Heritage and Natural Asset-Related Projects

Cover photos courtesy of Jason Riedy under a Creative Commons license 
Program Evaluation of ARC’s Tourism, ii

Cultural Heritage and Natural Asset-Related Projects

Table of Contents

EXECUTIVE SUMMARY ............................................................................................. iv

SECTION 1: INTRODUCTION AND METHODOLOGY ......................................1 

SECTION 2: PROJECT DATA AND ANALYSIS ......................................................5 

SECTION 3: ESTIMATING IMPACTS FROM THE PORTFOLIO OF ARC 

TOURISM, CULTURAL HERITAGE AND NATURAL ASSET-RELATED 

PROJECTS .......................................................................................................................29 

SECTION 4: REFINING METRICS AND METHODS FOR MEASURING 

EMPLOYMENT IMPACTS ..........................................................................................31 

SECTION 5: SUSTAINABILITY & THE TRIPLE BOTTOM LINE .....................44 

SECTION 6: TBL & SUSTAINABLE TOURISM .....................................................55 

SECTION 7: THE ARC PROJECT PORTFOLIO & THE TBL ...............................64 

SECTION 8: A MODEL FOR IMPLEMENTING TBL IN RURAL AMERICA .66

Appendices

Appendix A Advisory Panel list .............................................................................. A-1 

Appendix B Survey Instrument ................................................................................B-1 

Appendix C Survey Summary Report .................................................................... C-1 

Appendix D Protocols ................................................................................................ D-1 

Appendix E Case Studies ........................................................................................... E-1

Appendix F The Triple Bottom Line: Overview and 

Application to Tourism ............................................................................................... F-1 

Appendix G Sustainable Development / Triple Bottom Line 

Literature Review ........................................................................................................ G-1
Program Evaluation of ARC’s Tourism, iii

Cultural Heritage and Natural Asset-Related Projects

This page left intentionally blank 
Program Evaluation of ARC’s Tourism, iv

Cultural Heritage and Natural Asset-Related Projects

EXECUTIVE SUMMARY

The Regional Technologies Strategies (RTS) project team, including Mt. Auburn 

Associates and Appalachian State University, was charged with examining and critiquing 

the Appalachian Regional Commission's (ARC) investment in Tourism, Cultural 

Heritage and Natural Asset-Related projects with a specific focus on how those projects 

were evaluated. The projects were examined within the context of the ARC’s Strategic 

Plan entitled Moving Appalachia Forward: ARC Strategic Plan, 2005–2010.

Goal 1: Increase job opportunities and per capita income in Appalachia to reach 

parity with the nation

Goal 2: Strengthen the capacity of the people of Appalachia to compete in the global 

economy 

Goal 3: Develop and improve Appalachia's infrastructure to make the Region 

economically competitive

Goal 4: Build the Appalachian Development Highway System to reduce Appalachia's 

isolation

Goals 1, 2 and 3 are most relevant to the tourism projects analyzed here. These projects 

were funded to directly and indirectly improve the ability of ARC residents to build 

sustainable economic futures based on the heritage, history, beauty and internal 

entrepreneurial resources of the region. 

We were asked to look at the portfolio of funded projects, how they were evaluated under 

ARC guidelines, and examine the projects’ reported impacts. From there we examined 

how well the evaluation guidelines helped grant recipients tell the story of the projects’ 

successes and failures. We then considered ways in which the evaluation procedures and 

rules could be modified to help grant recipients and ARC improve the evaluation, use 

evaluation to improve the progress of on-going projects and finally give a more robust, 

holistic and complete picture of the impacts these important programs have on 

Appalachian people and their communities. 

At the same time RTS was asked by the ARC and the Ford Foundation to take our 

analysis to a next step and examine how the projects impact not just direct economic 

success but how they simultaneously positively or negatively impact social and 

environmental goals as well. This broader perspective, sometimes called sustainable 

development or the Triple Bottom Line (TBL) is relevant to the types of economic 

development projects typically funded by ARC within the tourism program. 

Consideration of TBL issues is not an explicit element of the ARC 2005-2010 Strategic 

Plan but is embedded within the strategies that follow from its four main goals.

ARC and the Ford Foundation agreed to informally use these simultaneous research 

efforts to more fully and systematically look at how the organizations can improve the 

lives of the people of Appalachia through efforts that build on their mutual commitment 

to economic, social and environmental progress. This document embeds some of the 

TBL work completed by the RTS team. 
Program Evaluation of ARC’s Tourism, v 

Cultural Heritage and Natural Asset-Related Projects

We look to provide analysis and guidance on evaluating projects to meet the needs of 

ARC and where appropriate suggest ways that evaluation methods can be modified to 

reflect the broader vision of ARC as reflected in its Strategic Plan. In particular this 

report looks at the following questions: 

Is ARC using the best available metrics?

Do the metrics tell ARC what it needs to know to evaluate their programs?

Are the metrics easily measured and verified?

Do the metrics and evaluations assist recipients in managing their projects?

Do the metrics and evaluations provide ARC with the documentation required to 

substantiate funding requests for its programs? 

What are recommendations to build a new framework for evaluation?

What is TBL and what is its potential application to ARC projects?

PROJECT ANALYSIS 

The analysis begins with a portfolio of 132 ARC projects within the Tourism, Cultural 

Heritage and Natural Asset-Related program. Total project costs were $28.8 million of 

which ARC provided $10.8 million. Reported actual outcomes included 583 job created, 

520 jobs retained and 55 new businesses created. The projects were spread amongst all 

the ARC states. 

We analyzed the projects using three main techniques:

Two separate surveys: We sent surveys to project managers on two separate 

occasions. The first on-line survey primarily addressed qualitative metrics while 

the second survey measured specific results (outputs) and outcomes of the project, 

as well as impacts across the triple bottom line. The first survey had a response 

rate of 51 percent while the second had a response rate of 60 percent. 

Interviews: The project team interviewed representatives from 32 projects. These 

included both project managers and community “stakeholders.” During the 

process 93 (32 project managers and 61 stakeholders) individuals were 

interviewed. 

Case Studies: For a selection of the projects, project staff conducted detailed case 

studies. These included site visits to project locations and interviews with both 

project representatives and community stakeholders. 

Survey Results

First Survey

Our initial survey of project managers found a high degree of satisfaction with their 

projects. Over 86 percent felt that their project had mostly or completely met its goals 

and two-thirds reported that the project was still significantly in place and use (Figures 

ES-1 and ES-2). 
Program Evaluation of ARC’s Tourism, vi

Cultural Heritage and Natural Asset-Related Projects

Figure ES-1. To what extent did the project achieve its goals?

A l ttle

4% Some

10%

Mostly

43%

Completely

43%

Figure ES-2. To What Extent are the Initiatives in Use?

Not at all used

9%

Minor Use

12%

Some Use

12%

Strong Use

22%

Completely in Use

45%
Program Evaluation of ARC’s Tourism, vii

Cultural Heritage and Natural Asset-Related Projects

The greatest service improvements reported on the survey were to 1) tourism attractions, 

2) cultural facilities, 3) visitor facilities, 4) small business support and 5) education and 

training (Figure ES-3). These projects and their services in turn had the most significant 

impacts on 1) preservation of cultural heritage, 2) tourism revenues, 3) employment, 4) 

visual landscape and 5) sales of local goods (Figure ES-4).

The survey, for the most part reflected limited impacts to non-economic TBL measures 

with the exceptions of cultural issues and preservation or improvement of natural and 

built environments such as visual landscapes.

Second Survey

The primary purpose of the second survey was to measure specific results – outputs and 

outcomes of the project. Outputs focused on the numbers of individuals, businesses and 

communities served through the project, the amount of additional funds leveraged by the 

project, any materials developed through the project, and any programs and plans 

developed. Outcomes focused more on quantifiable measures of project success e.g. jobs 

created and retained, businesses improved, communities improved, etc. In all cases, 

project managers were asked to assign a number value to the project’s impact 

Table ES-1 presents the survey results and, where available, comparable data from the 

ARC project database for outputs. The “Reported on Survey” column refers to actual 

estimated impacts within the categories provided by respondents for their projects. The 

“Projected Outputs” column refers to the projected outputs for the same projects from the 

ARC database. We used projected instead of actual data reported for two reasons. First 

the actual number is provided at project closeout and therefore does not reflect the “three 

years once the ARC-funded services are delivered or the project is completed.” 

Table ES-1. What specific results (outputs) were actually achieved by this project?

Outputs Category

Reported 

on Survey

Projected 

Outputs

Ratio 

Reported to 

Projected

Participants Served 1,322,520 89,591 14.8 

Businesses Served 2,790 2,664 1.0 

Nonprofit Entities Served 892 NA NA 

Public Agencies Served 141 NA NA 

Communities Served 880 NA NA 

Visitor Attractions Developed 346 NA NA 

Programs and Plans Developed 474 31* 15.3 

Meetings and Events Held 1,236 NA NA 

Promotional Materials Developed 1,405,918 NA NA 

 * - Total of "New Programs Developed” and "New Strategic Planned Developed"

 NA -- Information not available in ARC project database 
Program Evaluation of ARC’s Tourism, viii

Cultural Heritage and Natural Asset-Related Projects

Second, it is important to compare the actual numbers to the projections to start to 

understand whether projects are successful and whether project developers have the 

expertise and ability to estimate impacts and track actual results. The results from the 

first survey, our interviews, and the case studies suggest that project managers are 

concerned about their ability to project and track project impacts.

As reflected in Table ES-1 the 69 projects reflected in the survey demonstrate substantial 

impacts with the regions they serve. The survey reflects that the projects served many 

more participants and generated substantially more programs and plans than projected. 

While this reflects respondents’ experience it is likely that the definitions that were used 

in the original proposals were not the same ones that the respondents used for the survey. 

This is an important result as it provides more evidence that grantees do not have robust 

or consistent understanding of the measures that ARC focuses on. Unless everyone is 

“on the same page” it is unlikely that reported results can be meaningfully compared to 

the original projections.

The “Ratio Reported to Projected” can provide an effective and succinct measure of 

project effectiveness if the definitions used are consistent. In the three instances where 

survey output answers can be compared to the ARC database, two categories, 

“participants served” and “programs and plans developed”, likely reflect definitional 

inconsistency. It is unreasonable to expect that the projects will serve, for example, 15 

times more participants than projected. For “Businesses Served” the ratio suggests that 

the project leaders were able to make reasonable estimates of this measure and deliver 

outputs. 
Program Evaluation of ARC’s Tourism, ix

Cultural Heritage and Natural Asset-Related Projects

Table ES-2. Who actually benefited from this project? What results were actually 

achieved?

Outcomes Category

Reported on 

Survey

Projected 

Outcomes

Ratio 

Reported to 

Projected

Participants Improved 363,594 NA NA 

Businesses Improved 872 NA NA 

Nonprofit Services Improved 190 NA NA 

Public Services Improved 111 NA NA 

Communities Improved 502 NA NA 

Leveraged Private Investment $19,343,116 $47,511,751 0.4* 

Jobs Created 1,257 1,783 0.7

Jobs Retained 512 306 1.7 

Businesses Created 110 39 2.8 

Businesses Retained 106 NA NA 

Business Sales Increased $7,962,073 NA NA 

 NA -- Information not available in ARC project database

 * Removing a questionable $30 million leveraged investment projection in a 

single project results in a ratio of 1.1.

The survey reflects that the projects had substantial positive impacts (outcomes) within 

the communities they served. For example, from Table ES-1 we see that 2,790 

businesses were served (a specific output) in some fashion and that output was a factor in 

improving 872 businesses (a specific outcome). The survey results for jobs created and 

retained and businesses created indicates that while these measures are fairly obvious and 

well understood the ability to project the impacts is not a trivial effort. 

In addition to the output and outcome measures, the second survey asked each grant 

recipient to rate their projects impact on their communities’ economic health, on issues of 

economic competitiveness, on social issues and on the environment. For each category, 

project managers were asked to state their project’s impact on a scale ranging from very 

negative to very positive. Significantly, on no category did recipients rate their impact as 

either negative or very negative.

Survey respondents indicated the greatest economic impact coming in three main outputs: 

Business assets/revenues, public assets/revenues and employment, with all three of these 

indicators having 67 or 68 percent of respondents showing a positive or very positive 

impact. Table ES-3 shows the respondents’ estimate of economic impact of their 

projects.
Program Evaluation of ARC’s Tourism, x 

Cultural Heritage and Natural Asset-Related Projects

Table ES-3: On a scale from very negative to very positive, how would you rate the 

impact of your project on the following economic, competitiveness, social and 

environmental measures?

Answer Options Very 

negative Negative Neutral Positive Very Positive 

Population 0.0% 0.0% 56.1% 30.3% 13.6%

Employment 0.0% 0.0% 31.8% 57.6% 10.6%

Personal income 0.0% 0.0% 50.0% 40.9% 9.1%

Household assets/ 

wealth 0.0% 0.0% 69.7% 24.2% 6.1%

Business assets/ 

revenues

0.0% 0.0% 31.8% 56.1% 12.1%

Public assets/ revenues 0.0% 0.0% 33.3% 50.0% 16.7%

Job Stability 0.0% 0.0% 50% 43.9% 6.1%

Efficiency 0.0% 0.0% 51.5% 34.8% 13.6%

Productivity of land, 

labor, energy or capital 0.0% 0.0% 57.6% 31.8% 10.6% 

Access to markets 0.0% 0.0% 50.0% 28.8% 21.2%

Civic life and

governance

0.0% 0.0% 43.9% 36.4% 19.7% 

Health 0.0% 0.0% 65.2% 25.8% 9.1%

Education 0.0% 0.0% 28.8% 43.9% 27.3%

Public safety and 

access

0.0% 0.0% 71.2% 19.7% 9.1% 

Culture, arts, other 

amenities 0.0% 0.0% 16.7% 36.4% 47.0% 

Public services 0.0% 0.0% 57.6% 30.3% 12.1%

Other community 

assets 0.0% 0.0% 19.7% 48.5% 31.8% 

Air & water quality 0.0% 0.0% 78.8% 13.6% 7.6%

Land and natural 

resources

0.0% 0.0% 47.0% 31.8% 21.2% 

Recreational 

opportunities 0.0% 0.0% 31.8% 25.8% 42.4% 

Respondents were also asked to estimate the impact their project on various indicators of 

a region’s competitiveness, including job stability; efficiency; productivity of land, labor, 

energy or capital; and, access to markets. In this set of indicators, the responses tended to 

be more ambiguous, with more than half stating that their project had a neutral impact in 

each of the four indicators. Increased access to markets had the most positive response, 

with 21% stating that their project had a very positive impact on the region’s access to 

markets.

The social measures most positively impacted by the ARC-funded projects tended to be 

ones in which the increase was an explicit goal. Thus, the fact that 83% of respondents 
Program Evaluation of ARC’s Tourism, xi

Cultural Heritage and Natural Asset-Related Projects

indicated that they there was a positive impact on culture, arts and other amenities is 

likely due to the fact that so many projects focused on expanding those very offerings. 

Similarly, many of the funded projects focused on expanding educational offerings 

through enhanced training or programs, making the 72% who pointed to a positive or 

very positive impact on education a more likely result. 

The respondents were asked to gauge their impact on a set of environmental indicators. 

In terms of air and water quality, the fact that none of the respondents believed that their 

project had a negative impact on the environment is impressive. Since hardly any of the 

projects had explicit environmental focuses, it is not surprising that the vast majority of 

respondents thought their project had at best neutral environmental impacts. The 

indicator in which most respondents assigned a positive or very positive impact was on 

expansion of recreational opportunities. 

Interview Results

As noted above, the RTS team conducted interviews for 32 projects including 32 project 

managers or directors and 61 stakeholders who saw impacts from their perspective. 

The interviews reveal a positive and optimistic attitude about projects, despite having to 

deal with numerous challenges. Overall, interviews reveal that: 

The projects would not have been viable without ARC funding and for most the ARC 

grant started the project;

Nearly all project managers reported overall positive relationships with ARC and its 

staff;

A significant number reported that the “jobs created” measure was insufficient to 

measure the true impact of the project;

Some projects acted as a catalyst for environmentally friendly development; 

Many interviewees reported an improvement in community collaboration; and 

Most cited that the most difficult challenge or impediment to success had to do with 

funding 

The interviews show that ARC is considered a key partner from both a funding 

standpoint and as an agency that works with the local grantees in a helpful and respectful 

way. There were questions raised about cumbersome application and reporting 

procedures and the inability of the “jobs created” metric to reasonably reflect true project 

impacts. As one stakeholder remarked, it often seemed that the formula being used was 

designed for a completely different kind of project. 

Lastly, the extensive comments on projects as a catalyst for environmentally friendly 

development and improvements in community collaboration reflect the role that ARC 

projects are already serving within the TBL framework.
Program Evaluation of ARC’s Tourism, xii

Cultural Heritage and Natural Asset-Related Projects

Figure ES-3. To what extent did the project result in the 

development, expansion, or enhancement of the following? 

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Tourism attractions

Cultural facilities and events

Support for small businesses

Education and training programs

Visitor facilities and services

Community facilities and services

Recreational facilities

Local conservation activities

Amount of public open space

Support for local agriculture

Infrastructure (e.g., water & sewer)

Environmentally-friendly transport

Moderate to Great Impact No or Small Impact Don't Know

Figure ES-4. What impact did the project have on the following?

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Preservation of cultural heritage

Tourism revenues

Employment opportunities

Visual landscape

Sales of locally-produced items

Quality of jobs

Household incomes

Employment of high-skilled labor

Population retention or growth

Poverty reduction

Water conservation

Air and water pollution reduction

Resident health

Energy conservation

Positive Impact Slight Impact No Impact Don't Know
Program Evaluation of ARC’s Tourism, xiii

Cultural Heritage and Natural Asset-Related Projects

Case Study Results

The case studies provide the most nuanced view of the project process. Some of the most 

thought-provoking impressions include: 

Projects, by necessity, often evolve from initial conception. Flexibility in project 

implementation based on changing circumstances has the potential to improve 

projects and their impacts. 

The vectors of impact were broader than initially envisioned. Increased 

collaboration and spin-off projects were among the areas where unforeseen 

positive were found. 

Impacts are often difficult to track, estimate and justify. There are a number of 

reasons for this including technical challenges, lack of grantee expertise in data 

collection and evaluation, and resource shortages. 

Local and regional politics and relationships can have significant positive and 

negative impacts. In some cases, potential partners did not collaborate lessening 

the potential impact of the project.

Project proponents often felt the measured project metrics did not reflect the most 

important impacts of the project. 

There are often un-measured spillover and synergy impacts from projects. This is 

particularly evident in the Crooked Road region of southwest Virginia where the 

state and ARC have embarked on a strategic series of projects that have helped 

create a creative cluster.

As in the interviews, we were told that many projects had impacts on the social 

and environmental metrics of TBL even though the projects had not been planned 

to affect these elements.

Clearly the excellent working relationship between ARC and its grantees bodes well for 

developing and implementing positive improvements to the evaluation process. This 

positive and trusting relationship is a platform for positive and collaborative 

enhancement. Project grantees will be a key element in efforts to improve the evaluation 

process.

ESTIMATING IMPACTS 

The quantitative portion of the second survey combined with the full universe of projects 

in the ARC database provides a structured method to estimate the overall impacts of the 

tourism, cultural heritage and natural asset-related portfolio of projects funded by ARC. 

This survey enabled us to collect data from project managers in a way that mostly avoids 

the problem of the estimated impacts provided at project closeout. The final report 

submitted to ARC at the completion of a project is required to estimate “actual” impacts. 

The estimates are supposed to include impacts going forward three years under the 

correct assumption that impacts normally take time to occur after a project is completed. 

The survey strongly suggests that under these circumstances the managers focus more on 

present conditions, not three years out, and appear to vastly underestimate project 

impacts.
Program Evaluation of ARC’s Tourism, xiv

Cultural Heritage and Natural Asset-Related Projects

In our survey, the project managers were able to look back at their projects after more 

time had elapsed and projects had had a chance to mature and generate impacts within 

their communities. When we compare the survey estimates which are based on a longer 

time horizon we find that the estimated impacts are much more consistent with the initial 

pre-project projections rather than the immediate post closure estimates. 

Table ES-2 includes a column that is a ratio of the survey estimates of outcomes 

compared to the initial projected outcomes from the original proposal for jobs created and 

retained, businesses created and leveraged private investment. As an example, the survey 

indicates that the surveyed projects generated 1,257 jobs compared to the initial 

projection for those projects of 1,783 jobs, generating a ratio of 0.7. The proposals 

appear to have somewhat overestimated the ultimate job impacts. For jobs retained the 

survey estimate of 512 is higher than the original projection of 306, generating a ratio of 

1.7 suggesting that the original projections underestimated the ultimate impacts.

Assuming that the 69 projects for which surveys were completed are representative of the 

full universe of 132 projects allows us to generate estimates of impacts for the full 

universe using the estimated ratios.

Table ES-4 displays the generated estimates of impacts from the ARC portfolio of 

tourism, cultural heritage, and natural resource-related projects.

Table ES-4 Estimated Impacts of the Universe of ARC Tourism Projects

Impact Category

Initial 

Projection 

of Impacts

Surveybased 

Adjustment 

Ratio

Estimated 

Post Project 

Impacts 

Estimated 

Units Impacts 

of ARC 

Funding

Jobs Created 3,671 0.70 2,588 $4,161 

Jobs Retained 5,616 1.67 9,397 $1,146 

Leveraged Private Investment $65,575,691 0.41 $26,697,357 $0.40 

New Businesses Created 165 2.82 465 $23,139 

Businesses Served 7,148 1.05 7,486 $1,438 

The Initial Projection of Impacts column is the sum of the original projections from the 

proposals submitted to ARC as reflected in the ARC provided database. The Estimated 

Post Project Impacts column adjusts the projected impacts based on the ratios estimated 

from the survey results. So, for example, applying the 0.7 ratio we estimated to the 

original projection for job creation gives us an estimate of 2,588 jobs created by the 132 

projects examined in this project.

The Estimated Unit Impacts of ARC Funding column shows that a new job was created in 

the community for every $4,161 of ARC-provided dollars. A new business was created 
Program Evaluation of ARC’s Tourism, xv

Cultural Heritage and Natural Asset-Related Projects

for every $23,139 in ARC funding. For every $0.40 invested by ARC in the projects, 

$1.00 of leveraged private investment occurred. 

REFINING AND IMPROVING METRICS 

The tourism, cultural heritage and natural asset-related projects included in this 

evaluation primarily address Goal 1 of ARC’s 2005-2010 strategic plan – to “increase job 

opportunities and per capita income in Appalachia to reach parity with the nation.” ARC 

uses two measures to assess the outcomes of projects funded to achieve this goal. The 

primary measure is number of jobs created and retained and the secondary measure is 

leveraged private investment (LPI). 

ARC has developed very specific definitions of each of these measures, as described in 

its Fiscal Year 2006 Performance and Accountability Report: 

 “Jobs created” includes direct hires that will be made as a result of the project’s 

operation. Also included are private-sector jobs that will be created within three 

years after project completion. 

“Jobs retained” refers to the number of workers enrolled in specific training 

programs or to the number of jobs at businesses that will be retained because of an 

investment that makes the companies more competitive. 

“Leveraged private investment” represents private-sector, non-project financial 

commitments that follow and are the result of the completion of an ARC-supported 

project or the delivery of services under an ARC-supported project. 

It is clear that job creation and retention are fundamental to ARC’s mission and that the 

commission’s congressional overseers assess its performance largely on this basis. As it 

stands the present system’s methodology provides insufficient means for grantees to 

accurately measure job impacts. In fact, as it stands now, grantee reporting provides little 

useful information about employment impacts and does not permit ARC to develop 

meaningful estimates of the overall employment impacts of this group of projects. 

Current problems

ARC’s tourism, cultural heritage and natural asset-related projects typically seek to 

achieve one or both of two direct outcomes:

Increase visitation and tourism revenues 

Increase sales of locally produced products

These outcomes can lead to job creation/retention and additional private investment. 

Measuring these employment and investment impacts requires two steps. First, accurate 

data must be obtained on the amount of tourism revenues or business sales generated. 

Second, methods must be employed whereby estimates of the impacts of increases in 

tourism revenues and business sales on employment and private investment can be 

developed. 
Program Evaluation of ARC’s Tourism, xvi

Cultural Heritage and Natural Asset-Related Projects

Examination of data reporting methods for the projects included in this evaluation 

uncovered a number of problems that prevent ARC from obtaining accurate and complete 

data on jobs and private investment. 

 Methods Used by Grantees to Document Employment Impacts are 

Inconsistent and Often Methodologically Unsound 

For tourism projects well-established methods exist to measure the economic impacts of 

tourism projects. These involve a three-step process. First, mechanisms are put in place 

to track the number of visitors to tourism attractions. Second, surveys are conducted to 

ascertain the role played by the marketing of attractions and the resulting amount of 

visitor spending that occurs. Third, economic models are used to estimate the impact of 

increased visitor spending on local income and full-time equivalent (FTE) employment. 

ARC tourism grantees do not, by and large, employ these methods. Some do not report 

employment impacts at all, but use proxies such as increased visitation to tourism 

attractions. Some measure employment impacts using other methods. Depending on 

method, they can lead to systematic over- or under-estimation of impacts. In addition this 

haphazard approach makes it difficult for ARC to compare impacts across projects.

Adopting established methods for economic impact analysis of tourism projects, while 

adding complexity and cost to the outcome reporting process, could at least partly address 

these problems. 

 There is No Practical Way to Measure Leveraged Private Investment (LPI) 

for Most Projects 

In some cases, private investment leveraged by an ARC-funded project can be identified 

and measured. Even in these cases, however, methodological issues arise, for example, 

on purchasing versus leasing and on how to handle working capital. 

Since leveraged private investment is essentially used to generate job impacts, a preferred 

method would be to skip the LPI and move directly to estimating jobs impacts. An 

impact assessment methodology that fully captured employment impacts would obviate 

the need to measure LPI. If additional local sales generated by ARC-funded projects 

could be estimated in a reasonably accurate manner, applying economic impact models as 

could provide a better picture of impacts. 

 The Project Reporting Time Frame is Too Short to Fully Capture 

Employment Impacts 

Most of the outcome data received by ARC from grantees is through the final report, 

which is submitted within 30 days of the end of the project and in most cases before 

significant employment impacts have occurred. At best, the reports can inform ARC 

about the extent of project activity completion. Perhaps the most consistent frustration 
Program Evaluation of ARC’s Tourism, xvii

Cultural Heritage and Natural Asset-Related Projects

voiced by project managers was that the employment impacts occur after the project is 

complete. While ARC staff conducts validation visits for a small number of projects two 

or three years after project close-out, the sample is far too small to draw any inferences 

about the longer-term employment impacts of these projects. 

Instituting post-project reporting requirements could at least partly address this problem, 

although at additional cost to grantees. Even then the methodological issues for grantees 

would be formidable. 

 Impacts of Many ARC-funded Tourism Projects Cannot be Easily Isolated 

from Other Factors Influencing Visitation and Visitor Spending 

Additional tourism spending can be more easily attributable to some, typically larger, 

ARC tourism projects than others. Established impact analysis methods can estimate the 

impacts of these types of projects if the appropriate visitation and spending data are 

collected. We refer to these as “tourism generators.”

However, there are many other, typically small, ARC projects that, while contributors to 

an area’s tourism development, can’t easily be linked to additional visitor spending. 

Methods could be employed to track these smaller projects but it would be difficult to 

accurately assess their role in increasing overall visitor spending. These projects are 

more accurately described as “tourism contributors.”

RECOMMENDATIONS FOR A NEW FRAMEWORK FOR 

MEASURING EMPLOYMENT IMPACTS 

These issues suggest that ARC should develop a new framework for measuring the 

employment impacts of its project portfolio. They also suggest that a number of criteria 

should be applied in establishing this recommended framework: 

Methods employed should be reasonably reliable and consistent. 

They should be designed to ensure that the impacts measured are attributable to 

the ARC project. 

They should be made as easy as possible to execute and proportionate to the 

grantee’s resources and the cost of the project. 

They should be tailored to the project type and stage of development. 

 Establish Standardized Practices to Assist Grantees with Impact 

Measurement 

The most appropriate way, as noted above, to measure the employment impacts of ARC’s 

tourism projects is to first measure increases in local revenues attributable to these 

projects, and then convert these numbers into employment equivalents using economic 

impact models. These tools have been used extensively by academic researchers, 

government tourism agencies, and private consultants. This would require technical and 
Program Evaluation of ARC’s Tourism, xviii

Cultural Heritage and Natural Asset-Related Projects

funding assistance for grantees and preferably the development of an online economic 

development model that can be directly accessed by the grantees.

 Tailor Measurement Methods to Project Characteristics 

While measurement methods should be standardized, they should also be tailored to take 

into account different project types and time frames. Most, importantly, the impacts of 

implementation projects should be measured differently than those of planning projects. 

Another distinction should be made between tourism implementation projects that are 

“tourism generators” and those that are “tourism contributors.” 

 Require and Assist Applicants to Develop an Employment Impact 

Measurement Plan 

ARC should require all applicants to prepare an employment impact measurement plan as 

part of their project application. The plan should describe what impacts will be measured, 

what methods will be used, when it will be done, and who will be responsible for data 

collection and analysis. A budget line for impact measurement should be included in the 

project budget and ARC should be prepared to assist applicants with this process. 

 Establish Post-grant Reporting Requirements

As noted earlier, it is typically premature to measure the employment impacts of this 

group of projects at the end of the grant period. Final reports should instead focus on 

reporting outcomes that can serve as preliminary indicators of the nature and extent of 

employment impacts that might be expected within 1-3 years of project completion. 

In order to obtain meaningful information on actual employment impacts, ARC should 

require, with some incentive mechanism, grantees to submit post-grant reports at 

specified intervals, perhaps one and three years after project implementation. The threeyear reporting interval meshes with ARC’s definition of job creation, which anticipates 

measurement of private sector jobs created within three years of project completion while 

the one-year report would be used as a management tool to help with project needed 

modifications. 

AN EX POST FACTO ESTIMATE OF EMPLOYMENT IMPACTS 

The project files provided by ARC indicated that the projects (those that are closed) 

generated 583 jobs. Those projects were initially projected by grantees, as reflected in 

their grant requests, to create 2,113 jobs. The mass of evidence described in this report 

suggests that grant recipients cannot be expected to accurately estimate employment 

impacts and therefore this large differential is not surprising. We endeavored to find an 

alternative means of estimating the impacts of the project portfolio ex post facto.
Program Evaluation of ARC’s Tourism, xix

Cultural Heritage and Natural Asset-Related Projects

Using an analysis of the Crooked Road region of Virginia where ARC has made 

significant investments we compared reported project impacts to the estimated impacts 

based on a economic development model like those described above. Our basis was the 

2008 study conducted by Sustainable Development Consulting (SDC) and entitled 

Economic Impact Assessment of the Crooked Road: Virginia’s Heritage Music Trail. 

The study estimated that 445 full-time equivalent (FTE) jobs were created by the 

Crooked Road initiative. How does this compare to the estimated impacts as reported by 

ARC grantees for the same portfolio of projects? The grantees reported that their projects 

generated 80 jobs or 18% of the estimated total reported by SDC. On the other hand the 

445 FTE estimate is quite close to the initial projections made by the ARC grantees of 

416. 

This ratio of grantee job estimates to the economic model estimates provides a means of 

estimating job impacts of the full portfolio package of ARC tourism projects. Applying 

this ratio we estimate that the actual job impacts for the ARC portfolio are 3,243, 

compared to the reported estimate of 583. These results strongly support our 

recommendation that ARC investigate a practical implementation of an economic impact 

model to more accurately reflect the success of its investments in Appalachia.

THE TRIPLE BOTTOM LINE DEFINED

The Triple Bottom Line is built on the broader topic of sustainability. Sustainability is 

defined as “ development that meets the needs of the present without compromising the 

ability of future generations to meet their own needs.” TBL focuses more narrowly on 

specific activities (by companies, organization, governments, etc.) and their impacts on 

the economy, society and environment. The economic dimension is generally 

straightforward but the social and environmental less so. 

The Economic Dimension of TBL 

Perhaps the easiest dimension to capture during triple bottom line evaluation is the 

economic dimension – in this case employment, income and investment. Industryspecific measures such as for tourism are also readily available to assess the economic 

performance of an activity or project. 

The Social Dimension of TBL

The social dimension is often thought of in terms of social capital. Robert Putnam 

describes social capital as the “trust, norms and networks needed to facilitate 

cooperation.” A more robust framework specifies four dimensions of social impact, 

including an 1) individual’s well-being, 2) community well-being, 3) employment 

experiences and satisfaction, and 4) organizational impact. These are often not easily 

measured.
Program Evaluation of ARC’s Tourism, xx

Cultural Heritage and Natural Asset-Related Projects

The Environmental Dimension of TBL

The environmental dimension is referred to by some as natural capital. Natural resources 

such as air, water, energy, forests, minerals, and soil can be thought of as the “capital” 

upon which our existence depends. Measuring the environmental dimension can be 

complicated, time-consuming and expensive. 

The potential value of integrating TBL into ARC project planning and evaluation 

includes cost efficiencies (reduced energy and materials use), better living and work 

environments, more successful marketing of attractions as “green” and improved 

stakeholder relations. Also, adopting the TBL approach can improve a tourism 

development organization’s strategic decision making. 

As we note above, project managers easily identified TBL relevant impacts when 

discussing their projects. This was particularly true for the social dimension. 

A MODEL FOR IMPLEMENTING TBL IN ARC PROJECTS

The Conservation Fund’s “Creating Asset-Based Economies in Western North Carolina” 

project funded by ARC provides an excellent model for implementing TBL within the 

ARC portfolio. The project provides small grants to community organizations that 

support “triple bottom line” initiatives building a base of community support for 

entrepreneurship. The Conservation Fund provides technical assistance to the community 

organizations on implementing and evaluating their TBL project. 

Two features are fundamentally important to the project’s success:

It is intentional in nature; organizations must demonstrate that their proposed 

project integrates TBL as a basic operational goal.

The project requires and instills a collaborative, continual training and learning 

process that integrates evaluation into project design and management.

The technical assistance involves workshops before and during the project. The process 

itself is taught in the workshops using a straightforward manual that use seven steps to 

guide the grantees through designing the TBL project, its management and evaluation. 

The Conservation Fund program is not the only way to operationalize TBL into 

organizations receiving grants. It does provide an excellent building block for thinking 

about next steps for ARC to develop an explicit platform for building Appalachian Triple 

Bottom Line initiatives. The Conservation Fund program involves a level of handholding, 

training and workshops that may be unrealistic for an ARC-wide implementation.

We believe a practical program can be built. The skeleton of the process is outlined in 

the seven steps described in the Conservation Fund’s manual. ARC does not need to be 

involved in all parts of the process but we recommend two critical elements: 
Program Evaluation of ARC’s Tourism, xxi

Cultural Heritage and Natural Asset-Related Projects

An explicit acknowledgment by ARC of the relevance of TBL

ARC must be upfront in adopting the Strategic Plan’s suggestion that project success 

often goes beyond the traditional measures of employment, income and investment. 

While these traditional impacts provide the backbone for improving the lives and 

livelihoods of the people of Appalachia, they do not represent the complete picture. 

Within this acknowledgment ARC will need to clearly define the why and how of the 

TBL. This might include the development of a series of straightforward background 

guidance documents or manuals. 

Communicating a set of potential impacts and measurement options

The surveys, interviews and case studies we conducted, along with the Conservation 

Fund project experience, clearly show that grantees implicitly understand that the work 

they do within Appalachian communities impacts and is impacted by economic, social 

and environmental circumstances. Most grantees are used to thinking in terms of jobs, 

income and investment. Not only must they be assured that broader goals are acceptable, 

but they will need assistance in making the transition to a TBL perspective including 

defining potential TBL impacts and measurement methods for those impacts. 

For details on our analysis and recommendations please refer to the main report. 
Program Evaluation of ARC’s Tourism, Page 1 

Cultural Heritage and Natural Asset-Related Projects

SECTION 1: INTRODUCTION AND METHODOLOGY

THE SETTTING 

The Regional Technologies Strategies (RTS) project team, including Mt. Auburn 

Associates and Appalachian State University, was charged with examining and critiquing 

the Appalachian Regional Commission's (ARC) investment in Tourism, Cultural 

Heritage and Natural Asset-Related projects with a specific focus on how those projects 

were evaluated. The projects were examined within the context of the ARC’s Strategic 

Plan entitled Moving Appalachia Forward: ARC Strategic Plan, 2005–2010. The four 

overarching goals within the plan are: 

Goal 1: Increase job opportunities and per capita income in Appalachia to reach 

parity with the nation

Goal 2: Strengthen the capacity of the people of Appalachia to compete in the global 

economy 

Goal 3: Develop and improve Appalachia's infrastructure to make the Region 

economically competitive

Goal 4: Build the Appalachian Development Highway System to reduce Appalachia's 

isolation

For this analysis Goals 1, 2 and 3 are most relevant as the Tourism, Cultural Heritage and 

Natural Asset-Related projects were funded to directly and indirectly improve the ability 

of ARC residents to build sustainable economic futures based on the heritage, history, 

beauty and internal entrepreneurial resources of the region. These projects rely on 

building and rehabilitating facilities and infrastructure to make the region more accessible 

and attractive to visitors, provide ways and means for residents to build pride and 

community based on their assets and then market those assets, train crafts people, artisans 

and artists to improve the quality, markets and marketability of their works and train 

workers to serve in newly created jobs which the tourism and visitation markets created. 

Individual projects have specific and perhaps modest goals but the overall impact of the 

program is to be a key component in helping bring Appalachia and its mostly rural 

residents into the mainstream of US economic progress. 

We were asked to look at the portfolio of funded projects, how they were evaluated under 

ARC guidelines, and examine the projects’ reported impacts including the key metrics of 

jobs created and retained, businesses created, and leveraged private investment that are 

important in ARC’s reporting to the Office of Management and Budget (OMB). We 

examined the impact estimates and applied ARC data provided in combination with two 

surveys of project managers we developed reasonable estimates of the impact of the 

project portfolio on jobs, businesses, leveraged private investment and other parameters 

of interest. From there we examined how well the evaluation guidelines helped grant 

recipients tell the story of the projects’ successes and failures and whether the evaluation 

systems and reports provided ARC staff (both state and federal) with the information and 

guidance they need to effectively plan and evaluate their programs. From this baseline 

we then considered ways in which the evaluation procedures and rules could be modified 
Program Evaluation of ARC’s Tourism, Page 2 

Cultural Heritage and Natural Asset-Related Projects

to help both the grant recipients and ARC improve the evaluation, use evaluation to 

improve the progress of on-going projects and finally give a more robust, holistic and 

complete picture of the impacts these important programs have on Appalachian people 

and their communities.

At the same time RTS was granted funds by the Ford Foundation to take our analysis to a 

key next step and examine how the projects reflect and impact the increasing interest in 

looking at these efforts from a perspective that includes not just direct economic 

development but how they simultaneously positively or negatively impact social and 

environmental goals as well. This broader perspective, sometimes called sustainable 

development and within this report referred to as the Triple Bottom Line (TBL) for 

economic, social and environmental impact can be applied to the types of economic 

development projects typically funded by ARC. The application of TBL to ARC projects 

is developed in Sections 5-8 below and includes recommended steps if ARC adopts the 

TBL framework.

Consideration of TBL issues is not an explicit element of the ARC 2005-2010 Strategic 

Plan but is embedded within the strategies that follow from the four main goals and are 

documented within the Strategic Plan document. The full plan is available online at 

http://www.arc.gov/index.do?nodeId=2296. 

For example, the strategies within Goal 1 on increasing job opportunities and incomes 

begin with a strong focus on Civic Entrepreneurship, a key component in the social focus 

of TBL with its emphasis on leadership, collaboration and citizen involvement. Other 

strategies suggest a focus on using and protecting the historic and environmental assets to 

build long-term economic success. Strategies within the Goal 2 on global 

competitiveness speak to the need to improve childcare, education, health care and 

healthy lifestyles as ways to make the region more competitive but also to strengthen the 

social fabric as workers build skills and productivity. Goal 3 strategies to build 

infrastructure improvements include efforts to protect the land and water from 

environmental degradation. 

ARC and the Ford Foundation agreed to informally use these simultaneous research 

efforts to more fully and systematically look at how the organizations can improve the 

lives of the people of Appalachia through efforts that build on their mutual commitment 

to economic, social and environmental progress. This document embeds much of the 

TBL work completed by the RTS team. 

We look to provide analysis and guidance on evaluating projects to meet the needs of 

ARC and, where appropriate, suggest ways that evaluation methods can be modified to 

reflect the broader vision of ARC as reflected in its Strategic Plan. In particular this 

report looks at the following questions: 

Is ARC using the best available metrics?

Do the metrics tell ARC what it needs to know to evaluate their programs?

Are the metrics easily measured and verified?
Program Evaluation of ARC’s Tourism, Page 3 

Cultural Heritage and Natural Asset-Related Projects

Do the metrics and evaluations assist recipients in managing their projects?

Do the metrics and evaluations provide ARC with the documentation required to 

substantiate funding requests for its programs? 

Recommendations to build a new framework for evaluation 

TBL and its potential application to ARC projects

REPORT STRUCTURE AND METHODOLOGY 

ARC provided the RTS team with an initial universe of 212 projects that were available 

to review and evaluate. However, ARC trimmed that project list further if the project had 

been analyzed as a part of a recent entrepreneurial evaluation effort. In the course of the 

evaluation and in discussion between the team and ARC staff the project list was reduced 

further:

We removed projects from earlier than 2000. Problems were foreseen with 

evaluating older projects, as individuals with knowledge of the projects were 

likely to have moved on. 

We did not consider projects funded after 2006 because they were likely to be too 

early in project development for realistic evaluation. 

“Minor” projects, such as those that funded attendance at an offsite workshop or 

conference, were not considered.

These decisions left the project team with a universe of 132 projects to evaluate. 

To gather information about the projects, the team employed three main techniques: 

Surveys: two on-line surveys were sent to project managers. If the project 

manager was no longer with the organization, then other knowledgeable 

individuals were encouraged to complete the survey. In some early project cases 

we could not identify a suitable individual to take the survey. For the initial 

survey 98 surveys were sent (through electronic means) and 51 were completed, 

for a response rate of 52 percent. For the second survey 114 surveys were sent 

(through electronic means) and 69 were completed, for a response rate of 60 

percent. The survey instruments are included as Appendix B and summary 

analyses of the surveys can be found in Appendix D. 

Interviews: The project team interviewed representatives from 32 projects. These 

included both project managers and community “stakeholders.” These 

stakeholders represent local businesses or organizations whose operations were 

impacted by the funded projects. During the process 93 (32 project managers and 

61 stakeholders) individuals were interviewed. (The original interviews are 

provided as a separate document.) The RTS team also interviewed relevant ARC 

staff at the Washington D.C. headquarters building. The interview protocols for 

project managers and stakeholders are included in Appendix D. 

Case Studies: For a selection of the projects, project staff conducted detailed case 

studies. These included site visits to project locations and interviews with both 
Program Evaluation of ARC’s Tourism, Page 4 

Cultural Heritage and Natural Asset-Related Projects

project representatives and community stakeholders. The case studies are 

included in Appendix E. 

In addition to this detailed analysis of funded projects, the team also conducted 

significant research on the ways in which tourism projects have been traditionally 

evaluated and how the triple bottom line approach can be used to evaluate these types of 

projects. Accordingly, two literature reviews were commissioned with experts within the 

field, both of which are included as appendices F and G. 

The project team used the services of an advisory board of experts in the fields of rural 

economic development and evaluation to guide its work. The Advisory Board, whose 

membership is listed in Appendix A, met twice for two-day meetings in North Carolina, 

where they provided invaluable input on a wide range of project-related subjects. In 

addition, individual members of the Panel reviewed materials during the course of the 

project providing important feedback to project staff. 

This report includes the following sections: 

This introduction and description of methodology 

A general summary of the projects included in the analysis including surveys 

results and interview and case study learnings

Estimates of key performance measures for the portfolio of projects examined 

An analysis of the metrics and recommendations on the future direction 

A robust background and evaluation on the TBL and a proposed application to 

ARC projects

The series of supporting appendices 
Program Evaluation of ARC’s Tourism, Page 5 

Cultural Heritage and Natural Asset-Related Projects

SECTION 2: PROJECT DATA AND ANALYSIS

CHARACTERISTICS OF PROJECTS FROM ARC FILES

To choose and analyze the ARC tourism, cultural heritage and natural asset-related 

projects the RTS project team was provided with: 

An electronic spreadsheet with project data from the ARC project database of all 

projects funded within the program 

Project files for those projects that were ultimately included in the analysis. The 

project files had varying levels of detail.

Some basic characteristics of the 132 projects as determined from the project database:

The database included 72 closed and 60 open projects. As noted in Section 1, open 

projects were begun before 2007. 

Project sponsorship (Figure 1) was:

o Commission sponsored: 35

o Alabama: 6

o Georgia: 5

o Kentucky: 6

o Maryland: 7

o Mississippi: 3

o New York: 6

o North Carolina: 8 

o Ohio: 11 

o Pennsylvania: 7 

o Tennessee: 2

o Virginia: 29

o West Virginia: 7

Total project costs were $28.8 million of which ARC provided $10.8 million. 

The database includes initial projections and actual outcomes (for completed projects) 

on the key parameters that ARC uses in evaluating projects and reporting program 

results to the Office of Management and Budget. The key outcome indicators 

(projected and actual) include numbers of jobs created and retained, non-project 

private investment generated by the project (referred to as leveraged private 

investment), and new businesses created. 

The database also includes initial projections and actual outputs (for completed 

projects). Outputs are activities that can lead to positive impacts. Outcome indicators 

(projected and actual) include people served (e.g. students or trainees), businesses 

served or assisted and programs or plans created. 

Other outcomes that were sometimes but inconsistently reported include: 

o Numbers of visitors 

o Seasonal jobs 

o Documents (brochures, guides) produced 
Program Evaluation of ARC’s Tourism, Page 6 

Cultural Heritage and Natural Asset-Related Projects

o Artists and businesses served

o Training provided 

o Workshops held 

o Websites created

o Trails created

The following listing totals the reported actual and projected key performance outcomes 

and the ARC and total funding of projects from all 132 projects in the ARC provided 

database:

Category Measure

ARC Funding $10,768,574 

Total project funding all sources* $28,790,654 

Total jobs created actual 682 

Total jobs created projected 3,671 

Total jobs retained actual 520 

Total jobs retained projected 5,616 

Total new businesses created actual 55 

Total new businesses created projected 165 

Leverage Private Investment actual $926,200 

Leverage Private Investment projected $65,575,691 

* Includes ARC, local, other Federal, and state funding 

Figure 1. Projects by State

0

5

10

15

20

25

30

35

40

TennesseeMississippi

Georgia

Alabama

Kentucky

New York

Maryland

Pennsylvania

West Virginia

North Carolina

Ohio

Virginia

Commission
Program Evaluation of ARC’s Tourism, Page 7 

Cultural Heritage and Natural Asset-Related Projects

The project spreadsheet, while a good starting point, does not provide sufficient or 

consistent data that can realistically provide the data needed to judge evaluation. The 

project files provide a more consistent source of records such as ARC documentation, 

correspondence, and progress and final reports. Nevertheless, the project files have 

vastly different levels of detail. Grantee reports occasionally provide substantive levels 

of detail and critical information useful in in-depth evaluation but more often the reports 

fulfill the requirements but do not give a substantive basis on which to judge projects. 

ONLINE SURVEY RESULTS

The realization of the inadequacy of the internal records and files was a key concern for 

ARC and a fundamental rationale for conducting this research project. To supplement 

the file data, RTS used a variety of methods for collecting information on impacts as 

described in Section 1 of this report. Most of the data are qualitative in nature but the 

two online surveys, completed by grantee project managers (or sometimes an individual 

in the grantee organization more knowledgeable about the project) do provide valuable 

quantitative data. The survey instruments are shown in Appendix B and summaries of 

results are shown in Appendix C. As noted above the initial survey focused on 

categorical qualitative answers while the second also delved into quantitative estimates of 

specific outputs and outcomes from projects. 

To assess the impacts of tourism and asset-building projects, RTS developed and 

distributed two surveys to project managers from the universe of selected projects. The 

purpose of the surveys was to measure the impact of the projects across a wide range of 

categories, using both qualitative and quantitative measures. 

Universe of selected projects: In consultation with ARC staff, RTS created a suitable 

database of projects falling in the category of tourism and asset-building projects, with 

closing dates from the period 2000 to the present. To ensure that the survey gained the 

most valuable information possible, the following two types of projects were eliminated 

from the survey: 

• Projects whose impacts had been evaluated in prior ARC studies. Primarily, 

these projects were ones addressed in a previous evaluation of entrepreneurial 

related programs. ARC staff designated those that should not be included in the 

final survey. 

• Projects whose sole purpose was the creation of a strategic plan. While planning 

for such activities as building a conference center or developing a marketing plan 

are inherently valuable, the impacts generally come directly from implementation 

rather than the creation of a plan. It was felt that inclusion of these projects would 

skew the impacts unnecessarily.

Process of collection: The surveys were created using the online survey tool of 

surveymonkey.com. An introductory email was sent to all listed project managers on file 

with the ARC. If emails showed a failure to delivery, contact was made with the 
Program Evaluation of ARC’s Tourism, Page 8 

Cultural Heritage and Natural Asset-Related Projects

organization that received the information to obtain updated contact information. If the 

project manager was no longer with the organization, then other knowledgeable 

individuals were encouraged to complete the survey. In some early project cases we 

could not identify a suitable individual to take the survey. In all cases, survey 

respondents were contacted several times both by phone and electronically to encourage 

their participation in the survey. Each project was contacted a minimum of three times, 

if contact information was available. 

 First Survey

The initial survey focused on qualitative impressions about the projects. Of the 98 

surveys sent (through electronic means), 51 were completed, for a response rate of 52 

percent. To insure confidentiality the surveys did not ask for identifying information 

such as name of the respondent or the project name or number. 

Purpose and structure of the first survey: The survey served three main purposes. 

1) To determine if project managers believed that the projects had been successful and 

met their goals: Respondents were asked about overall success in meeting goals and in 

what ways the project had positively impacted the communities it served. Specific 

questions were asked about whether the project resulted in the development, expansion, 

or enhancement of infrastructure and programs and the ways the projects had affected the 

economic success of the communities. Respondents used a scale ranging from “not at 

all” to “a great deal” to categorize the impacts for the various impact areas. 

2) To measure impact across the Triple Bottom Line: Project managers were asked to 

estimate their project’s impact across a range of economic, social and environmental 

measures some of which specifically address TBL. Project managers used a scale 

ranging from “not at all” to “a great deal” to categorize the impacts for the various 

categories. 

3) To discuss evaluation measures: Respondents were asked to discuss how they 

evaluated their projects and whether additional funding would have allowed them 

improve their evaluation. 

Respondents were mostly very satisfied with the success of their projects and their 

outcomes. On a scale of 1 to 5, with 1 being “Not at all” and 5 being “Completely,” 86% 

considered that the project mostly (4) or completely (5) achieved its goals (Figure 2). A 

strong majority (86.5%) reported that new initiatives, whether a program or organization, 

information delivery or facility, had mostly or completely been created as a result of the 

project. Most projects were reported to still be in place and in use (Figure 3). 

When asked about the impacts of the projects, most respondents reported moderate to 

strong impacts on tourism-related items: new tourism, cultural and visitor facilities (Table 

1 and Figure 4). This is unsurprising given the goals of the projects. Recreation 

facilities, small business and entrepreneurial support, education and training and 

recreation development were also significant. 
Program Evaluation of ARC’s Tourism, Page 9 

Cultural Heritage and Natural Asset-Related Projects

Lesser impacts were reported from the standpoint of the Triple Bottom Line perspective. 

Few projects generated facilities or enhancements in conservation, environmental 

infrastructure, open space, and water or energy conservation (Table 2 and Figure 5). 

Within the social realm, the projects have minimal impacts on poverty or health. These 

results are understandable and explained by two factors: project design did not include or 

consider environmental or social features; and ARC has other funding programs 

specifically targeted to infrastructure and other environmental issues.

Open-ended questions on project impact measurement issues revealed that many 

recipients relied on very informal or anecdotal measures of impacts. Lack of resources to 

track impacts was commonly mentioned – surveys, focus groups and mechanisms were 

suggested if resources were available. Some respondents expressed regret that evaluation 

had not been planned earlier in the process. Other respondents noted that it is too early to 

see the ultimate impacts. 
Program Evaluation of ARC’s Tourism, Page 10 

Cultural Heritage and Natural Asset-Related Projects

Figure 2. To what extent did the project achieve its goals?

A l ttle

4% Some

10%

Mostly

43%

Completely

43%

Figure 3. To What Extent are the Initiatives in Use?

Not at all used

9%

Minor Use

12%

Some Use

12%

Strong Use

22%

Completely in Use

45%
Program Evaluation of ARC’s Tourism, Page 11 

Cultural Heritage and Natural Asset-Related Projects

Table 1. To what extent did the project result in the development, expansion, or 

enhancement of the following? 

Project Impacts

Moderate to Great 

Impact

No or Small 

Impact

Don't

Know

Tourism attractions 86% 12% 2%

Cultural facilities and events 69% 29% 2%

Support for small businesses 67% 27% 6%

Education and training programs 65% 33% 2%

Visitor facilities and services 63% 37% 0%

Community facilities and services 53% 45% 2%

Recreational facilities 43% 55% 2%

Local conservation activities 29% 61% 10%

Amount of public open space 22% 71% 6%

Support for local agriculture 20% 69% 10%

Infrastructure (e.g., water & sewer) 16% 78% 6%

Environmentally-friendly transport 10% 76% 14%

Table 2. What impact did the project have on the following? 

Impact Category

Positive 

Impact

Slight 

Impact

No 

Impact

Don't 

Know

Preservation of cultural heritage 64% 21% 2% 13%

Tourism revenues 57% 23% 9% 11%

Employment opportunities 47% 34% 13% 6%

Visual landscape 47% 11% 30% 13%

Sales of locally-produced items 43% 30% 17% 11%

Quality of jobs 40% 32% 21% 6%

Household incomes 19% 34% 34% 13%

Employment of high-skilled labor 15% 26% 49% 11%

Population retention or growth 11% 30% 40% 19%

Poverty reduction 9% 28% 47% 17%

Water conservation 6% 2% 70% 21%

Air and water pollution reduction 6% 4% 66% 23%

Resident health 4% 6% 62% 28%

Energy conservation 4% 15% 57% 23%
Program Evaluation of ARC’s Tourism, Page 12 

Cultural Heritage and Natural Asset-Related Projects

Figure 4. To what extent did the project result in the 

development, expansion, or enhancement of the following? 

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Tourism attractions

Cultural facilities and events

Support for small businesses

Education and training programs

Visitor facilities and services

Community facilities and services

Recreational facilities

Local conservation activities

Amount of public open space

Support for local agriculture

Infrastructure (e.g., water & sewer)

Environmentally-friendly transport

Moderate to Great Impact No or Small Impact Don't Know

Figure 5. What impact did the project have on the following?

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Preservation of cultural heritage

Tourism revenues

Employment opportunities

Visual landscape

Sales of locally-produced items

Quality of jobs

Household incomes

Employment of high-skilled labor

Population retention or growth

Poverty reduction

Water conservation

Air and water pollution reduction

Resident health

Energy conservation

Positive Impact Slight Impact No Impact Don't Know 
Program Evaluation of ARC’s Tourism, Page 13 

Cultural Heritage and Natural Asset-Related Projects

 Second Survey

For the second survey RTS created a database of 114 specific projects, in consultation 

with ARC staff, falling in the category of tourism and asset-building projects, with 

closing dates from the period 2000 to the present. In this survey we also asked for the 

respondent name and title and number of the project though the respondents were 

guaranteed confidentiality. Having the project name and number allowed us to compare 

the numerical impact estimates reported to the data in the ARC-provided database on a 

project-by-project basis.

The final participation rate was 60 percent, a high figure for surveys of this nature. Many 

of the projects were almost ten years old by the time of the survey, which undoubtedly 

moved the response rate down. Failure to respond was almost exclusively because of 

departure or death of the original project manager. 

Universe of respondents: The 69 respondents to the survey represent a true crosssection of the types of projects funded under this ARC program. Some important 

features of the respondents include: 

• 82% of respondents were from projects closed out by ARC 

• Projects covered a full range of activities including development of marketing 

plans, expansion of tourist facilities, offering of educational services, and 

development of visitation trails.

• Respondents included projects in every ARC state with the exception of South 

Carolina.

• ARC investment varied from projects of $10,000 to $500,000, with the latter 

focused on construction and renovation projects.

Purpose and structure of the survey: The survey served three main purposes.

1) To measure specific results – outputs and outcomes of the project: Outputs focused on 

the numbers of individuals, businesses and communities served through the project, the 

amount of additional funds leveraged by the project, any materials developed through the 

project, and any programs and plans developed. Outcomes focused more on quantifiable 

measures of project success, e.g. jobs created and retained, businesses improved, 

communities improved, etc. In all cases, project managers were asked to assign a number 

value to the project’s impact.

2) To measure impact across the Triple Bottom Line: In addition to examining 

quantifiable outputs and outcomes, project managers were asked to estimate their 

project’s impact across a range of measures. Project managers used a scale ranging from 

a very negative impact to a very positive impact. Four broad categories were used for 

this section of the survey. 

• Economic measures: Participants were asked to gauge the general impact on the 

following categories: population; employment; personal income; household 

assets/wealth; business assets/revenues; and, public assets/revenues.
Program Evaluation of ARC’s Tourism, Page 14 

Cultural Heritage and Natural Asset-Related Projects

• Efficiency measures: Participants responded to a series of questions relating to 

the project’s impact on regional competitiveness. Specifically, participants were 

asked to gauge impact on: job stability; efficiency; productivity of land, labor, 

energy or capital; and, access to markets. While efficiency is not always included 

as a discrete component of the triple bottom line, creating a separate category 

allowed respondents to focus more directly on these measures rather than 

embedding them within other categories.

• Social measures: Participants were asked questions relating to the project’s 

impact across the following measures relating to the social component of the 

triple bottom line: civic life and governance; health, education, public safety and 

access; culture, arts, other amenities, and public assets.

• Environmental measures: All participants were asked to measure the qualitative 

impact of their projects on their community’s environment in the following three 

categories: air and water quality; land and natural resources; and recreational 

opportunities. 

3) To measure visitation patterns: ARC staff requested that we include questions relating 

to the purpose of visitation of tourists as well as issues relating to accommodation, mode 

of transport and length of stay. However, since very few of respondents kept track of this 

information, the vast majority of respondents chose not to respond to this portion of the 

survey and since it was not in the original scope of the project we do not provide a 

significant discussion of these questions in the report. 

Quantifiable measurements 

Much of the impetus behind ARC’s request for a second survey was to look at the 

specific measurable impacts – jobs created and retained, leveraged private investment, 

businesses created, etc. – that are used to document program effectiveness. The questions 

focused on outputs and outcomes. A brief description of the definitions will be useful. 

According to ARC provided materials (DataDictionary.doc dated October 16, 2007): 

• Output usually refers to activities, as opposed to actual outcomes such as 

businesses served as opposed to business created or business sales increased. 

• Outcome refers to tangible results such as jobs created or retained, businesses 

created or leveraged investment.

The ARC provided database provides projected and, for closed projects, actual impacts 

and activities for outputs and outcomes. 

The actual definitions are also important. ARC provided guidance on definitions is as 

follows, edited to remove definitions not relevant to this effort (Key definitions11-

06.doc): 
Program Evaluation of ARC’s Tourism, Page 15 

Cultural Heritage and Natural Asset-Related Projects

“There are just a few key performance measurements which are the main 

outcome measures for ARC programs and for reporting to OMB (Office of 

Management and Budget): jobs created, jobs retained, leveraged private 

investment, households served, students with improvements, and workers 

trained.

Jobs Created. Generally, there are two types of jobs that are reported as 

outcomes for ARC funded projects: non-construction project-related jobs, 

and private sector jobs that will result once the ARC funded services or 

project is completed. 

First, a grantee should determine if there are any direct hires that have 

been made as a result of the project’s operation (e.g. teachers, public 

safety, information services, etc.) Often these direct hires are the result of 

public sector spending or funding by private foundations. 

Second, a grantee should determine how many private sector jobs will be 

created in three years once the ARC-funded services are delivered or the 

project is completed. These job projections are usually related to 

additional investments in manufacturing plant and equipment, retail, 

commercial real estate development, or other operations that will lead to 

new hires. 

Jobs Retained. The definition of jobs retained is usually measured by the 

number of workers actually enrolled in a specific training program, or by 

the number of jobs at a business that will be retained because of an 

investment that is needed to keep the business and jobs in continued 

operations in the area. 

Leveraged Private Investment represents private sector financial 

commitments that are not part of the project funding but follow as a result 

of the completion of the ARC-supported project (such as an infrastructure 

project) or the delivery of services (e.g. worker training, marketing 

campaign, export promotion program). Leveraged private investment is a 

performance measurement since it is a desired outcome, and leveraged 

private investment is usually the principal reason that any project can 

report “jobs created.” 

Table 3 presents the survey results and, where available, comparable data from the ARC 

project database for outputs. [Note that for the following tables the database numbers 

reflect only the 69 projects associated with the survey.] The “Reported on Survey” 

column refers to actual estimated impacts within the categories provided by respondents 

for their projects. The “Projected Outputs” column refers to the projected outputs for the 

same projects from the ARC database. We used projected instead of actual data reported 

for two reasons. First the actual number is provided at project closeout and therefore 
Program Evaluation of ARC’s Tourism, Page 16 

Cultural Heritage and Natural Asset-Related Projects

Table 3. What specific results (outputs) were actually achieved by this project?

Outputs Category

Reported 

on Survey

Projected 

Outputs

Ratio 

Reported to 

Projected

Participants Served 1,322,520 89,591 14.8 

Businesses Served 2,790 2,664 1.0 

Nonprofit Entities Served 892 NA NA 

Public Agencies Served 141 NA NA 

Communities Served 880 NA NA 

Visitor Attractions Developed 346 NA NA 

Programs and Plans Developed 474 31* 15.3 

Meetings and Events Held 1,236 NA NA 

Promotional Materials Developed 1,405,918 NA NA 

 * - Total of "New Programs Developed” and "New Strategic Planned Developed"

 NA -- Information not available in ARC project database

does not reflect the “three years once the ARC-funded services are delivered or the 

project is completed.” 

Second, it is important to compare the actual numbers to the projections to start to 

understand whether projects are successful and whether project developers have the 

expertise and ability to estimate impacts and track actual results. The results from the 

first survey, our interviews (discussed below) and the case studies suggest that project 

managers are concerned about their ability to project and track project impacts. 

As reflected in Table 4 the 69 projects reflected in the survey demonstrate substantial 

impacts within the regions they serve. The survey reflects that the projects served many 

more participants and generated substantially more programs and plans than projected. 

While this reflects respondents’ experience it is likely that the definitions that were used 

in the original proposals were not the same ones that the respondents used for the survey. 

This is an important result as it provides more evidence that grantees do not have robust 

or consistent understanding of the measures that ARC focuses on. Unless everyone is 

“on the same page” it is unlikely that reported results can be meaningfully compared to 

the original projections.

The “Ratio Reported to Projected” can provide an effective and succinct measure of 

project effectiveness if the definitions used are consistent. In the three instances where 

survey output answers can be compared to the ARC database, two likely reflect 

definitional inconsistency as it is unreasonable to expect that the projects will serve, for 

example, 15 times more participants than projected. For “Businesses Served” the ratio 

suggests that the project leaders were able to make reasonable estimates of this measure 

and ultimately deliver outputs.
Program Evaluation of ARC’s Tourism, Page 17 

Cultural Heritage and Natural Asset-Related Projects

Table 4. Who actually benefited from this project? What results were actually 

achieved?

Outcomes Category

Reported on 

Survey

Projected 

Outcomes

Ratio 

Reported to 

Projected

Participants Improved 363,594 NA NA 

Businesses Improved 872 NA NA 

Nonprofit Services Improved 190 NA NA 

Public Services Improved 111 NA NA 

Communities Improved 502 NA NA 

Leveraged Private Investment $19,343,116 $47,511,751 0.4* 

Jobs Created 1,257 1,783 0.7

Jobs Retained 512 306 1.7 

Businesses Created 110 39 2.8 

Businesses Retained 106 NA NA 

Business Sales Increased $7,962,073 NA NA 

 NA -- Information not available in ARC project database

 * Removing a questionable $30 million leveraged investment projection in a 

single project results in a ratio of 1.1.

The survey finds that the projects had substantial positive impacts (outcomes) within the 

communities they served. For example, from Table 3 we see that 2,790 businesses were 

served (a specific output) in some fashion and, from Table 4, that output was a factor in 

improving 872 businesses (a specific outcome). The survey results for jobs created and 

retained and businesses created indicate that while these measures are fairly obvious and 

well understood the ability to project the impacts is not a trivial effort. 

As noted above in the discussion of the first survey and the interview results that follow 

this discussion (and consistent with the case studies in Appendix E), project managers 

indicated that estimating these types of impacts was difficult. Many used anecdotal 

methods and did not have an evaluation plan in place when the project began – a key 

requirement for an effective evaluation. They also indicated that they did not have 

adequate resources to conduct a systematic evaluation program.

The results for Leverage Private Investment numbers in Table 4 present another issue. 

The difference between the survey estimate of $19.3 million and the projection of $47.5 

million is due to a single project that projected $30 million in its proposal but reported no 

investment in their project closeout report and in the survey. The project was a small 

$50,000 tourism promotion project (we do not provide an additional description of the 

project to maintain the promised confidentiality of the respondent). In discussions with 

the grantee and reflected in the project closure report, the $30 million in leveraged private 

investment that was projected in the proposal to ARC was based on the overall tourism 

industry in the region. It is extremely unlikely that such a small investment could lead to 
Program Evaluation of ARC’s Tourism, Page 18 

Cultural Heritage and Natural Asset-Related Projects

an impact of that size. This suggests that ARC staff may need to have a closer 

relationship with grantees to assist them in developing more reasonable estimates for 

project impacts. 

Qualitative measurements 

In addition to asking grant recipients about quantifiable outcomes and outputs, the survey 

inquired about impacts that are best expressed qualitatively. Specifically, the survey 

asked each grant recipient to rate their projects impact on their communities’ economic 

health, on issues of economic competitiveness, on social issues and on the environment. 

For each category, project managers were asked to state their project’s impact on a scale 

ranging from very negative to very positive. Significantly, on no category did recipients 

rate their impact as either negative or very negative. In this section of the report, we go 

into detail on each of the broad categories. 

Survey respondents indicated, as seen in Table 5 the greatest economic impact coming in 

three main outputs: Business assets/revenues, public assets/revenues and employment, 

with all three of these indicators having 67 or 68 percent of respondents showing a 

positive or very positive impact. This is not particularly surprising since for the vast 

majority of the projects, these categories were the ones assigned specific numeric targets 

in the application process. Thus, the projects had these impacts specifically in mind. 

There do appear to have been positive impacts on economic measures not specifically 

named in the grant application process. For instance, 50% of respondents stated that their 

project had a positive or very positive impact on personal income and 44% stated that the 

project had positively impacted the community’s population. Less clear were the projects 

impact on household assets and wealth. 70 percent of respondents characterized that the 

impact on household wealth as only neutral. This should not be seen as a condemnation 

of those projects, which only showed neutral impact on this indicator, but rather a 

reflection of the fact that these projects did not pursue that goal in any explicit way.

Respondents were also asked to estimate the impact of their project on various indicators 

of a region’s competitiveness (Table 6). The indicators included job stability; efficiency; 

productivity of land, labor, energy or capital; and access to markets. In this set of 

indicators, the responses tended to be more ambiguous, with more than half stating that 

their project had a neutral impact in each of the four indicators. Access to markets had 

the most positive response, with 21% stating that their project had a very positive impact 

on the region’s access to markets. For tourism projects access to markets likely means 

the ability to attract visitors from regions and communities who may not have known 

about a particular destination. Thus projects that focus on increasing awareness through 

the creation of trails or expansion of advertising through brochures or on-line materials 

tended to report more impacts in this indicator. 
Program Evaluation of ARC’s Tourism, Page 19 

Cultural Heritage and Natural Asset-Related Projects

Table 5: On a scale from very negative to very positive, how would you rate the 

impact of your project on the following economic measures?

Answer Options Very 

Negative Negative Neutral Positive Very Positive 

Population 0.0% 0.0% 56.1% 30.3% 13.6%

Employment 0.0% 0.0% 31.8% 57.6% 10.6%

Personal income 0.0% 0.0% 50.0% 40.9% 9.1%

Household assets/ 

wealth 0.0% 0.0% 69.7% 24.2% 6.1%

Business assets/ 

revenues

0.0% 0.0% 31.8% 56.1% 12.1%

Public assets/ revenues 0.0% 0.0% 33.3% 50.0% 16.7%

Table 6: On a scale from very negative to very positive, how would you rate the 

impact of your project on the following economic measures?

Answer Options Very 

negative Negative Neutral Positive Very Positive 

Job Stability 0.0% 0.0% 50.0% 43.9% 6.1%

Efficiency 0.0% 0.0% 51.5% 34.8% 13.6%

Productivity of land, labor, energy 

or capital 0.0% 0.0% 57.6% 31.8% 10.6%

Access to markets 0.0% 0.0% 50.0% 28.8% 21.2%

The social measures most positively impacted by the ARC-funded projects tended to be 

ones in which the increase was an explicit goal (Table 7). Thus, the fact that 83% of 

respondents indicated that there was a positive impact on culture, arts and other amenities 

is likely due to the fact that so many projects focused on expanding those very offerings. 

Similarly, many of the funded projects focused on expanding educational offerings 

through enhanced training or programs, making the 72% who pointed to a positive or 

very positive impact on education unsurprising. Of course, just because these programs 

had the goals of expanding cultural offerings or increasing educational opportunities is 

not guarantee that they would actually achieve success across these measures. The 

generally positive reviews given by project managers should be seen in a favorable light. 
Program Evaluation of ARC’s Tourism, Page 20 

Cultural Heritage and Natural Asset-Related Projects

Table 7: On a scale from very negative to very positive, how would you rate the 

impact of your project on the following social measures?

Answer Options Very 

negative Negative Neutral Positive Very Positive 

Civic life and governance 0.0% 0.0% 43.9% 36.4% 19.7%

Health 0.0% 0.0% 65.2% 25.8% 9.1%

Education 0.0% 0.0% 28.8% 43.9% 27.3%

Public safety and access 0.0% 0.0% 71.2% 19.7% 9.1%

Culture, arts, other amenities 0.0% 0.0% 16.7% 36.4% 47.0%

Public services 0.0% 0.0% 57.6% 30.3% 12.1%

Other community assets 0.0% 0.0% 19.7% 48.5% 31.8%

The respondents were asked to gauge their impact on a set of environmental indicators 

(Table 8). In terms of air and water quality, the fact that none of the respondents believed 

that their project had a negative impact on the environment is impressive. Since hardly 

any of the projects had explicit environmental focuses, it is not surprising that the vast 

majority of respondents thought their project had at best neutral environmental impacts. 

The indicator in which most respondents assigned a positive or very positive impact was 

on expansion of recreational opportunities. Again this should not be thought of as 

particularly surprising since many of the projects had increasing access to parks or 

facilities as an expressed goal at the time of the grant application. From that standpoint, 

in the eyes of the project manager, it appears that a large percentage—68%— have 

reached their goal.

Table 8: On a scale from very negative to very positive, how would you rate the 

impact of your project on the following environmental measures? 

Answer Options Very 

negative Negative Neutral Positive Very Positive 

Civic life and governance 0.0% 0.0% 43.9% 36.4% 19.7%

Health 0.0% 0.0% 65.2% 25.8% 9.1%

Education 0.0% 0.0% 28.8% 43.9% 27.3%

Public safety and access 0.0% 0.0% 71.2% 19.7% 9.1%

Culture, arts, other amenities 0.0% 0.0% 16.7% 36.4% 47.0%

Public services 0.0% 0.0% 57.6% 30.3% 12.1%

Other community assets 0.0% 0.0% 19.7% 48.5% 31.8%

INTERVIEW RESULTS

As noted in Section 1, the RTS team conducted interviews for 32 projects including 32 

project managers or directors and 61 stakeholders who saw impacts from their 

perspective. The complete interview database is provided as a separate document.
Program Evaluation of ARC’s Tourism, Page 21 

Cultural Heritage and Natural Asset-Related Projects

The general attitude of all participants was positive and optimistic about their projects, 

despite having to deal with numerous challenges. Overall, interviews reveal that: 

The projects would not have been viable without ARC funding and for most the ARC 

grant started the project;

Nearly all project managers reported overall positive relationships with ARC and its 

staff;

Project managers faced numerous challenges concerning funding, convincing 

officials to support their efforts, and managing project delays; 

Generally projects have positive intermediate outcomes, but “open” projects have yet 

to impact regional economic development; 

A significant number reported that the “jobs created” measure was insufficient to 

measure the true impact of the project;

Employment goals are considered long-term goals, while gains in social capital were 

thought of as short-term successes; 

Some projects acted as a catalyst for environmentally friendly development; 

Many interviewees reported an improvement in community collaboration;

Almost all were successful at achieving goals but some did not do it in a timely 

manner (they needed extensions for financial reasons); 

Almost half of stakeholders were at the table when the goals were set;

Most cited that the most difficult challenge or impediment to success had to do with 

funding and; 

Virtually all thought that the project had been successful to date. 

Views on the ARC program and staff 

It is clear that grantees have a high regard for the ARC program and how they have 

interacted with staff. Most project managers remarked that ARC funds were “crucial” or 

“critical” to the success of their project, particularly in getting projects off the ground. 

Because of the lack of tangible deliverables, procuring money for the design or planning 

component of a project can be difficult. As such, many of the ARC funds were used for 

this purpose. 

Project managers generally reported positive working relationships with ARC. A few 

project managers felt that the reporting requirements and application process were too 

cumbersome for the relatively small amount of funding awarded. The one commonly 

cited critique was that more guidance on report format would have been helpful. 

Most project managers remarked that ARC was flexible to work with. Comments to this 

point included, “They did all that they could within reason,” “I never felt like there was a 

stupid question,” and “My contact was always in touch and I never got buried in 

paperwork.” 

Evaluation and Impact Measurement

Respondents had thought about impact measurement issues and had clear opinions. Most 

interviewees commented that although “jobs created” is a traditional metric for economic 

development, it insufficiently captured the outcomes of these projects. As one project 
Program Evaluation of ARC’s Tourism, Page 22 

Cultural Heritage and Natural Asset-Related Projects

manager alluded, although tourism may not create additional jobs, it can support and 

bolster many of the existing ones. Measuring this is difficult. Another suggested that 

measuring capacity would be a more appropriate metric. Interviewees almost 

unanimously agreed that employing long-term metrics would also help demonstrate 

impact. 

Common metrics included completion of physical or infrastructural improvements, 

number of new businesses created, number of visitors/attendance and number of jobs 

created. Some of the asset mapping projects also included a goal number of assets to be 

included on an inventory. While all of these metrics were deemed appropriate by project 

managers, many remarked that like jobs created, these metrics failed to capture some of 

the other significant outcomes, like increased social capital or environmental 

conservation. As one stakeholder remarked, it often seemed that the formula being used 

was designed for a completely different kind of project. 

Challenges

Despite the diversity of projects reviewed, four common challenges persisted. The first 

were political challenges. A number of interviewees remarked that convincing local 

officials of the promise of their projects was a significant impediment to success. 

A second challenge came from confronting negative stakeholder and community 

perceptions. Many of the project managers, particularly those with less tangible final 

products suggested that community members were suspicious of tourism as an economic 

development tool. In areas that had previously relied upon manufacturing, tourism 

seemed too “soft” of a solution. 

A third challenge came from getting the word out, both to enough people and to the right 

people. In many communities, project managers described the communities as splintered. 

Finding the right avenue to contact people was difficult, particularly for project managers 

who were not part of the immediate community. Furthermore, some communities lacked 

central organizations, so project managers were faced with the challenge of identifying 

possible participants before specific work on the project could begin. Although the 

school system would prove to be a good contact point in one of the asset mapping 

projects, for example, the project manager and stakeholders first had to resolve the 

residual distrust many members of the community had from negative experiences with 

school. 

Finally, there was extensive resistance to collaboration, particularly in some of the more 

distressed communities. Business owners did not want to cooperate for fear that their 

competition would get ahead. In other communities, historical conflicts were 

unfortunately rekindled. 

Triple Bottom Line

Project managers overwhelmingly noted significant social improvements. In the 

communities that were splintered, people began to come together. In communities where 

loss of industry had resulted in a communal depression, people began to take pride in 
Program Evaluation of ARC’s Tourism, Page 23 

Cultural Heritage and Natural Asset-Related Projects

assets previously taken for granted. In project after project, project managers remarked 

that by building an economic development plan around something unique to a 

community, you build something that cannot be outsourced. Other social impacts noted 

included an increasing tendency towards regionalism. In communities that had 

previously been myopic, many of the projects helped communities to look beyond 

political and geographic borders. 

For the projects that required alterations of physical infrastructure, environmental impacts 

were highly salient, going beyond the required environmental impact statements. Even in 

areas reported to be conservative about environmental protection, an understanding 

existed that the natural assets upon which these projects were based needed a level of 

protection so as to support sustainability. Of the projects with physical impacts, project 

managers commonly remarked that considering environmental impact was unavoidable 

and that these issues were increasingly persistent in all their projects.

Economic impacts were readily mentioned. Although not all projects reported jobs 

created, each project listed an expected financial impact in the initial application. During 

interviews, many project managers stated that although they could not give exact revenue 

impacts, all offered anecdotes of economic impacts, from new businesses opening, 

expansion of existing businesses and increases in visitation. One project manager 

remarked that the largest economic impact was that people were now in his area not by 

accident, but on purpose. Although the project was too new to report economic impacts, 

it was his position that this change in visitorship points to a positive economic impact.

CASE STUDY RESULTS 

Case studies were conducted at project sites in the Northern, Central and Southern 

regions of ARC. The complete case studies are found in Appendix E. We attempted to 

cover a range of project types including focus, project size, types of projected outcomes. 

We offer two shortened case studies that document some of the crosscutting 

characteristics we found. One describes an effort to assist in the renovation of a resort 

and conference center and the second profiles a project to build the agri-business and 

agri-tourism of a regional economy. 

ATWOOD LAKE RESORT RENOVATIONS 

PROJECT DESCRIPTION 

ARC awarded a grant of $200,000 to the Muskingum Watershed Conservancy District in 

2003 to pay for a portion of the renovation of the Atwood Lake Resort and Conference 

Center in Carroll County, Ohio. The resort, Carroll County’s largest private employer 

and only major lodging facility, has 123 lodging units, meeting space for up to 350, an 

18-hole golf course, and other recreational facilities. The resort current employs 50 yearround workers and 100 seasonal workers. 
Program Evaluation of ARC’s Tourism, Page 24 

Cultural Heritage and Natural Asset-Related Projects

The ARC funding supported three components of the project: 1) bringing the facility into 

compliance with the Americans with Disabilities Act (ADA), primarily by installing an 

elevator to provide access to guest rooms and meeting space; 2) installing new windows 

to make the facility more energy efficient; and 3) improving the facility’s public spaces. 

The total cost of these three components was $2.5 million. 

The projected outcome of the project was the retention of the resort’s 50 full-time jobs 

and the creation of 10 additional jobs over a five-year period. 

PROJECT OUTCOMES 

The resort is clearly a critical source of support for the tourism economy in Carroll 

County and the surrounding region. It is the only full-service resort in the county. In 

fact, the county has only six other lodging facilities other than campgrounds—five bed 

and breakfasts and a recently constructed 43-room limited service motel with no 

recreational amenities. Surrounding counties have additional lodging facilities, but few 

full-service resorts. Tourism is a major component of the region’s economic base and 

second only to agriculture in Carroll County according to local tourism and economic 

development officials. 

In addition to attracting visitors to the region, the resort is a primary contributor to the 

region’s tourism marketing infrastructure. According to the director of the Carroll 

County Convention and Visitors Bureau (CVB), the resort generates approximately 75 

percent of the county’s room tax revenue. Half of this, approximately $60,000, is 

allocated to the Convention and Visitors Bureau for tourism marketing. 

Tourism officials in surrounding counties recognize the importance of the resort as a 

regional tourism attraction because of the limited number of resort properties with 

comparable features in the region. In this context, more guests at the resort translate into 

more visitors to surrounding counties. This is why local tourism organizations such as 

the convention and visitors bureau in adjoining Tuscarawas County include the resort in 

their marketing packages. 

MEASUREMENT ISSUES 

It can reasonably be argued that the project led to the retention of all of the jobs at the 

resort, since the resort’s survival appeared problematic had the renovations not been 

undertaken. The role of ARC funding is less clear, however. Given ARC’s small 

financial role in the project, it is likely that the renovations would have proceeded 

without ARC support, although possibly with some reduction in scope. 

While the ARC funding application indicated that the project would result in the creation 

of ten new jobs, direct employment at the resort did not increase. No effort has been 

made to estimate indirect job creation or retention from the project. This could be done 

by calculating the number of visitor-days spent at the resort, applying available data on 
Program Evaluation of ARC’s Tourism, Page 25 

Cultural Heritage and Natural Asset-Related Projects

non-lodging visitor spending, and using an economic impact model to estimate total 

indirect spending and employment. Expenditures resulting from an increase in visitordays after the renovations would translate to new job creation. The state of Ohio has 

developed a model to estimate direct and indirect spending and employment resulting 

from tourism at the county level, and this model could likely be adapted to estimate the 

project’s indirect impacts. 

BROOME COUNTY AGRI-TOURISM AND MARKETING 

PROJECT 

PROJECT DESCRIPTION 

ARC awarded a grant of $24,206 to the Cornell Cooperative Extension in Broome 

County, New York, in 2003 for an initiative to strengthen the county’s agriculture 

industry through diversification and the development of agri-tourism. The total cost of 

the project was $96,697. The $72,491 local match included $68,991of in-kind support 

from the grantee and $3,500 from the Broome County Chamber of Commerce. 

The initiative had several components: 

Providing training and technical assistance to farmers in agri-tourism, marketing, 

product diversification, and business planning and promotion. 

Forming three producer purchasing and/or marketing groups. 

Developing agri-tourism and direct-to-consumer marketing tools. This would 

include creating and distributing an agri-tourism marketing brochure, promoting 

tour packages to motor coach tour owners, creating a countywide logo for 

agricultural products, and installing agri-tourism signage. 

Creating and distributing a five-year comprehensive agri-tourism development 

plan. 

The initiative was expected to result in an increase of 50 percent in direct sales from 

farmers to consumers, the expansion of 50 percent of existing agri-businesses to include 

agri-tourism and/or an alternative enterprise, the creation of 25 new agri-businesses, and 

the creation of 20 new jobs. 

The Extension’s executive director and agricultural economic development specialist 

believe that the ARC funding was essential to completing the project work plan. It 

enabled the Extension to hire a part-time program assistant to provide additional staffing 

for the project. The agricultural economic development specialist, who directed the 

project, would not have been able to undertake many of the tasks included in the work 

plan given her existing workload. They also believe that the ARC-funded project laid the 

groundwork for subsequent initiatives by demonstrating the potential for agri-tourism and 

direct-to-consumer marketing, and by enabling them to build relationships with business 

organizations and political leaders and leverage additional resources. 
Program Evaluation of ARC’s Tourism, Page 26 

Cultural Heritage and Natural Asset-Related Projects

PROJECT OUTCOMES 

Agri-tourism and agri-marketing efforts in Broome County have continued and have 

gained momentum since completion of the project. As local political officials and 

economic development professionals have become more aware of the economic benefits 

of agri-tourism and local purchasing of agricultural products, new efforts have emerged 

to more fully capture these benefits. Examples include a county-commissioned 

feasibility study for an all-weather farmers’ market and a proposal to develop a shareduse commercial kitchen. 

The agricultural economy has tended to function in a separate sphere from other 

segments of the local economy with its own set of relationships and organizational 

structures. The project and ensuing activities have brought the agricultural community 

together with other local economic stakeholders in collaborative efforts that have 

identified common interests and built relationships. The project has also built 

relationships, organizational structures, and leadership among non-traditional agricultural 

producers (e.g., through the marketing and production groups). This has increased their 

ability to work together to address common challenges and opportunities. 

Looking to the future, there is a general consensus among agricultural, economic 

development, tourism, and political representatives that agri-tourism and local direct-toconsumer sales have strong growth potential and will contribute more to the local 

economy in years to come, particularly if the efforts that have been initiated in recent 

years are maintained and expanded. 

MEASUREMENT ISSUES 

There is considerable evidence that the project and ensuing activities have helped local 

agri-businesses to increase sales to visitors and local consumers. In a random sampling 

of business listed in the agri-tourism guide half of the businesses attributed an increase in 

sales to the guide. The growth in patronage of the farmers’ markets is another general 

indicator. Finally, the level of engagement and enthusiasm among agri-businesses 

themselves indicates that they see the value of these activities in their bottom lines. 

That being said, the Extension has only been able to estimate through observation the 

degree to which the economic impacts projected in the ARC funding proposal were 

achieved. It estimated 12 new agri-businesses formed as of mid-2008 compared to an 

initial goal of 25, and an increase of 15 agri-business jobs compared to an initial goal of 

20. The Extension has not been able to document increases in direct sales from farmers 

to consumers for various reasons. The Extension is exploring the development of 

practical methods to obtain sales data, but it is uncertain whether data collection barriers 

can ever be fully addressed. 

Data collection issues aside, the Extension staff have concluded that it was unrealistic to 

expect the magnitude of impacts projected in the ARC funding proposal within the 
Program Evaluation of ARC’s Tourism, Page 27 

Cultural Heritage and Natural Asset-Related Projects

timeframe of the project. While they see evidence of progress, this is a process that takes 

place over a period of several years, not one or two. Therefore, the timeframe for 

assessing the economic impacts of this type of project must extend far beyond the end of 

the ARC grant period. 

OVERALL CASE STUDY IMPRESSIONS 

Several common impressions were found across the range of projects including the 

additional case studies included in Appendix E. 

Projects, by necessity, often evolve from initial conception. Flexibility in project 

implementation based on changing circumstances has the potential to improve 

projects and their impacts. For instance, the focus of an effort to build the 

ceramics industry in Southeastern Ohio changed due to a downturn in the 

economic viability of the targeted firms.

The vectors of impact were broader than initially envisioned. The Broome 

County agri-tourism project, for example, led to increased collaboration and spinoff projects such as the proposed shared-use commercial kitchen.

Impacts are often difficult to track, estimate and justify. For instance, while 

heritage trails in Tennessee and Georgia increased visitation to targeted sites, the 

expenditures of these visitors in the region’s shops and hotels are difficult to 

track. There are a number of reasons for this including technical challenges, lack 

of grantee expertise in data collection and evaluation, and resource shortages. 

Local and regional politics and relationships can have significant positive and 

negative impacts. In some cases, such as a crafts initiative in Yancey County, 

North Carolina, potential partners did not collaborate lessening the potential 

impact of the project.

Project proponents often felt the measured project metrics did not reflect the most 

important impacts of the project. 

Impacts often accrue to non-grantees such as surrounding counties and businesses 

unrelated to the project. 

There are often un-measured spillover and synergy impacts from projects. This is 

particularly evident in the Crooked Road region of southwest Virginia where a 

series of related projects funded by ARC have helped create a creative cluster that 

builds on the relationship between projects. 

Many projects had impacts on the social and environmental metrics of TBL even 

though the projects had not been planned to affect these elements.

SUMMARY 

The RTS team used a broad range of data collection and analytical tools both quantitative 

and qualitative. These varied methods of intelligence produced consistent results indicating 

that the findings are robust and provide consistent guidance for evaluation and 

recommendation of ARC evaluation procedures for Tourism, Cultural Heritage and Natural 

Asset-Related projects. Section 3 uses the analysis within this section to develop reasonable 
Program Evaluation of ARC’s Tourism, Page 28 

Cultural Heritage and Natural Asset-Related Projects

estimates of the impacts of the ARC tourism project portfolio on the key performance 

measures used to judge project success. Section 4 then applies these findings to the question 

of further refining metrics and improving methods of measuring impacts.
Program Evaluation of ARC’s Tourism, Page 29 

Cultural Heritage and Natural Asset-Related Projects

SECTION 3: ESTIMATING IMPACTS FROM THE PORTFOLIO 

OF ARC TOURISM, CULTURAL HERITAGE AND NATURAL 

ASSET-RELATED PROJECTS 

The second on-line survey discussed in Section 2 combined with the full universe of 

projects in the ARC database provides a structured method to estimate the overall 

impacts of the tourism, cultural heritage and natural asset-related portfolio of projects 

funded by ARC. 

The survey, which focused on quantitative outcomes and outputs, enabled us to collect 

data from project managers in a way that mostly avoids the problem of the estimated 

impacts provided at project closeout. The final report submitted to ARC at the 

completion of a project is required to estimate “actual” impacts. The estimates are 

supposed to include impacts going forward three years under the correct assumption that 

impacts normally take time to occur after a project is completed. The survey strongly 

suggests that under these circumstances the managers focus more on present conditions, 

not three years out, and appear to underestimate project impacts. 

In our survey, the project managers were able to look back at their projects after more 

time had elapsed and projects had had a chance to mature and generate impacts within 

their communities. This was true except for a handful of the 69 projects surveyed, 

specifically three construction projects that have only recently been completed. When we 

compare the survey estimates which are based on a longer time horizon we find that the 

survey estimated impacts are much more consistent with the initial pre-project 

projections as compared to the immediate post closure estimates. 

Table 9 includes a column that is a ratio of the survey estimates of outcomes compared to 

the initial projected outcomes from the original proposal for jobs created and retained, 

businesses created and leveraged private investment. As an example, the survey indicates 

that the surveyed projects generated 1,257 jobs compared to the initial projection for 

those projects of 1,783 jobs, generating a ratio of 0.7. The proposals appear to have 

somewhat overestimated the ultimate job impacts. For jobs retained the survey estimate 

of 512 is higher than the original projection of 306, generating a ratio of 1.7 suggesting 

that the original projections underestimated the ultimate impacts.

Assuming that the 69 projects for which surveys were completed are representative of the 

full universe of 132 projects allows us to generate estimates of impacts for the full 

universe using the estimated ratios.

Table 9 displays the generated estimates of impacts from the ARC portfolio of tourism, 

cultural heritage, and natural-resource related projects.
Program Evaluation of ARC’s Tourism, Page 30 

Cultural Heritage and Natural Asset-Related Projects

Table 9. Estimated Impacts of the Universe of ARC Tourism Projects

Impact Category

Initial 

Projection 

of Impacts

Surveybased 

Adjustment 

Ratio

Estimated 

Post Project 

Impacts 

Estimated 

Unit Impacts 

of ARC 

Funding

Jobs Created 3,671 0.70 2,588 $4,161 

Jobs Retained 5,616 1.67 9,397 $1,146 

Leveraged Private Investment $65,575,691 0.41 $26,697,357 $0.40 

New Businesses Created 165 2.82 465 $23,139 

Businesses Served 7,148 1.05 7,486 $1,438 

The Initial Projection of Impacts column is the sum of the original projections from the 

69 project proposals submitted to ARC as reflected in the ARC provided database. The 

Estimated Post Project Impacts column adjusts the projected impacts based on the ratios 

estimated from the survey results as shown in Tables 3 and 4 of Section 2. So, for 

example, applying the jobs created 0.7 ratio we estimated in Table 4 to the original ARC 

database projection for job creation gives us an estimate of 2,588 jobs created by the 132 

projects examined in this analysis.

Probably the best way to look at these results is to look at the “bang-for-the-buck” of the 

$10.8 million of ARC funding for these projects. The Estimated Unit Impacts of ARC 

Funding column shows that a new job was created in the community for every $4,161 of 

ARC-provided dollars. The creation of a new business required $23,139 in ARC 

funding. For every $0.40 invested by ARC in the projects, $1.00 of leveraged private 

investment occurred. 
Program Evaluation of ARC’s Tourism, Page 31 

Cultural Heritage and Natural Asset-Related Projects

SECTION 4: REFINING METRICS AND METHODS FOR 

MEASURING EMPLOYMENT IMPACTS 

THE CONTEXT 

The tourism, cultural heritage and natural asset-related projects included in this 

evaluation primarily address Goal 1 of ARC’s 2005-2010 strategic plan – to “increase job 

opportunities and per capita income in Appalachia to reach parity with the nation.” ARC 

uses two measures to assess the outcomes of projects funded to achieve this goal. The 

primary measure is number of jobs created and retained. A secondary measure is 

leveraged private investment (LPI), under the assumption that private investment 

generated by ARC-funded projects will result in additional job creation and retention. 

ARC has developed very specific definitions of each of these measures, as described in 

its Fiscal Year 2006 Performance and Accountability Report: 

 “Jobs created” includes any direct hires that will be made as a result of the project’s 

operation, not including highway or building construction jobs. Also included are 

private-sector jobs that will be created within three years after ARC-funded services 

or projects are complete. These jobs are usually related to additional investments in 

manufacturing plants and equipment, and retail and commercial real estate 

development. Part-time jobs are converted to full-time equivalents and rounded up to 

whole numbers. 

“Jobs retained” refers to the number of workers actually enrolled in specific training 

programs, or to the number of jobs at businesses that will be retained because of an 

investment that is needed to keep the businesses and jobs in the area or in continued 

operation. 

“Leveraged private investment” represents private-sector, non-project financial 

commitments that follow and are the result of the completion of an ARC-supported 

project or the delivery of services under an ARC-supported project. It is generally 

estimated for the three-year period following the completion of a project and is 

separate from any direct private contribution to ARC-supported project funding. 

It is clear that job creation and retention is fundamental to ARC’s mission and that the 

commission’s congressional overseers assess its performance largely on this basis. 

Consequently, any methodology developed to measure the outcomes of ARC’s activities 

must put job creation and retention front and center. However, it has become equally 

apparent through this evaluation that efforts to determine the employment impacts of 

ARC-funded tourism, cultural heritage and natural asset-related projects present serious 

methodological challenges. In fact, as it stands now, grantee reporting provides little 

useful information about employment impacts and does not permit ARC to develop 

meaningful estimates of the overall employment impacts of this group of projects. This 

not only presents difficulties for outside evaluators conducting post-program evaluation 
Program Evaluation of ARC’s Tourism, Page 32 

Cultural Heritage and Natural Asset-Related Projects

but, more importantly, prevents ARC from developing accurate program performance 

data on an ongoing basis. 

This chapter begins by describing the problems that prevent ARC from obtaining 

accurate and complete information on the employment impacts of this group of projects. 

It concludes by presenting a new framework for data collection and analysis that seeks to 

address these problems. It should be emphasized, however, that there are no easy 

solutions and that ARC will have to weigh the relative costs and benefits of investing in 

new data collection and analysis methods. 

PROBLEMS WITH CURRENT MEAUREMENT METRICS AND 

METHODS 

ARC’s tourism, cultural heritage and natural asset-related projects typically fall into the 

following categories: 

♦ Planning projects

Tourism strategic plans 

Project feasibility studies 

Resource inventories 

Asset mapping 

♦ Implementation projects

Development of tourism or business marketing tools

Business training and technical assistance 

Development/improvement of tourism attractions 

Development/improvement of tourism infrastructure

In essence, these projects seek to achieve one or both of two direct outcomes: 

1. Increase visitation and tourism revenues, with a focus on heritage and recreational 

tourism. 

2. Increase sales of locally produced products, with a focus on crafts and agricultural 

products. 

Achieving these outcomes can, in turn, lead to job creation and retention and additional 

private investment. Measuring these employment and investment impacts requires two 

steps. First, accurate data must be obtained on the amount of tourism revenues or 

business sales generated. Second, methods must be employed whereby estimates of the 

impacts of increases in tourism revenues and business sales on employment and private 

investment can be developed. This is consequently a more complicated process than 

measuring the employment and investment impacts of ARC projects such as business 
Program Evaluation of ARC’s Tourism, Page 33 

Cultural Heritage and Natural Asset-Related Projects

incubators, industrial parks, and revolving loan funds that provide direct support for 

business startup, growth and attraction. 

Examination of data reporting methods for the projects included in this evaluation 

uncovered a number of problems that prevent ARC from obtaining accurate and complete 

data on jobs and private investment. 

1. Methods used by grantees to document employment impacts are inconsistent and 

often methodologically unsound. 

2. While it is likely that some of these projects generate additional private investment, 

there is no practical way to document these impacts for most projects. 

3. The project reporting time frame is too short to fully capture the employment impacts 

of these projects. 

4. The impacts of many ARC-funded tourism projects cannot easily be isolated from 

other factors influencing visitor spending. 

 Methods Used by Grantees to Document Employment Impacts are 

Inconsistent and Often Methodologically Unsound 

This issue primarily relates to implementation projects since planning projects do not 

typically have immediate or short-term employment impacts. (ARC could monitor the 

outcomes of planning projects to determine whether implementation resulted in job 

creation or retention, but this would require establishing a longer reporting time frame 

with the grantee, as discussed later in this chapter.) Among implementation projects, 

methodological issues differ for tourism and local sales projects.

Tourism Projects

ARC-funded tourism projects typically involve the development of new marketing tools 

such as brochures or websites, development or improvements to tourism attractions such 

as cultural and outdoor recreational facilities, or the development of tourism 

infrastructure such as way-finding signage and tourist information centers. 

Well-established methods exist to measure the economic impacts of tourism projects. 

These involve a three-step process. First, mechanisms are put in place to track the 

number of users of marketing tools or the number of visitors to tourism attractions. 

Second, surveys are conducted to ascertain the role played by the marketing or attractions 

in causing individuals to visit an area or extend their stay, and the resulting amount of 

visitor spending that occurs. Third, economic models are used to estimate the impact of 

increased visitor spending on local income and FTE employment. 

ARC tourism grantees do not, by and large, employ these methods. Some do not report 

employment impacts at all, but use proxies such as increased visitation to tourism 

attractions, startup of new tourism-related businesses, or hits on tourism web sites. Some 

measure employment impacts using other methods. These include: 
Program Evaluation of ARC’s Tourism, Page 34 

Cultural Heritage and Natural Asset-Related Projects

Counting job increases at the tourism attractions that received ARC funding; 

Counting job increases reported by new or existing local tourism-related 

businesses that may benefit from tourism-marketing or increased attendance at a 

tourism attraction; 

Calculating total changes in tourism-related employment from local or regional 

economic data.

Attribution

These methods suffer from serious limitations. None address the issue of attribution – 

the extent to which the observed increases in employment are attributable to the ARCfunded project. This can result in over-counting employment impacts, which is almost 

certainly the case when total local or regional tourism employment growth is counted. 

Conversely, the first two methods can also result in undercounting employment impacts. 

Counting job increases only at ARC-funded tourism attractions or at identifiable 

businesses that may benefit from tourism marketing or increased attendance at ARCfunded tourist attractions is unlikely to fully capture increases in tourism-related 

employment, which are typically widely diffused throughout the local economy. 

Another issue relating to attribution is determining the relative impact of ARC monies on 

a project. It is very unlikely that project managers will have the means and knowledge to 

accurately attribute impacts between funding sources. This is particularly true when 

ARC funds a specific element, say a museum exhibit, as part of a broader museum 

development project. There are basically three options for dealing with the issue of 

attribution. The first is to view ARC funding as a funding source without which the 

project would not have occurred. Under this option ARC would attribute the full level of 

impacts from the projects but use language that makes it clear what is being measured. A 

second method is to allocate benefits based on ARC funding as a percentage of total 

funding. This is likely to underestimate the value of the ARC contribution because in 

many cases the ARC funding is the basis for the project and attracts local and other 

matches that would not occur otherwise. Lastly ARC could adopt a hybrid system that 

uses the first method when ARC funding represents a large fraction of funding and uses 

an allocation method where ARC dollars are supportive but are not the basis for the 

project.

Differences in methods for counting job impacts across projects also makes it difficult for 

ARC to assess the relative impacts of different projects. Even if projects are of a similar 

type, comparing impacts would be misleading where methods for calculating 

employment varied significantly. 

Adopting established methods for economic impact analysis of tourism projects, while 

adding complexity and cost to the outcome reporting process, could at least partly address 

these problems. 
Program Evaluation of ARC’s Tourism, Page 35 

Cultural Heritage and Natural Asset-Related Projects

Local Sales Projects 

ARC-funded projects to increase sales of local products typically involve entrepreneurial 

training, technical assistance, or the development of new marketing tools such as 

marketing brochures or websites. Most local sales project grantees appear to collect 

some data on sales and/or employment impacts from the business beneficiaries of these 

projects. 

The ability to identify and obtain data from all or most of the beneficiaries of these 

projects makes it easier to estimate employment impacts than is the case with tourism 

projects. Beneficiaries can provide employment figures and are in the best position to 

assess the degree to which any employment changes can be attributed to the project. 

However, there are methodological issues with the way data is currently collected. First, 

as noted, some grantees do not collect any data. Second, data collections methods are 

inconsistent, making it impossible to aggregate or compare employment data. Third, 

asking project beneficiaries about new job creation is unlikely to fully capture 

employment impacts. Many of these business owners are small artisans or farmers. 

Sales increases resulting from the project may in many cases not result in new hiring, but 

may result in increased hours for existing employees. 

Taking these problems into account, a more appropriate method to estimate employment 

impacts of local sales projects would be to survey project beneficiaries on sales increases 

attributable to the project and then apply an economic model to estimate local FTE 

employment impacts. This would not only achieve consistency in reporting among these 

projects, but would provide comparable employment data to that generated for tourism 

projects using established methods of tourism economic impact analysis. 

 There is No Practical Way to Measure Leveraged Private Investment for 

Most Projects 

In some cases, private investment leveraged by an ARC-funded project can be identified 

and measured. For example if a greenway is developed or a museum is built or expanded 

and businesses serving visitors to these attractions open nearby, the capital investment 

made by these businesses can reasonably be attributed to the ARC project. Even in these 

cases, however, methodological issues arise. For example, if one business buys a 

property and another leases, how should their investments be compared? Should the 

lease of a property be capitalized as a proxy for capital investment? Should a working 

capital injection be considered a leveraged investment? 

A more fundamental problem is that all of the private investment leveraged by a tourism 

project cannot realistically be tracked. Tourism-related investments are widely dispersed 

throughout a local economy. They could involve renovating lodging rooms, adding 

capacity to a restaurant, starting a souvenir shop, or building a new gas station. No 

tracking mechanism could be expected to fully capture the number and scale of these 

investments. 
Program Evaluation of ARC’s Tourism, Page 36 

Cultural Heritage and Natural Asset-Related Projects

ARC essentially considers leveraged private investment as a proxy for job creation and 

retention. The assumption is that the more private businesses invest, the greater the 

employment impact. An impact assessment methodology that more fully captured the 

employment impacts of ARC-funded projects would obviate the need to measure LPI. 

If additional local sales generated by ARC-funded projects could be estimated in a 

reasonably accurate manner, applying economic impact models that use sales data to 

estimate employment impacts could provide a more complete picture of these impacts. 

These models not only estimate direct employment but also the multiplier effect of 

indirect and induced spending. Indirect impacts result from the supply of local goods and 

services to businesses experiencing increased sales as a direct result of ARC projects, 

while induced impacts result from local spending by employees who benefit either 

directly or indirectly from these sales increases. 

 The Project Reporting Time Frame is Too Short to Fully Capture 

Employment Impacts 

Most of the outcome data received by ARC from grantees is through the final report, 

which is submitted within 30 days of the end of the project. Because of the nature of 

ARC’s portfolio of tourism, cultural heritage and natural asset-related projects, few, if 

any, employment impacts have occurred by this time. At best, the reports can inform 

ARC about the extent to which activities contemplated in the proposal have been 

completed or progressed. For implementation projects, some initial data on increases in 

tourism levels or sales of local products may be available, but not enough to fully assess 

employment impacts. The nature of these projects also makes it difficult, if not 

impossible, to realistically project employment impacts three years out, as specified in 

ARC’s definition of “jobs created.” While ARC staff conducts validation visits for a 

small number of projects two or three years after project close-out, the sample is far too 

small to draw any inferences about the longer-term employment impacts of these 

projects. 

Instituting post-project reporting requirements could at least partly address this problem, 

although at additional cost to grantees. In the case of implementation projects, 

methodologies described above could be put in place to measure employment impacts 

three years out. In the case of planning projects, the process would be more complex. 

For some planning projects, such as feasibility studies, determining whether a project had 

been implemented and estimating its employment impacts could be relatively 

straightforward. For other types of projects, such as strategic plans or resource 

inventories, the challenges would be greater. Grantees would have to identify the 

implementation activities that resulted from these plans, of which there could be several. 

Moreover, the implementers may be organizations other than grantees, which would 

increase the difficulty of obtaining useful data. 

 Impacts of Many ARC-funded Tourism Projects Cannot be Easily Isolated 

from Other Factors Influencing Visitation and Visitor Spending 
Program Evaluation of ARC’s Tourism, Page 37 

Cultural Heritage and Natural Asset-Related Projects

Additional tourism spending can be more easily attributable to some ARC tourism 

projects than others. Some projects support the development or improvement or 

“destination” attractions that motivate tourists to visit an area or extend their stays. These 

include major museums, performance venues, heritage trails, and recreational trails. 

Other projects fund the development of marketing tools such as websites and brochures 

that may also motivate tourists to visit an area by increasing awareness of its attractions 

and amenities. Established impact analysis methods can estimate the impacts of these 

types of projects if the appropriate visitation and spending data are collected.

However, there are many other ARC projects that, while contributors to an area’s tourism 

development, can’t easily be linked to additional visitor spending. Some of these projects 

involve improvements to small heritage sites or cultural facilities. These projects may 

contribute to a larger tourism development strategy and may incrementally enhance the 

tourism experience, but may not in themselves be responsible for a measurable amount of 

additional visitor spending. Other projects, such as way finding signage or information 

distributed to visitors who have already arrived in an area at visitor centers or other 

locations, may result in increased visitation to specific tourism attractions or tourismrelated businesses, but again may be difficult to link with additional spending. Methods 

could be employed to track the role of these tools in directing visitors to specific sites and 

any spending that resulted, but it would be difficult to accurately assess their role in 

increasing overall visitor spending. 

RECOMMENDATIONS FOR A NEW FRAMEWORK FOR 

MEASURING EMPLOYMENT IMPACTS 

The issues raised above suggest that ARC should develop a new framework for 

measuring the employment impacts of its portfolio of tourism, cultural heritage and 

natural asset-related projects. They also suggest that a number of criteria should be 

applied in establishing this framework: 

Methods employed should be reasonably reliable and consistent. 

They should be designed to ensure that the impacts measured are attributable to 

the ARC project. 

They should be made as easy as possible to execute and proportionate to the 

grantee’s resources and the cost of the project. 

They should be tailored to the project type and stage of development. 

ARC should take the following steps to put this new framework for employment impact 

measurement into place. 

 Establish Standardized Practices to Assist Grantees with Impact 

Measurement 

In order for data on employment impacts to be meaningful, ARC should adopt 

standardized practices for measuring job creation and retention and require grantees to 
Program Evaluation of ARC’s Tourism, Page 38 

Cultural Heritage and Natural Asset-Related Projects

apply these practices. At the same time, ARC should provide grantees with technical 

assistance to ensure consistency and minimize grantee time commitments and cost. 

As indicated earlier in this chapter, the most appropriate way to measure the employment 

impacts of ARC’s tourism, cultural heritage and natural asset-related projects is to first 

measure increases in local revenues attributable to these projects, and then convert these 

numbers into employment equivalents using economic impact models. Methodologies 

for economic impact analysis are well established and, in the tourism realm, have been 

specially adapted to measure the impacts of cultural, heritage, and agricultural tourism 

projects. Other methods have been developed to measure the impacts of tourism 

marketing programs, including conversion and advertising tracking studies. And in 

recognition of the growing role of the Internet in tourism marketing, new methods are 

being developed and refined to measure the impacts of tourism marketing websites on 

visitation levels. These tools have been used extensively by academic researchers, 

government tourism agencies, and private consultants. 

As one example, the National Park Service developed the Money Generation Model 

(MGM2) to estimate local economic impacts of National Park Visitors. The model has 

been made available to other organizations, using a simple fill-in-form for estimating 

economic impacts. It requires three basic inputs: 1) the number and types of visitors; 2) 

visitor spending patterns; and 3) local economic ratios and multipliers. Multipliers may 

be obtained from the MGM2 model or from other regional input-output models. A 

custom version of the model has been developed for use by heritage areas. 

ARC should work with experts in the field to identify best practices in economic impact 

measurement applicable to its tourism, cultural heritage and natural asset-related projects, 

adopt a set of measurement methods, and assist grantees to apply these methods. It could 

take a number of steps to accomplish this: 

Providing publications and training seminars to grantees on economic impact 

measurement;

Developing standardized survey instruments, interview protocols, and other data 

collection tools and providing instructions on data collection methods; 

Providing on-line access to an economic impact model that would calculate the 

employment impacts of ARC projects based on visitation and visitor spending 

data collected by grantees; 

Working with state tourism agencies that already collect tourism data to 

customize data collection for local areas; 

Creating a technical assistance fund that could be used by grantees to hire 

university researchers or consultants to assist with data collection and analysis. 

Australia provides an example of how a federal government agency can assist local 

tourism organizations to conduct research on tourism impacts. Tourism Research 

Australia (TRA) has established a Destination Visitor Survey program, which involves 

collaboration between TRA and state and local tourism organizations. For each project, 
Program Evaluation of ARC’s Tourism, Page 39 

Cultural Heritage and Natural Asset-Related Projects

TRA ensures a sound research design to deliver high quality data. The agency has 

developed a model for its Visitor Profile and Satisfaction Program that includes research 

design, fieldwork methodology, questionnaires, analysis, and reporting. This 

standardized model enables local organizations to minimize the cost and time normally 

taken to complete such projects. 

 Tailor Measurement Methods to Project Characteristics 

While measurement methods should be standardized, they should also be tailored to take 

into account different project types and time frames. Most, importantly, the impacts of 

implementation projects should be measured differently than those of planning projects. 

In general, implementation projects are expected to directly result in job creation and 

retention, while planning projects are one step removed. Consequently, impact 

measurement for implementation projects should focus on quantitatively measuring job 

creation and retention achieved as a result of project completion. For planning projects, 

the measurement process is two-fold. First it must be determined whether the outcomes 

of planning projects – studies or plans completed – result in projects or programs that are 

likely to generate employment impacts. If so, a second determination must be made 

about whether quantitative impact measurement is practical. For example, if a feasibility 

study results in the development of a destination tourism attraction, employment impacts 

can be quantified using established methodologies. On the other hand, if a tourism 

strategic plan leads to the implementation of several projects by multiple actors, 

quantitative impact measurement becomes more difficult. In such cases, ARC could 

develop information tools that would enable it to make a more qualitative assessment of 

employment impacts rather than attempting to fully quantify these impacts. For example, 

if certain projects are implemented as a result of a tourism strategic plan, and the 

implementers of some of those projects collect and analyze data suggesting additional 

visitation and tourism spending, ARC could reasonably conclude that some job creation 

has occurred although it would be unable to quantify the full employment impacts of the 

planning project. 

Another distinction should be made between tourism implementation projects that are 

“tourism generators” and those that are “tourism contributors.” As noted earlier in this 

chapter, some tourism implementation projects such as development or improvement of 

destination attractions or development of marketing tools can be considered tourism 

generators because they attract visitors to an area and generate additional spending. 

Other projects such as improvements to small heritage sites or cultural venues, 

development of way finding signage, or local distribution of tourism information are 

more appropriately considered tourism contributors because they enhance the visitor 

experience and may increase visitation to specific sites, but can’t easily be linked to 

overall increases in visitor spending. For such projects, ARC could develop information 

tools that focus on tracking visitation and spending at specific sites. Increased visitation 

and spending could be assumed to be at least partly attributable to the ARC project and 

could be considered evidence of some contribution to job creation or retention, although 

the impact could not be quantified. See Figure 6 for a graphical display of the process. 
Program Evaluation of ARC’s Tourism, Page 40 

Cultural Heritage and Natural Asset-Related Projects

 Require and Assist Applicants to Develop an Employment Impact 

Measurement Plan 

ARC should require all applicants to prepare an employment impact measurement plan as 

part of their project application. Application guidelines should articulate ARC’s 

expectations regarding impact measurement, with distinctions made between planning 

and implementation projects, and between tourism generator and tourism contributor 

projects. The plan should describe what impacts will be measured, what methods will be 

used, when it will be done, and who will be responsible for data collection and analysis. 

A budget line for impact measurement should be included in the project budget. 

ARC should be prepared to assist applicants with this process. This should include 

providing publications and training seminars on impact measurement and introducing 

applicants to the tools available from ARC to assist with this task. ARC should also 

review the plan and suggest changes when necessary. Finally, ARC should be prepared 

to share any costs with the grantee. 
Program Evaluation of ARC’s Tourism, Page 41 

Cultural Heritage and Natural Asset-Related Projects

 Establish Post-grant Reporting Requirements

As noted earlier, it is typically premature to measure the employment impacts of this 

group of projects at the end of the grant period. Final reports should instead focus on 

reporting outcomes that can serve as preliminary indicators of the nature and extent of 

employment impacts that might be expected within 1-3 years of project completion. In 

the case of planning projects, the final report should focus on whether the anticipated 

research or plan was completed, what, if any, implementation steps have been taken or 

are anticipated, and what employment impacts are expected to result within three years of 

implementation. In the case of implementation projects, the report should focus on what 

implementation steps have been completed, whether there is any initial evidence of 

employment impacts, and what employment impacts are expected to result within three 

years of project completion. 

In order to obtain meaningful information on actual employment impacts, ARC should 

require grantees to submit post-grant reports at specified intervals, perhaps one and three 

years after project implementation. For planning projects, ARC may want to extend the 

reporting time frame to more fully capture the employment impacts of project 

implementation. The three-year reporting interval meshes with ARC’s definition of job 

creation, which anticipates measurement of private sector jobs created within three years 

of project completion. This report would provide the results of the employment impact 

measurement plan described above. The one-year reporting interval would enable ARC 

to assess how the project is progressing and to determine whether ARC should intervene 

to assist projects experiencing particular challenges. This would be particularly relevant 

for planning projects that have not moved toward implementation, giving ARC the 

opportunity to take action to “protect” its planning investment. 

Obtaining post-project reports is likely to be difficult if grantees have no incentive to 

provide them. ARC should consider ways to put such incentives into place, perhaps by 

executing separate contracts for impact assessment costs, to be paid upon submission and 

acceptance of the assessment report. Another option would be to keep the grant open for 

three years, withholding final payment until submission and acceptance of the assessment 

report. 

AN EX POST FACTO ESTIMATE OF EMPLOYMENT IMPACTS 

The project files provided by ARC indicated that the projects (those that are closed) 

generated 583 jobs. Those projects were initially projected by grantees, as reflected in 

their grant requests, to create 2,113 jobs. This suggests that projects actually create only 

one-quarter as many jobs as expected. This result seems to conflict strongly with the 

results of our fieldwork: surveys, interviews and case studies. While some projects failed 

or were less successful than expected, project directors overwhelming considered the 

projects successful. As we note above, 86 percent reported that the projects were 

completely or mostly successful. 
Program Evaluation of ARC’s Tourism, Page 42 

Cultural Heritage and Natural Asset-Related Projects

Within this section we have described the difficulties grantees have in measuring 

employment impacts. These pitfalls bring into question the grantees reported 

employment impact estimates. We therefore examined ways to ex post facto determine a 

more accurate job estimate.

Our review of the interviews we conducted tell us that project managers often felt that 

employment impacts were significant but that they had no way of accurately estimating 

them. They reported that local restaurants indicated increases in business and hotels had 

higher occupancy rates but translating the anecdotal evidence into employment estimates 

was difficult. The interviews with economic development stakeholders sometimes 

reported employment increases but they were not always consistent with project manager 

numbers or estimates by other stakeholders. The methodological issues discussed in this 

section suggest that project managers are often not in a position to adequately estimate 

the impacts in any event.

The material earlier in this section describes the difficulties of estimating employment 

impacts and suggests the investigation of various impact models that can assist grantees 

and ARC in better estimating the full economic and employment impacts of projects. We 

therefore turned to other studies that used such models to see if they provide a way to 

generate a reasonable approximation of true employment impacts from the ARC tourism 

project portfolio we were provided. 

Several analyses are available of impacts from tourism developments in Appalachia and 

similar regions but most deal with a specific, typically large and high profile project. 

Fortunately a recent study was done of a portfolio of projects within one area, specifically 

of the Crooked Road region of Virginia. ARC has provided funding for virtually all of 

the Crooked Road projects (mostly in tandem with other Federal, state, or local funds) 

and therefore provides a good test-bed for application to not only its investments in 

Virginia but to the overall ARC portfolio. 

The 2008 study was conducted by Sustainable Development Consulting (SDC) and 

entitled Economic Impact Assessment of the Crooked Road: Virginia’s Heritage Music 

Trail. The analysis was based on surveys and other data sources that estimated visitation, 

dollars spent, length of stay and other direct impacts. These data were then examined 

within the US Department of Commerce Regional Input-Output Modeling System (RIMS 

II). The RIMS II model is a widely utilized and respected model and is used in a broad 

array of economic impact analyses.

They estimated that the Crooked Road Music Trail generated $22.8 million in annual 

revenue to the region and accounted for 445 full-time equivalent (FTE) jobs. How does 

this compare to the estimated impacts as reported by ARC grantees for the same portfolio 

of projects? The grantees reported that their projects generated 80 jobs or 18% of the 

estimated total reported by SDC. On the other hand the 445 FTE estimate is quite close 

to the initial projections made by the ARC grantees of 416. 
Program Evaluation of ARC’s Tourism, Page 43 

Cultural Heritage and Natural Asset-Related Projects

The ratio of grantee job estimates to the estimates generated by a widely accepted 

economic model provides a means of estimating job impacts by the full portfolio package 

of ARC tourism, cultural heritage and natural asset-related projects. This model suggests 

that the actual job impacts for the ARC portfolio are 3,243, compared to the reported 

estimate of 583. This estimate is much closer to the survey-based estimate of 2,588 we 

developed in Section 3. These results strongly suggest that ARC investigate a practical 

implementation of an economic impact model such as RIMS II to more accurately reflect 

the success of its investments in Appalachia. Preferably the project managers would 

access an online model on a project-by-project basis. Alternatively ARC or an outside 

consultant could use a model to look at the recent portfolio of projects as a whole.
Program Evaluation of ARC’s Tourism, Page 44 

Cultural Heritage and Natural Asset-Related Projects

 

SECTION 5: SUSTAINABILITY & THE TRIPLE BOTTOM LINE

As noted, the Ford Foundation asked us to examine the application of Triple Bottom Line 

(TBL) strategies (defined as simultaneously building economic, social and environmental 

assets) to economic development projects, specifically projects that were built on 

tourism, cultural heritage and natural assets. The Ford project was conducted in tandem 

with this evaluation. Specifically, Ford is interested in applying the tenets of TBL using 

a four-pronged development strategy that uses cluster-based development, value chains, 

entrepreneurship, and financial services as a combined development model for TBL. 

They felt that the ARC work would provide a platform for building a better 

understanding of the opportunities and challenges for adopting a TBL perspective in rural 

areas including Appalachia.

In this section, we look to analyze how sustainable development and, more specifically, 

TBL can provide a logical platform for building stronger communities. We will describe 

the development of the triple-bottom-line from a tool that companies use to broadly 

describe their corporate impacts to a more general structure under which communities, 

organizations, funders and companies can develop and evaluate strategies that reflect 

concern for the holistic health of human action. In particular this report looks at the 

following questions: 

How can we use sustainable development as a launch pad for TBL? 

What has been the logical and theoretical development of TBL and what are its 

strengths and shortcomings? 

What metrics can be used to evaluate how initiatives impact economic, social and 

environmental elements?

Can the metrics be effectively measured, verified and used to build better and 

more sustainable communities?

How has TBL been used in the development of the portfolio of ARC projects? 

What methods can be used to help communities and organizations think about the 

TBL and build programs, strategies that reflect our mutual concern for a healthy, 

vibrant, equitable and sustainable future?

In addition to the detailed analysis of funded projects described in the sections above, the 

team also conducted significant research on the ways in which tourism projects have been 

traditionally evaluated and how the triple bottom line approach can be used to evaluate 

these types of projects. Two reviews of sustainability and TBL were prepared during this 

project, one by Michael Evans, Carol Pollard and James Stoddard of Appalachian State 

University and another by Scott Sawyer, an expert in the field of sustainability. Their 

efforts form a strong academic and theoretical underpinning for the following sections. 

The full reports are included as appendices but much of their work is reflected and 

summarized here.

TBL is built on the bedrock of sustainable development theory and practice. Before 

looking at TBL, which can be thought of as a subset of sustainability, we need to look 
Program Evaluation of ARC’s Tourism, Page 45 

Cultural Heritage and Natural Asset-Related Projects

first at sustainability and its definitions. Appendix G, written by Scott Sawyer, provides 

a more thorough portrait for those who want to learn more of its history, development and 

theoretical and scientific underpinnings. Following the discussion of sustainable 

development, the section will turn to the triple bottom line. Both appendices F and G 

contain more detail on TBL and the literature surrounding its use. 

SUSTAINABILITY AND SUSTAINABLE DEVELOPMENT 

The concept of the triple bottom line originated from the notion of sustainability and 

sustainable development. Ecologically sustainable development (ESD) thinking was first 

espoused in the Brundtland Report (World Commission on Environment and 

Development, 1987) and reiterated during Agenda 21 and the Rio Declaration on 

Environment and Development (1992). 

The Brundtland Report definition has become the standard: 

“Sustainable development is development that meets the needs of the 

present without compromising the ability of future generations to meet 

their own needs.” 

A more specific definition clarifies sustainability as:

“the use of natural resources to support economic activity without 

compromising the environment’s carrying capacity, which is its ability to 

continue producing those economic goods and services ” (Manning and 

Dougherty 1995) 

As we have developed more complex social and economic organization we have moved 

to more resource extractive (non-renewable) methods to sustain these systems with the 

most obvious being our increasing reliance over time on non-renewable energy. We live 

in an economy that is built on exploitation of natural resources that are inherently finite.

Both of the above definitions point to the quandary faced by mankind: how do we 

exploit (or perhaps nurture) the natural environment and its resources without destroying 

the ecosystems and environmental services it provides? How do we use resources in 

ways that do not destroy the basis of our survival while at the same time dealing with 

issues of poverty, hunger, inequality, economic opportunity, health and governance? 

The evidence is clear that much of what we call economic development including tourism 

development is unsustainable. As Sawyer notes we are changing our environment and 

ecosystems at an increasingly rapid rate and that most of our major ecosystems are being 

used unsustainably and appear to be on paths from which they may not recover without 

significant change in human behavior. Global warming may be just the most talked 

about these looming changes but there are other systems that may also be approaching 

collapse.
Program Evaluation of ARC’s Tourism, Page 46 

Cultural Heritage and Natural Asset-Related Projects

In addition the negative impacts of non-sustainable and exploitive economic development 

have tended to fall on those most vulnerable. For example one report notes that “Nearly 

all of the studies over the past two decades have found that environmental hazards under 

investigation are inequitably distributed according to race or income or both.” (Mohai 

1995). Within the Appalachian region the negative environmental impacts of coal 

mining clearly fall on the poor neighbors of the mines. 

Some analysts believe we are living in an economic and social system in which our 

primary focus on one part of the triple bottom line, economy, not only negatively affects 

the social and environment, but threatens even the economic element that drives it.

Amidst this doom and gloom Sawyer points out: 

Sustainable development advocates believe that a less apocalyptic future is 

possible. In the specific example of energy, societies can only meet the 

needs of the present without compromising the ability of future 

generations to meet their needs by conserving energy, using it more 

efficiently, and through a switchover to renewable energy (e.g., wind, 

solar, geothermal, hydro, biomass, biofuels, ocean tidal, waste, etc.). 

Unsustainable development, then, refers to societal dependency on an 

economic process - “the treadmill of production” - that requires 

increasing withdrawals of energy and material from ecosystems (and the 

generation of waste and pollution) in order to promote societal progress 

for some segments of society while further impoverishing other segments 

of society (Schnaiberg and Gould, 1994). 

Ultimately this leads to the need to focus on an “interconnected core of principles that 

provide a kind of game plan for avoiding thresholds while defining the contours of 

sustainable development”: 

 

Social and Environmental Justice: Advocates of social and environmental justice stress 

the need to promote opportunity and equity within our society, between societies, 

between generations, and between species.

Stewardship of Built and Natural Environment: Stewardship advocates invoke Aldo 

Leopold’s well known concept of the land ethic to stress the need to maintain the 

resiliency of life support systems such as forests in perpetuity (Hawken et al., 1999), 

while historic preservation advocates and others stress the need to maintain the fabric of 

communities.

Ecological Modernization: Advocates of ecological modernization, or natural capitalism 

(Hawken et al., 1999) stress the need to green societal institutions (e.g., economic 

institutions, political institutions) through changes in values, as well as changes in 

technology, such as biomimicry (Benyus, 1997), renewable energy, and cradle-to-cradle 

production processes (Mol and Sonnenfeld, 2000). 
Program Evaluation of ARC’s Tourism, Page 47 

Cultural Heritage and Natural Asset-Related Projects

Examples of the differences between traditional and sustainable economic development 

are summarized in Table 10.

Table 10: Traditional and Sustainable Economic Development 

Source: Sawyer, Appendix G 

Sustainable development, as a definition and goal, is not without its critics. It is accused 

of being inconsistent, vague, and too broad to be useful. While there are clearly 

differences in how sustainable development is being used as a construct, there can be 

little argument about the need for it. As Sawyer summarizes, the critiques have the 

“unmistakable odor of a red herring.” 

PEOPLE, PLANET, PROFITS: DEVELOPMENT OF THE TRIPLE 

BOTTOM LINE

Unlike sustainable development, the triple bottom line began as a straightforward tool 

designed for businesses that wished to increase their business focus to more than just 

monetary profits. It is part of an overall move, mostly by large publicly held 

corporations, to institute corporate social responsibility or CSR into their business 

model. Companies that adopt CSR policies are expected to track and self-regulate their 

behavior in ways that look beyond short-term monetary gain. 

There has been a significant development of “infrastructure” surrounding CSR. There 

are certification for companies to win, social accountability standards, an ISO 14000 

environmental management standard, and a United Nations working group developing 

reporting and disclosure guidance. 
Program Evaluation of ARC’s Tourism, Page 48 

Cultural Heritage and Natural Asset-Related Projects

Criticisms of CSR are many. Neoclassical economists claim that the sole role of 

corporations is to generate profits for stockholders. Others claim, with substantial 

evidence, that corporations use CSR as nothing more than a marketing tool and report 

only the good while burying the bad. Global corporations, including a majority of the 

world’s largest oil companies, conduct and publicize annual CSR reports. 

Others contend that while corporations may have ulterior motives, the adoption of CSR 

does provide a certain amount of transparency and a tool for stockholders to push 

corporations to more responsible practices. In addition, as we will discuss below, there is 

a solid single bottom line reason to adopt CSR: it may lead to a more profitable and 

sustainable business. 

It is true that many organizations have adopted CSR as an accounting method or a slick 

public relations campaign and have overlooked the need to establish and follow a 

corporate philosophy in keeping with its intentions. However, when CSR is embraced to 

its fullest extent, it forces organizations to engage with their full complement of external 

stakeholders (e.g., customers, vendors, residents, non-profit activists).

TBL is a term often used to describe the economic, social and environmental

accountability of a firm but it also used within a broader range of organizations including 

the public sector. A key element of corporate TBL requires that corporate performance 

should be geared not only to benefit its “shareholders” but all of its “stakeholders” 

including groups such as the local community within which business operations are 

conducted. In the same way, organizational or public sector TBL requires the integration 

of a broader range of people and issues. For example, the potential explicit adoption of 

TBL by the Appalachian Region Commission would require them to evaluate their 

projects on more than just jobs created, incomes increased and investments leveraged.

TBL Defined

The term “Triple Bottom Line” evolved in the mid 1990’s when the consulting group 

AccountAbility coined the term which was later popularized by John Elkington (1998) in 

his book Cannibals With Forks: The Triple Bottom Line of 21st Century Business. The 

triple bottom line is a concept that expands the notion of organizational performance 

evaluation to include not only the traditional financial bottom line to one that also 

includes environmental quality and social justice. Others contend that the elements of the 

triple bottom line can be combined in pairs resulting in some of the popular terms used to 

describe sustainable projects. For example, eco-efficiency involves optimizing economic 

and environmental goals, fair trade is concerned with conducting economic activity with 

attention to social consequences, and environmental justice involves achieving social 

equity while respecting environmental protection. (Marshall and Toffel 2005) 

From a corporate standpoint it is important to note that TBL was intended to be a way of 

thinking about corporate social responsibility, not just an accounting methodology. 

Elkington stressed the integration of philosophy and accounting measures in 
Program Evaluation of ARC’s Tourism, Page 49 

Cultural Heritage and Natural Asset-Related Projects

implementing TBL and suggested that there are seven drivers that have emerged to 

support its use (Table 11). 

Table 11: Drivers of the Triple Bottom Line 

Drivers Characteristics

Markets Unlike the past when business used competition as an excuse not to 

incur additional costs (for environmental and socially responsible 

behavior), future business will use TBL thinking to build a case for 

action and investment in environmental and social infrastructure.

Values Society’s changing values towards more environmental and social 

consciousness will influence corporate culture. 

Transparency Organizations are facing growing international transparency about their 

environmental and social policies which is being facilitated by new 

societal value systems and radically different information technologies.

Life-Cycle 

Technology 

There is a new emphasis on cradle-grave performance of products that 

examines product performance from extraction of raw materials 

through recycling or disposal.

Partners New forms of partnerships are evolving among business organizations 

and campaigning groups (e.g., Greenpeace).

Time Organizations are realizing that they must pay attention to long term 

rather than short term performance.

Corporate 

Governance

Top level organizational managers will be forced to pay attention to 

new issues such as the design of corporations and boards and their 

value chains of ‘business ecosystems.’

(adapted from Elkington, J. (2004) “Enter the Triple Bottom Line”)

As Table 11 suggests, first-rate TBL uses relevant, common indicators to make it easier 

to compare performance or “value added” across organizations and requires honest, open 

and transparent disclosure. Ideally, it has been suggested that TBL should lead to 

improvements in corporate performance. TBL reports should include key goals for 

improving organizational performance into the future – preferably quantitative and timebound goals. The Global Reporting Initiative, a program developed under the auspices of 

nearly 20 agencies including the World Business Council for Sustainable Development, 

the United National Environment Program and the World Resources Institute, 

standardizes TBL measures and areas to be reported and currently has more than 1,300 

participating organizations. The GRI suggests five categories of TBL indicators (GRI 

2008). These include: 

1. Economic performance 

2. Environmental performance

3. Social performance

4. Labor practices and “decent” work

5. Public policies and implementation measures
Program Evaluation of ARC’s Tourism, Page 50 

Cultural Heritage and Natural Asset-Related Projects

The Economic Dimension of TBL 

Perhaps the easiest dimension to capture during triple bottom line evaluation is the 

economic dimension. The economic dimension can be assessed using traditional 

financial performance indicators such as sales revenue, profit, return on investment or 

shareholder value models. Other industry-specific measures are also readily available to 

assess the economic performance of an operation. For example, in the tourism industry, 

indicators such as heads in beds (for hotels) or visitation (for attractions) are popular 

measures. More difficult to assess are the social and environmental dimensions of the 

triple bottom line. These dimensions are discussed next.

The Social Dimension of TBL

It has been suggested that the social dimension of TBL, sometimes referred to as social 

capital consists of two components: human capital (employees, contractors, suppliers and 

advisors) and investment by the social systems that support the business (Dwyer, 2005; 

Sauvante, 2001). A leading authority on social capital, Robert Putnam (1993) describes 

social capital as the “trust, norms and networks needed to facilitate cooperation.” Others 

suggest that social capital is “created from a myriad of everyday interactions between 

people” and identify key dimensions such as valuing self and others; trust (interpersonal 

and generic); connection (community participation and social networks); multiple 

relationships; and reciprocity in relationships. Some social indicators suggested by the 

GRI include: employee retention rates; job satisfaction levels and investment per 

employee in illness and injury prevention. 

Miller, Buys and Summerville (2007) provide an interesting example of the application 

of the social dimension. They developed a framework and indicators to assess the social 

dimension of the TBL for the Australian dairy industry. In their view, the social 

dimension involves social well-being, working with employees, their families, the local 

community, and society to improve their quality of life. Organizations with social 

awareness better understand and account for the consequences of their operations on the 

social well-being of the communities affected by those operations. Their framework 

specifies four dimensions of an operation’s social impact, including an 1) individual’s 

well-being, 2) community well-being, 3) employment experiences and satisfaction, and 

4) organizational impact.

The Environmental Dimension of TBL

The environmental dimension is referred to by some as natural capital. For example, 

Onyx and Bullen (2000) and Schnake (1991) suggest that the availability of natural 

resources such as forests, minerals, fish and soil have long been taken for granted despite 

the fact that this is the “capital” upon which our existence depends (i.e., clean air and 

water). As Onyx (2000) so succinctly put it “wealth that destroys the basis of life is no 

wealth at all.” Environmental indicators suggested by the GRI include items such as total 

energy use, use of recycled materials and water sources significantly affected by an 

organization’s use of water. In effect companies and organizations need to selfinternalize the externalities of their production.
Program Evaluation of ARC’s Tourism, Page 51 

Cultural Heritage and Natural Asset-Related Projects

CRITICISMS OF THE TBL METHODOLOGY

The critics of TBL and its value are many. Both appendices F and G provide a depth of 

technical detail on the issues so we provide a summary here. Categories of criticism 

include the following. 

One critique notes that TBL focuses principally on controlling for bad effects and not 

necessarily enhancing the positive effects of products. Some propose a focus on the 

triple TOP line where the emphasis is on increasing the good effects and thus, by 

definition, eliminating the bad.

Others argue about measurement claiming that social and environmental bottom lines 

cannot be measured in objective ways (as can the economic bottom line) and therefore, it 

is not possible to compute something similar to a social or environmental net profit. In 

essence then, there is no common currency to allow for the expression of good and bad 

magnitudes with respect to the social and environmental bottom lines.

It is also argued that because of the difficulties of measuring social and environmental 

impacts corporations pick and choose those measures that put them in the best light. If 

there is no generally accepted set of measures, unlike for example the Generally 

Accepted Accounting Principles (GAAP), critics claim that there is no way to assure that 

the organization is giving a true accounting. 

RESPONSES TO THE CRITICISM 

While it is clear that the development of TBL as a concept and methodology is not 

mature, none of the issues above are insurmountable. 

Proponents suggest organizations that support the use of triple bottom line metrics do not 

have to assess their social and environmental bottom lines the same way that their 

financial or economic bottom lines are assessed. In addition, they have pointed out that 

even the popular net income (profit) aggregate measure of financial performance was 

deemed a deficient indicator of financial performance. One need only look at the recent 

global economic and financial crisis to see that measuring profits is not a trivial act and 

that neither regulation nor GATT requirements were able to generate accurate 

assessments of company finances. Since financial performance cannot be summarized 

with a single objective number, one should not expect to assess social and environmental 

performance in this manner. It is also certainly true that organizational performance is 

multidimensional. (Pava 2007) 

To summarize, some suggest that TBL is nothing new. However, TBL thinking may 

produce some useful and dependable indicators of societal and environmental indicators 

of “value added” in relation to corporate responsibility and accountability. Furthermore, 

an organization’s motivation to use TBL might also include its attempt to differentiate 

itself from its competition to attract investors and customers who might be willing to pay 

more to support ethical firms. 
Program Evaluation of ARC’s Tourism, Page 52 

Cultural Heritage and Natural Asset-Related Projects

The main challenge associated with TBL remains measurement. The social and 

environmental measurement and comparability of organizations and industrial sectors 

over time is proving to be a very difficult task. As Norman and MacDonald (2003) point 

out, it is commonly understood that in modern management theory; “if you can’t measure 

it, you can’t manage it.” Current TBL measurement efforts are time-consuming, vague, 

sometimes badly misleading (i.e., public relations rhetoric) and can provide a 

smokescreen for poor performance. 

As Sawyer notes the “gist of the triple bottom line argument is that social and 

environmental performance can be measured in fairly objective ways, and that firms 

should use these results in order to improve their social (and environmental) 

performance. Moreover, they should report these results as a matter of principle, and in 

using and reporting on these additional bottom lines firms can be expected to do better by 

their financial bottom line in the long run (Norman and MacDonald, 2004). The triple 

bottom line is seen as a metaphor to remind us that corporate performance is multidimensional (Pava, 2007)”. 

Perhaps most importantly, there is strong logic and evidence that suggests firm adoption 

of TBL often improves the financial bottom line. If this is the case then firms might 

adopt TBL and have substantial incentives to develop real and accurate measurements 

across the spectrum of people, planet and profit. Sawyer summaries 13 potential avenues 

through which TBL can strengthen the financial bottom line. More detail is found in 

Appendix G. 

1. Anticipating the inevitable, for example foreseeing regulatory change resulting from 

environmental harm, helps corporations get ahead of the competition 

2. Better reputation management is believed to lead to easier hiring of the best talent 

and higher retention of top talent. 

3. Increased employee productivity and worker creativity may occur when employees 

work in a cleaner and healthier environment. 

4. Reducing risk of product liability and major environmental disasters leads to easier 

financing and greater credibility with banks and other financial institutions. 

5. Reduced costs by reducing material use and building stronger locally/regionally 

based supply chains 

6. Reducing (and recapturing) energy expenditures lowers costs.

7. Making money from prevention (design, consulting, green architecture, etc.) as well 

as making money from cutting-edge environmentally-relevant research and development. 

8. Reducing the costs of waste handling and disposal and turning wastes into inputs 

and outputs. 

9. Making money from clean-up (e.g., environmental mitigation, restoration, and 

research and development on new environmental technologies). 

10. Maintaining market shares with old customers who want more environmentally 

friendly products as well as creating new distribution opportunities to new customers who 

are more environmentally sensitive.
Program Evaluation of ARC’s Tourism, Page 53 

Cultural Heritage and Natural Asset-Related Projects

11. Building new centers of technical expertise in the firm that can turn into new profit 

centers. 

12. Diffusing learning and efficiencies (via energy savings, waste reduction, new 

supply chain connections, process innovations, etc.) within the firm. 

13. Building synergies that develop with new products, production processes, trade 

alliances, supply chain expansion, etc., once a critical mass of sustainable business 

activities are underway and linked with one another. 

The case for using TBL as a means to improve the financial bottom line will, of course, 

vary substantially from sector to sector and business to business. 

UTILITY OF TBL FOR EVALUATION OF TOURISM DEVELOPMENT 

PROJECTS 

Do the 13 bottom line impacts from the adoption of TBL apply to sustainable tourism? 

What benefits accrue to the tourism organization for implementing within a TBL 

framework?

Dwyer (2005) provided insight into the benefits that accrue to the tourism development 

organization resulting from the adoption of the TBL approach. Included in these benefits 

are efficiencies and cost savings, improved market positioning, better stakeholder 

relations, improved strategic decision making along with benefits to the wider 

community. 

With regard to efficiencies and cost savings, TBL reporting can identify potential cost 

savings such as reduced operating costs via the reduction of materials and energy use, 

enhanced operational and design efficiencies, recycling and reusing wastes, as well as 

reduced transportation storage and packaging costs. Lower compliance costs may result if 

regulators have a better understanding of an organization’s operations. Human resource 

costs may be reduced when employees are attracted and retained by an organization that 

focuses on community and environmental values. Finally, capital costs may be reduced as 

the organization has improved access to “green” and “ethical” investment funds. 

From a marketing perspective, tourism development organizations can benefit from 

adopting the TBL approach through improved market positioning. Improved market 

positioning can result when the organization’s brand image is enhanced when consumers 

become aware of the environmental and social sensitivity of the organization. In 

addition, the competitive differentiation afforded by the TBL approach can assist the 

organization in appealing to new markets, and encouraging repeat visitation. 

Enhanced tourism development organization stakeholder relationships can result as the 

organization’s beneficial activities permeate throughout the community fostered by 

increased transparency. These enhanced relationships can also provide a competitive 

advantage as the organization’s positive reputation is communicated among stakeholders 

strengthening the organization’s brand. 
Program Evaluation of ARC’s Tourism, Page 54 

Cultural Heritage and Natural Asset-Related Projects

Finally, adopting the TBL approach can improve a tourism development organization’s 

strategic decision making. The TBL approach forces the organization to focus on 

managing those tasks that get measured, systematize the best practices and benchmarks, 

improve risk management activities, improve the quality of information for decision 

makers, and facilitate information sharing throughout the organization which promotes 

integrated decision making within the organization and with other organizations in the 

community. 

As we note there are unique sets of sustainability issues faced by different industrial 

sectors such as the automotive, banking and tourism sectors, among others. For these 

reasons, to use TBL to evaluate sustainable tourism, it is necessary to explore TBL in the 

particular context of sustainable tourism. The next section ties together TBL with 

sustainable tourism and describes potential variables that can measure TBL within this 

industry. 
Program Evaluation of ARC’s Tourism, Page 55 

Cultural Heritage and Natural Asset-Related Projects

SECTION 6: TBL & SUSTAINABLE TOURISM

There are of course a myriad of industries within rural America. For the purposes of this 

effort in collaboration with the ARC, we look at the application of TBL and TBL 

measures to the portfolio of ARC tourism, cultural heritage and natural asset-related 

projects provided to the RTS project team. (For brevity we will use the term “tourism” to 

refer to these projects.) As we suggested in the previous section we will look at the TBL 

measures and methods that are applicable to it. Some of these measures will translate to 

other sectors while others are unique to tourism. The process, though, should provide a 

guidebook for looking at other sectors and their particular TBL issues. 

LINKING TBL, SUSTAINABILITY AND SUSTAINABLE TOURISM 

The concepts of TBL, sustainability, sustainable development and sustainable tourism are 

closely linked. As Manning and Dougherty (1995) suggest, sustainable tourism is the 

best way to preserve the “golden goose” of tourism. They feel it is a viable tool to 

maintain and enhance a destination’s competitiveness. It is suggested that sustainable 

tourism and development finds a balance between economic prosperity, environmental 

protection and social equity. 

Sustainable Tourism Defined 

For our purposes, sustainable tourism refers to a level of tourism activity that can be 

maintained over the long term because it results in a net benefit for the social, economic, 

natural and cultural environments of the area in which it takes place. 

Sustainable tourism is often used as an umbrella concept, under which terms, such as ecotourism, heritage and cultural tourism, and agri-tourism, may fall. Each sub-sector 

suggests unique measures relevant to analysis using the TBL framework. Here we take a 

different tack, by looking at rural tourism (using the portfolio of ARC projects) and 

describe how they might be effectively examined or measured using a TBL framework. 

We will look at tourism in the same way we might look at an auto plant except the 

variables of interest will differ. We want to answer the question “How does this tourism 

asset work from the standpoint of people, planet and profit and how can we make it 

sustainable?”

Developing Measures of TBL for Sustainable Tourism

As stated earlier, one of the most challenging aspects of implementing TBL is the 

difficulty associated with developing meaningful economic, social and environmental 

indicators. Of these, the greatest challenge is associated with attempting to quantify 

social and environmental impacts. Analysts suggest that, in assessing tourism impact, the 

initial question when considering appropriate indicators must be “what should the 

tourism and recreation industry sustain?” followed by a need to identify indicators that 

can be monitored to determine if current policies are facilitating sustainability taking into 

consideration the context in which they exist. 
Program Evaluation of ARC’s Tourism, Page 56 

Cultural Heritage and Natural Asset-Related Projects

The following discussion describes general areas of indicators with the three elements of 

TBL. After that we will describe the specific measures within these general areas.

Economic Indicators. Economic measures of tourism are the most straightforward to 

establish and have long been in place. Typically, economic impact is measured by hotel 

occupancy rates, number of nonresident visitors, per capita tourist expenditures and 

lodging revenues, number of tourism employees and income from tourism. Others have 

developed indicators focused on employment issues, destination economic benefits 

seasonality and poverty alleviation. A comparison of the economic indicators that 

emerged from previous research is shown in Table 11. 

Table 11: Review of Economic Indicators of Tourism Impact 

Theme Indicators

Seasonality Degree of seasonality

Strengthening Shoulder Seasons and Low Seasons

Provision of sufficient infrastructure year-round

Short-term and seasonal employment

Employment Number and quality of employment in tourism sector

Professional and personal development

Contentment from work

Lack of skilled labor

Labor income

Destination 

Economic Benefits

Business investment in tourism

Tourism revenue

Community Expenditures

Net Economic Benefits

Changes in cost of living

Financial rate of return to operators

Gross operating surplus of different tourism industry sectors

Hotel occupancy rates

Number of nonresident visitors

Amount of money leaving tourism locality

Poverty Alleviation Stabilizing and improving community income

Improving local employment opportunities

Achieving equitable distribution of tourism funds/benefits 

across the community

Number and type of development programs in place

Community survey assessment of usefulness and success of 

various development programs

Evaluating less tangible, non-economic, livelihood priorities

Social Indicators. As discussed previously, social impact is a term that is closely linked 

with the concept of social capital, i.e., improving trust, encouraging cooperation and 
Program Evaluation of ARC’s Tourism, Page 57 

Cultural Heritage and Natural Asset-Related Projects

collaboration, recognizing and enhancing individual and organizational networks and 

fostering life-long learning (Rogers, 2003). 

To date, research has suggested a number of social impacts of tourism on communities. 

These include state of the local economy, maturity of the tourism destination and level of 

community attachment. Other suggested social indicators fall into broad categories of 

support for access and equity, pressure on services, pride and sense of belonging to local 

area, support for cultural and artistic endeavors, regional showcase and community health 

and safety issues. A detailed comparison of the social indicators that have emerged from 

previous literature is shown in Table 12. 

Table 12: Review of Social Indicators of Tourism Impact

Theme Indicators

Background Resident population

Visitors to the Area

Land use, Particularly Tourism Development

Social Environment Access, especially parking

Highway traffic count

Housing Affordability

Business Success

Safety in the Community

Crowding

Tourism Development

Resident attitudes towards tourism

Pressure on Health and Social Services

Pressure on Police

Number of Complaints by local residents

Positive Participation in Community Activities

Change in Character of Local Community Such as 

Development of Local Community Groups

Maintenance of Cultural Heritage Through 

Enhancement of Attractions 

Resident perceptions of quality of life

Cultural Development: events (number and type)

Increased Awareness of Destination (Increased 

Visitation)

Increased Awareness of Destination ( New 

Investment/Business Opportunity in the Region) 

Change in Crime Patterns

Change in Social Problems (e.g., alcohol, drugs)

Malnutrition

Family Support

Education and Training
Program Evaluation of ARC’s Tourism, Page 58 

Cultural Heritage and Natural Asset-Related Projects

It is interesting to note, however, that to date no widely accepted method of measuring 

social capital in tourism has been developed. 

Environmental Indicators. Environmental indicators suggested by researchers are 

varied. Some analysts recommend measures that focus on energy use, water use, 

greenhouse gas emissions and ecological footprint. Others have generated a more 

complete list, for example, the most recent list generated by the World Tourism 

Organization (2004) who polled 62 experts from more than 20 countries, is the most 

comprehensive resource on this topic. Their list includes management of natural 

resources (waste, water, energy, etc.), climate change, visual impact of tourism and 

measuring the impact of tourism on the natural environment. A comparison of 

environmental indicators that have emerged from previous research are listed in Table 13. 

Table 13: Review of Environmental Indicators of Tourism Impact 

Theme Indicators

Tourism as a Contributor to Nature 

Conservation 

Measuring impact of tourism on natural 

environment 

Financing for biodiversity conservation and 

maintenance of protected areas

Site-specific regulations

Provision of opportunities for participation by 

tourists in conservation

Intensity of use

Managing scarce resources

Greenhouse gas emissions

Water availability and conservation

Limiting Environmental Impacts of 

Tourism Activity

Sewage treatment

Solid waste management

Water pollution

Air pollution

Controlling Noise Levels

Managing Visual impacts of Tourism

While Tables 5-8 provide a useful starting point for applying TBL to tourism, they do 

not, in most cases, provide the detail on actual measures that can be used in evaluation. 

For example, what does it mean to evaluate “water pollution” in this case? What specific 

metrics are available to truly judge a project’s impact on water pollution?

Based upon the previous literature review, a set of 115 items were adapted from 

Indicators of Sustainable Development for Tourism Destinations: A Guidebook (WTO 

2004). Of the 115 items adopted, 53 items were thought to measure the social bottom 

line; 23 items were thought to measure the environmental bottom line, and 39 items were 

thought to measure the economic bottom line. Additional detail on this process is found 

in Appendix F. 
Program Evaluation of ARC’s Tourism, Page 59 

Cultural Heritage and Natural Asset-Related Projects

After the items were identified, they were submitted to the Advisory Panel (Appendix A) 

to generate a smaller, more useful and appropriate set of indicators. The advisory 

members were provided with the following definitions for each of the TBL dimensions: 

The Economic Dimension: Refers to the financial 'revenue' shared by all commerce but 

also includes the additional economic benefits enjoyed by the host society. 

The Social Dimension: Sometimes called social capital or community well-being, refers 

to the human capital provided by employees, contractors, suppliers, and advisors as well 

as investment in the social and cultural systems that support an organization. 

The Environmental Dimension: Sometimes called natural capital, refers to the 

availability of natural resources upon which society’s existence depends.

The panel members were asked to place the 115 items into one of four categories, the 

economic dimension, the social dimension, the environmental dimension, and a fourth 

“none of the above” category. 

Combining the panel’s expertise with the information gleaned from the team’s research 

we ended up with 92 items in the scales, 20 that measure the economic dimension, 44 that 

measure the social dimension, and 28 that measure the environmental dimension. Tables 

14-16 present the final TBL indicators that resulted from this process. 

Returning to the case of “water quality” we see (Table 13) that water consumption, 

recapturing and reuse of water and sewage treatment are explicit metrics that can be 

measured.
Program Evaluation of ARC’s Tourism, Page 60 

Cultural Heritage and Natural Asset-Related Projects

Table 14: Final TBL Economic Indicators 

1

Degree to which the project impacts the number of people from outside the local 

community taking tourism jobs in the past year

2 

Degree to which the project impacts local community natural resource management 

programs 

3 Percent of jobs provided by the project that are less than six months per year

4 Degree to which the project impacts the local community via tourism taxes

5 Opportunities for promotion for employees of this tourism development project

6 Degree to which the project impacts low seasons

7 Degree to which the project impacts the local community's income

8 Percentage of jobs provided by the project that are full-time jobs

9 Total number of jobs provided by this tourism development project

10 Months per year the project operates 

11 Percent of jobs provided by the project that are full-year

12 Yearly revenue of this tourism development project

13 Asset value of this tourism development project

14 Employee retention rate of this tourism development project 

15 Average total local tourist spending at this tourism development project

16 Degree to which the project impacts employee family income

17 Degree to which the project impacts local employment opportunities

18 Per person fees charged for this tourism development project

19 Income levels of employees of this tourism development project

20 

Degree to which the project impacts the operation and support of micro, small, and 

medium sized enterprises (MSMEs)
Program Evaluation of ARC’s Tourism, Page 61 

Cultural Heritage and Natural Asset-Related Projects

Table 15: Final TBL Social Indicators 

1 Degree to which the project impacts local satisfaction with tourism

2 Degree to which the project impacts the number of tourism complaints by local residents

3 Degree to which the project impacts community social services

4 Degree to which the project impacts local community development programs

5 Degree to which the project impacts community participation rate in tourism

6 Degree to which the project impacts the local community's cultural values

7 Degree of local skilled labor for this tourism development project

8 Degree to which the project impacts the local community's physical security

9 Degree to which the project impacts the preservation of the local culture

10 Degree to which the project impacts the local community's educational programs 

11 Degree to which the project impacts residents continuing with local customs

12 Degree to which the project impacts residents continuing with local language

13 Degree to which the project impacts locals continuing with local music

14 Degree to which the project impacts locals continuing with local cuisine

15 Degree to which the project impacts cultural activities

16 Degree to which the project impacts affordable housing for local residents

17 Degree to which the project impacts the local community's health management programs

18

Degree to which the project impacts the number of local residents that left the community 

last year

19 Frequency of employee training programs for this tourism development project

20 Degree to which the project impacts net migration in/out of the local community

21 Degree to which the project impacts local community food security

22 

Degree to which the project impacts the number of complaints by community residents 

regarding tourism 

23 Degree to which the project impacts conserving local cultural sites

24 Degree to which the project impacts the local community in conserving local monuments

25

Degree to which the project impacts the local community by minimizing damage to local 

heritage sites

26

Degree to which the project impacts the maintenance and preservation of local 

cultural/heritage sites

27 Percentage of employees of the project that are qualified/certified/degreed

28 Degree to which the project impacts the local community's poor

29

Degree to which the project impacts the local community by enforcing local alcohol 

regulations

30 Degree to which the project impacts tourist behavior near children's play areas

31 Degree to which the project impacts local education programs on substance abuse

32 Degree to which the project impacts local health programs for substance abuse

33 Degree to which the project impacts childcare facilities for workers

34 Degree to which the project impacts family-friendly work shifts

35 Degree to which the project impacts training programs for local residents
Program Evaluation of ARC’s Tourism, Page 62 

Cultural Heritage and Natural Asset-Related Projects

Table 15 Final TBL Social Indicators, continued

36 Degree to which the project impacts work programs for local residents

37 Degree to which the project impacts the number of crimes involving tourists

38 Degree to which the project impacts pressure on local health services

39 Degree to which the project impacts pressure on police

40 Degree to which the project impacts the number of local community groups

41 Degree to which the project impacts local senior citizens

42 The degree to which the project impacts community sports programs

43 The degree to which the project impacts community festivals and events

44 The degree to which the project impacts community wellness centers
Program Evaluation of ARC’s Tourism, Page 63 

Cultural Heritage and Natural Asset-Related Projects

Table 16: Final TBL Environmental Indicators 

1 Degree to which the project impacts local effluent treatment facilities

2

Degree to which the project impacts the local community by supporting local noise 

regulations

3

Degree to which the project impacts the local community by supporting local congestion 

regulations

4 Degree to which the project impacts a smoke free environment

5 Degree to which the project impacts local community's ability to provide drinking water

6 Degree to which the project impacts local community conservation programs

7 Degree to which the project impacts suppliers' waste management activities

8 

Degree to which the project impacts the number of local environmental assessment 

projects

9 Degree to which the project impacts the number of local conservation projects

10 Degree to which the project impacts the number of local protected areas

11 Degree to which the project impacts local environmentally friendly modes of transport

12

Degree to which the project impacts local environmentally friendly waste disposal 

methods

13 Degree to which the project impacts local noise pollution

14

Degree to which the project impacts opportunities for tourists to practice conservation 

(recycling, etc.)

15 Degree to which the project impacts local ordinances to minimize environmental impacts

16 Degree to which the project impacts energy consumption

17 Degree to which the project impacts local use of renewable energy sources

18 Degree to which the project impacts local greenhouse gas emissions

19 Degree to which the project impacts local water consumption

20 Degree to which the project impacts local recapturing and reuse of water

21 Degree to which the project impacts sewage treatment in the local community

22 Degree to which the project impacts solid waste disposal in the local community

23 Degree to which the project impacts air pollution in the local community 

24 Degree to which the project impacts noise pollution in the local community

25 Degree to which the project impacts the local visual landscape

26 Degree to which the project impacts the loss of open land in the local community

27 Degree to which the project impacts public open spaces such as parks

28 Degree to which the project impacts suppliers' waste management activities

 
Program Evaluation of ARC’s Tourism, Page 64 

Cultural Heritage and Natural Asset-Related Projects

SECTION 7: THE ARC PROJECT PORTFOLIO & THE TBL

Section 2 above described analysis of the ARC funded projects based on an online survey 

of project managers and stakeholders. Here we will quickly re-look at the results from 

the standpoint of TBL implications. 

As we noted above, when asked about the impacts of the projects, most respondents 

reported moderate to strong impacts on tourism-related items: new tourism, cultural and 

visitor facilities. Tables 17 and 18 below reproduce parts of Tables 1 and 2 from Section 

2 highlighting impacts that are most relevant to the broader view of impacts practiced 

under TBL analysis. Note that we have removed some of the traditional economic 

measures that are a focus of the present ARC evaluation system. These measures are 

relevant in a TBL framework. 

From a TBL standpoint some projects affected enhancements in conservation, 

environmental infrastructure, open space, and water or energy conservation (Table 17). 

Table 18 reflects limited impacts on social and environmental dimensions such as 

poverty, health and pollution. As we noted above, these results are explained by two 

factors: project design did not include or consider environmental or social features; and 

ARC has other funding programs specifically targeted to infrastructure and other 

environmental issues. 

From both tables it is clear that the cultural assets are the most common impacts that fall 

squarely into the TBL framework. This is not surprising since many of the projects are 

based on the music, art, crafts, history and buildings – the cultural assets that provide the 

tourism draw. Conservation and visual landscape also are often affected as many projects 

improve the natural and built environment. 

Table 17: To what extent did the project result in the development, expansion, or 

enhancement of the following? 

Project Impacts

Moderate to Great 

Impact

No or Small 

Impact

Don't 

Know

Cultural facilities and events 51% 10% 2% 

Local conservation activities 22% 39% 10%

Amount of public open space 18% 53% 6% 

Environmentally-friendly transport 16% 61% 14%

Infrastructure (e.g., water & sewer) 8% 63% 6% 
Program Evaluation of ARC’s Tourism, Page 65 

Cultural Heritage and Natural Asset-Related Projects

Table 18: What impact did the project have on the following? 

Impact Category

Positive 

Impact

Slight 

Impact

No 

Impact

Don't 

Know

Preservation of cultural heritage 64% 21% 2% 13% 

Visual landscape 47% 11% 30% 13%

Population retention or growth 11% 30% 40% 19% 

Poverty reduction 9% 28% 47% 17%

Water conservation 6% 2% 70% 21% 

Air and water pollution reduction 6% 4% 66% 23%

Resident health 4% 6% 62% 28% 

Energy conservation 4% 15% 57% 23%

The interviews we conducted also revealed some aspects of TBL issues and impacts from 

the ARC projects. As noted in Section 2, some projects positively affected collaboration 

and environmentally friendly development. Case studies also reflected a basic 

understanding of TBL issues and positive and negative TBL issues within ARC projects. 

Clearly ARC project managers and stakeholders within their rural communities have 

some inherent understanding of the importance of including social and environmental 

dimensions and of broadening the evaluation of the economic dimension to more than 

just jobs. This suggests that there would be value in explicitly recognizing TBL as an 

appropriate method to evaluate ARC projects as well as other similar projects and 

initiatives. The question then becomes “How do we put a TBL perspective and 

evaluation system in place in rural America?”

ARC has recently funded projects that explicitly target TBL outcomes. Interestingly a 

project conducted by the Conservation Fund of North Carolina directly applies a 

straightforward method to implement TBL thinking into rural communities. The project, 

“Creating Asset-Based Economies in Western North Carolina” is described thusly: 

“TCF’s Resourceful Communities Program (RCP) proposed to expand the 

Creating New Economies Fund (CNEF), providing small grants of up to 

$15,000 to local agencies and/or nonprofit organizations for communitydriven, asset-based development initiatives that generate economic, social 

and environmental (“triple bottom line”) returns-on-investment.” 

The following section describes this model that has been applied in rural communities 

within North Carolina. 
Program Evaluation of ARC’s Tourism, Page 66 

Cultural Heritage and Natural Asset-Related Projects

SECTION 8: A MODEL FOR IMPLEMENTING TBL IN ARC 

PROJECTS

The Conservation Fund of North Carolina (The Fund) was awarded an ARC grant to 

build a program, “Creating Asset-Based Economies in Western North Carolina,” to 

provide small seed-money grants, up to $15,000, to community organizations in the ARC 

region of the state to develop programs and initiatives built on the framework of TBL. 

The Fund had received previous funds from other organizations to implement the 

program in other parts of the state. This means the system has been tested in two of the 

disadvantaged rural areas of America, Appalachia and the Southeast Crescent Black 

Belt1.

The project employs two basic strategies with the first of greatest interest in this 

discussion: 

1) Small grants to support “triple bottom line” projects that build a base of community 

support for entrepreneurship; and 

2) Asset-based lending program that provides technical assistance and loans for startup and expansion of natural resource-based businesses.

Strategy 1 is of greatest relevance here. The program has two features of direct interest 

to our purposes. 

The first is that it is intentional in nature; organizations must demonstrate that their 

proposed project integrates TBL as a basic operational goal. The process begins with 

the project’s mission and vision and then follows with goals, objectives, outputs and 

outcomes – and all reflect the commitment to TBL. At no time do the project leaders turn 

to each other and say “Okay now we do that TBL thing.” 

As we saw in Section 2, many ARC projects in the tourism portfolio had TBL 

implications and the project managers and stakeholders understood and implicitly hoped 

for outcomes that addressed social and environmental concerns. It is particularly 

interesting that many of the comments we heard reflected that social and environmental 

issues either helped or sometimes hindered the success of the project. In other words 

TBL was dragged into the process, not as forethought, but as an unforeseen driver of 

 

1 The Black Belt is a region of the southeastern United States. Although the term originally describes the 

prairies and dark soil of central Alabama and northeast Mississippi, it has long been used to describe a 

broad region in the American South characterized by a high percentage of African Americans. 

Communities in the Black Belt commonly face acute poverty, rural exodus, inadequate education 

programs, low educational attainment, poor health care, substandard housing, and high levels of crime and 

unemployment. While African American residents are disproportionately affected, these problems apply 

broadly to all ethnic groups in the Black Belt. There are various definitions of the region and its boundaries, 

but it is generally considered a band through the center of the Deep South, stretching from as far north as 

Delaware to as far west as eastern Texas. (http://en.wikipedia.org/wiki/Black Belt %28U.S. region%29, 

accessed August 24, 2009)
Program Evaluation of ARC’s Tourism, Page 67 

Cultural Heritage and Natural Asset-Related Projects

success or failure. The TBL was unavoidable even if it was not planned as part of the 

project’s scope. 

Second, the project requires and instills a collaborative, continual training and 

learning process that integrates evaluation. Grant applicants and recipients are 

required to attend workshops (the Fund pays basic travel and per diem expenses) in 

which they learn the basic elements of TBL as an explicitly integrated goal of action and 

go through a rigorous but straightforward group exercise that takes them through the 

steps of building an evaluation system. The evaluation system is instilled not simply as a 

reporting mechanism but as a fundamental part of the project. Evaluation is not seen as 

an add-on in this process. The evaluation process is used to drive stakeholder 

involvement, build excitement, guide action and help the organization recognize and 

correct problems that inevitably come up. Evaluation is a value-added instead of an 

unnecessary drain on time, money and resources. 

The required project workshops are part of the Fund’s Resourceful Communities 

Program2

. The workshops are built on a manual, Measuring a Movement: Evaluating 

Outcomes in Community Sustainable Development, which takes individuals through the 

complete process of developing and implementing their evaluation program (Gamble, et 

al, 2005). The process has seven steps outlined in the manual’s table of contents: 

Step 1: Getting started with participatory evaluation

Step 2: Define the outcomes you want to measure 

Step 3: Identify & select indicators to measure program outcomes

Step 4: Collecting indicator information

Step 5: Putting the evaluation process in place 

Step 6: Analyze and report your findings 

Step 7: Using your findings 

This makes clear that the organization must be intentional, participatory, and explicit 

about the evaluation. The workshop also instills the idea that TBL evaluation is a 

valuable management tool that actually increases project impacts and organizational 

effectiveness. The evaluation process is shown to be an upfront investment in time and 

resources that more than repays for itself during the project lifecycle. In that way it is 

completely consistent with the discussion in Section 5 of the bottom line impacts of TBL 

on private sector organizations. TBL as an organizational focus becomes an important 

management tool.

The Measuring a Movement process is not a panacea. It is designed for small 

community-based organizations. It is not a tool for companies though much of the 

material is relevant. (As noted above the project does include a lending program for 

commercial ventures and there are projects that build on both elements of the program.) 

 

2 http://www.resourcefulcommunities.org/
Program Evaluation of ARC’s Tourism, Page 68 

Cultural Heritage and Natural Asset-Related Projects

The community development model presented in the manual is instructive. At the top 

we see the Community Vision that explicitly includes community economic development, 

environmental conservation and social justice and equity. It also focuses attention on the 

fact that the process is not a step but is fully integrated where the Reflection element 

feeds back into a possible reexamination of the Community Vision. 

Operationalizing the process is completed using a logic model (shown on the following 

page) that takes the continuous circular model and provides the practical structure for 

implementation. It addresses the “who, what, how, and when?” questions as well as the 

fundamental “how did we do in the short, medium- and long-term?”

We spoke with the Fund staff to get a more nuanced view on how the process works. We 

asked whether it was difficult for the organizations to modify or push their proposed 

projects to mold them to the TBL framework. Unexpectedly the staff reported that not 

only were individuals able to quickly articulate TBL aspects and goals but that the give-
Program Evaluation of ARC’s Tourism, Page 69 

Cultural Heritage and Natural Asset-Related Projects

and-take within the workshop process helped others to think about how their projects 

could impact their communities in economic, social and environmental ways they had not 

realized.

One of the key parts of this program is that recipients are required (and funded) to return 

for workshops during the project funding cycle. The Fund staff report that these sessions 

become great methods to demonstrate what is working, what is problematic and how 

others have dealt with unforeseen issues. It becomes a collaborative learning process that 

builds and transfers expertise. 
Program Evaluation of ARC’s Tourism, Page 70 

Cultural Heritage and Natural Asset-Related Projects

RECOMMENDED NEXT STEPS

The Conservation Fund program is not the only way to operationalize TBL into 

organizations receiving grants but it does provide an excellent building block for thinking 

about next steps for ARC to develop an explicit platform for building Appalachian Triple 

Bottom Line initiatives. The Conservation Fund program involves a level of handholding, 

training and workshops that may be unrealistic for an ARC-wide implementation.

A program to integrate TBL perspectives into projects will require changes from the 

standpoint of both ARC and grant recipients. It will require that both ARC staff and 

project managers add expertise and knowledge about new measures and evaluation 

procedures. ARC staff and grantees will need to work together to agree on the 

appropriate goals and metrics that, in addition to the traditional employment, income and 

investment metrics, provide a means to understand projects more holistically. Lastly 

adding new measures and evaluation missions could add a additional administrative 

burden that takes away from the practicalities of getting a project off the ground and 

managing it to a successful result. These burdens will fall on both ARC staff and 

grantees. 

It should be noted that TBL impacts are often only realized over a longer-term time scale. 

While all involved realize that change often occurs slowly, the administrative and 

contractual system for projects is a stumbling block for a longer-term project view. ARC 

and grantees work under project open and closing schedules, reporting periods and 

financial agreements. It is unclear how integrating a longer-term perspective can be 

reconciled with the traditional management structure though we suggest options in 

Section 5 above. 

We believe a practical program can be built. The skeleton of the process is outlined in 

the seven steps described above. ARC does not need to be involved in all parts of the 

process but two elements will be critical:

An explicit acknowledgment by ARC of the relevance of TBL

ARC must be upfront in adopting the Strategic Plan’s suggestion that project success 

often goes beyond the traditional measures of employment, income and investment. 

While these traditional impacts provide the backbone for improving the lives and 

livelihoods of the people of Appalachia, they do not represent the complete picture.

Within this acknowledgment ARC will need to clearly define the why and how of the 

TBL. This might include the development of a series of straightforward background 

guidance documents or manuals. The guidance can be developed using much of the 

language and research found in this document as well the straightforward materials used 

by the Conservation Fund in its work. 

Communicating a set of potential impacts and measurement options

The surveys, interviews and case studies we conducted, along with the Conservation 

Fund project experience, clearly show that grantees implicitly understand that the work 
Program Evaluation of ARC’s Tourism, Page 71 

Cultural Heritage and Natural Asset-Related Projects

they do within Appalachian communities impacts and is impacted by economic, social 

and environmental circumstances. Most grantees are used to thinking in terms of jobs, 

income and investment. Not only must they be assured that broader goals are acceptable, 

but they will need assistance in making the transition to a TBL perspective. 

A key element is a “cheat sheet” of potential TBL impact measures and measurement 

methods for those impacts. Sections 7 and 8 of this report, the support materials in the 

appendices and the Conservation Fund’s Resourceful Communities Program can form the 

basis for this guidance. 

SUMMARY 

Adoption of a TBL perspective is clearly consistent with the strategic goals of ARC. 

Adding TBL to program evaluation could have significant impact on how projects are 

envisioned and conducted and broaden the view of grantees to include more holistic 

development options. Fully integrating social and environmental considerations into 

projects will not be without upfront costs but the benefits are likely to outweigh the costs 

and help build a stronger Appalachia. 
Program Evaluation of ARC’s Tourism, Page 72 

Cultural Heritage and Natural Asset-Related Projects

REFERENCES

Additional references are found in the documents in appendices F and G. 

Agenda 21, the Rio Declaration on Environment and Development, (1992). Available at: 

http://www.un.org/esa/sustdev/documents/agenda21/english/agenda21toc.htm (Accessed 

on January 29 2008) 

Benyus, Janine M. 1997. Biomimicry: Innovation Inspired by Nature. New York: Perennial.

Dwyer, L. (2005). Relevance of Triple Bottom Line Reporting to Achievement of 

Sustainable Tourism: A Scoping Study, Tourism Review International 9,79-93. 

Elkington, J. (2004). Enter the Triple Bottom Line. In A. Henriques and J. Richardson 

(Eds.) The Triple Bottom Line, Does It All Add Up? Assessing the Sustainability of Business and 

CSR, London: Earthscan Publications Ltd., 1-16 

__________ (1999), Triple Bottom Line Revolution -- Reporting for the Third 

Millennium, Australian CPA, 69(1), 75 - 77. 

__________ (1998). Cannibals with Forks: The Triple Bottom Line of 21st Century 

Business. New Society Publishers: Gabriola Island, BC and Stony Creek, CT 

Gamble, Dorothy, Marie Weil and Nicole Kiefer (2005). Measuring a Movement: 

Evaluating Outcomes in Community Sustainable Development. The Resourceful 

Program.

Global Rreporting Initiative (2008). 

http://www.globalreporting.org/ReportingFramework/G3Guidelines/. (Accessed January 

29 2008) 

Hawken, Paul, Amory Lovins, and L. Hunter Lovins. 1999. Natural Capitalism: Creating the 

Next Industrial Revolution. Boston: Little, Brown & Company.

Manning, E. W. and Dougherty T.D. ( 1995). Sustainable Tourism: Preserving the 

Golden Goose. Cornell Hotel and Restaurant Administration Quarterly, 36(2): 29-42. 

Marshall, J. D. and Toffel, M.W. (2005), Framing the Elusive Concept of Sustainability: 

A Sustainability Hierarchy, Environmental Science & Technology, 39 (3), 673-682. 

Miller, E., Buys, L. and Summerville, J. (2007), Quantifying the Social Dimension of 

Triple Bottom Line: Development of a Framework and Indicators to Assess the Social 

Impact of Organizations, International Journal of Business Governance and Ethics, 3 (3), 

223-237. 
Program Evaluation of ARC’s Tourism, Page 73 

Cultural Heritage and Natural Asset-Related Projects

Mohai, Paul. 1995. “The Demographics of Dumping Revisited: Examining the Impact of 

Alternate Methodologies in Environmental Justice Research.” Virginia Environmental Law 

Journal. 14: 615-653. 

Mol, Arthur P. J. and Gert Spaargaren.. 2000. “Ecological Modernization Theory in Debate: A 

Review.” Environmental Politics. 9 (1): 17-49. 

Norman, Wayne and Chris MacDonald. 2004. “Getting to the Bottom of “Triple Bottom 

Line””. Business Ethics Quarterly. 14(2): 243-262. 

Onyx, J. and Bullen, P. (2000) Measuring Social Capital in Five Communities. Journal 

of Applied Behavioral Science, 36(10): 23-42. 

_________ (2000). Social Capital Is Good Business. BHERT NEWS, Issue 9: 11-13. 

Pava, Moses L. 2007. “A Response to “Getting to the Bottom of ‘Triple Bottom Line’”. 

Rogers, M. (2003). Small Towns: Big Picture – Social Sustainability Indicators and the 

Arts. Social Dimensions of the Triple Bottom Line in Rural Australia. Bureau of Rural 

Sciences.

Sauvante, M. (2001). The ‘Triple Bottom Line: A Boardroom Guide. Director’s 

Monthly 25(11): 1-6. 

Schnaiberg, Allan and Kenneth Alan Gould. 1994. Environment and Society: The Enduring 

Conflict. New York: St. Martin’s Press.

Schnake, M. (1991). Organizational Citizenship: A Review, Proposed Model and 

Research Agenda. Human Relations, 44(7): 735-759. 

World Commission on Environment and Development (Brundtland Commission) (1987), 

Our Common Future, New York: Oxford University Press. 

World Tourism Organization (WTO) (1996). What Tourism Managers Need to Know: 

A Practical Guide to the Development and Use of Indicators of Sustainable Tourism. 

Madrid, Spain: WTO Publications.

_____________________________ (2004). Indicators of Sustainable Development for 

Tourism Destinations: A Guidebook. Madrid, Spain: WTO Publications. 
Appendix A.  Advisory Panel Members
Appendix A.  Advisory Panel Members
Program Evaluation of ARC’s Tourism, Page A-1

Cultural Heritage and Natural Asset-Related Projects 

APPENDIX A: ADVISORY PANEL MEMBERS

Bo Beaulieu

Director

Southern Rural Development Center

Meric S. Gertler

Goldring Chair in Canadian Studies

Department of Geography and Program 

in Planning

University of Toronto

Amy Glasmeier

Center for Public Policy Research in 

Environment, Energy, and Community 

Well-Being

Penn State University

June Holley

Consultant

David McGranahan

Senior Economist

U.S. Department of Agriculture

Deb Markley

Co-Director

Center for Rural Entrepreneurship

Jonathan Morgan

Assistant Professor of Public 

Administration and Government

University of North Carolina at Chapel 

Hill

Mikki Sager

North Carolina Representative

The Conservation Fund

Jesse White, Special Advisor

Director

Office of Economic and Business 

Development

UNC-Chapel Hill

Project Funders

Greg Bischak 

Chief Economist

Appalachian Regional Commission

Phone: 202-884-7790

gbischak@arc.gov

Wayne Fawbush

Program Officer

Ford Foundation

Phone: (212) 573-4918

w.fawbush@fordfound.org

Kostas Skordas

Appalachian Regional Commission

Phone: 202 884-7720

gbischak@arc.gov
Appendix B.  Online Survey Instrument
Thank you very much for participating in this survey. All of your responses will be kept confidential.

1. On a scale from 1 to 5—with 1 being not at all and 5 being completely—to what 

extent did the project achieve its goals?

2. Has the project been closed out by ARC?

*

*

1 Not at all 2 3 4 5 Completely

Yes No
3. Did the project result in the implementation of any new initiatives such as a 

program, organization, informational tool, or facility development or improvement?

*

Yes No
4. To what extent are these initiatives still in place or in use? Please use a scale of 1 

to 5, with 1 being not at all used and 5 being completely in use.

5. If these initiatives are not completely in place or in use, please explain

*

1 Not at all used 2 3 4 5 Completely in 

use
6. To what extent did the project result in the development, expansion, or 

enhancement of the following? 

*

Not at all A little Moderately A great deal Don't Know

Support for small 

businesses and 

entrepreneurs

Support for local 

agriculture

Education and training 

programs

Tourism attractions

Visitor facilities and 

services

Community facilities and 

services

Cultural facilities and 

events

Local conservation 

activities

Local environmental 

infrastructure (e.g., water 

& sewer)

Amount of public open 

space

Recreational facilities

Use of environmentallyfriendly modes of 

transport
7. What impact did the project have on the following?

8. In what way and how frequently have you measured these impacts?

9. If you had additional funds, how would you measure these impacts differently?

*

Negative Impact No Impact

Slightly Positive 

Impact

Moderately 

Positive impact

Strongly Positive 

Impact

Don't Know

Employment opportunities

Quality of jobs

Employment of highskilled labor

Tourism revenues

Sales of locally-produced 

items

Household incomes

Poverty reduction

Population retention or 

growth

Resident health

Preservation of local 

cultural heritage

Visual landscape

Energy conservation

Water conservation

Air and water pollution 

reduction

*

*
10. As part of this survey process, we would like to contact indviduals or 

organizations that were impacted by your work. In the following spaces, please 

identify and if possible, provide contact information for people outside your 

organization who either played a role in your project or were impacted by your 

work. For example, you could give the name of a local hotel operator who saw an 

increase in visitors or the name of a local community group that helped you publicize 

your organization.

1st stakeholder (name, 

organization and contact 

info)

2nd stakeholder (name, 

organization and contact 

info)

3rd stakeholder (name, 

organization and contact 

info)
11. The answers you have given in this project will be confidential. However, in order 

to categorize responses we are asking individuals to give their name and the name 

of the organization they are representing. Please do so in the following space.

*
Thank you very much for completing the survey. If you have any questions, please contact Chris Beacham or Dan 

Broun at Regional Technology Strategies, Inc., 919-933-6699
Appendix C.  Online Survey Analysis
ARC Survey

1. On a scale from 1 to 5—with 1 being not at all and 5 being completely—to what extent did the project achieve its goals?

Response

Percent

Response

Count

1 Not at all 0.0% 0

2 3.9% 2

3 9.8% 5

4 43.1% 22

5 Completely 43.1% 22

answered question 51

skipped question 0

2. Has the project been closed out by ARC?

Response

Percent

Response

Count

Yes 72.5% 37

No 27.5% 14

answered question 51

skipped question 0

3. Did the project result in the implementation of any new initiatives such as a program, organization, informational tool, or 

facility development or improvement?

Response

Percent

Response

Count

Yes 86.5% 32

N135% 5
4. To what extent are these initiatives still in place or in use? Please use a scale of 1 to 5, with 1 being not at all used and 5 

being completely in use.

Response

Percent

Response

Count

1 Not at all used 9.4% 3

2 12.5% 4

3 12.5% 4

4 21.9% 7

5 Completely in use 43.8% 14

answered question 32

skipped question 19

5. If these initiatives are not completely in place or in use, please explain

Response

Count

14

answered question 14

skipped question 37
6. To what extent did the project result in the development, expansion, or enhancement of the following? 

Not at all A little Moderately A great deal Don't Know

Response

Count

Support for small businesses and 

entrepreneurs

8.2% (4) 18.4% (9) 30.6% (15) 36.7% (18) 6.1% (3) 49

Support for local agriculture 53.1% (26) 16.3% (8) 12.2% (6) 8.2% (4) 10.2% (5) 49

Education and training programs 20.4% (10) 12.2% (6) 26.5% (13) 38.8% (19) 2.0% (1) 49

Tourism attractions 2.0% (1) 10.2% (5) 16.3% (8) 69.4% (34) 2.0% (1) 49

Visitor facilities and services 16.3% (8) 20.4% (10) 16.3% (8) 46.9% (23) 0.0% (0) 49

Community facilities and services 22.4% (11) 22.4% (11) 26.5% (13) 26.5% (13) 2.0% (1) 49

Cultural facilities and events 10.2% (5) 18.4% (9) 20.4% (10) 49.0% (24) 2.0% (1) 49

Local conservation activities 38.8% (19) 22.4% (11) 16.3% (8) 12.2% (6) 10.2% (5) 49

Local environmental infrastructure 

(e.g., water & sewer)

63.3% (31) 14.3% (7) 14.3% (7) 2.0% (1) 6.1% (3) 49

Amount of public open space 53.1% (26) 18.4% (9) 10.2% (5) 12.2% (6) 6.1% (3) 49

Recreational facilities 36.7% (18) 18.4% (9) 20.4% (10) 22.4% (11) 2.0% (1) 49

Use of environmentally-friendly 

modes of transport

61.2% (30) 14.3% (7) 8.2% (4) 2.0% (1) 14.3% (7) 49

answered question 49

skipped question 2
7. What impact did the project have on the following?

Negative 

Impact

No 

Impact

Slightly 

Positive 

Impact

Moderately 

Positive 

impact

Strongly 

Positive 

Impact

Don't 

Know

Response

Count

Employment opportunities 0.0% (0) 12.8% (6)

34.0% 

(16)

38.3% (18) 8.5% (4) 6.4% (3) 47

Quality of jobs 0.0% (0)

21.3% 

(10)

31.9% 

(15)

38.3% (18) 2.1% (1) 6.4% (3) 47

Employment of high-skilled labor 2.1% (1)

46.8% 

(22)

25.5% 

(12)

10.6% (5) 4.3% (2) 10.6% (5) 47

Tourism revenues 0.0% (0) 8.5% (4)

23.4% 

(11)

23.4% (11)

34.0% 

(16)

10.6% (5) 47

Sales of locally-produced items 0.0% (0) 17.0% (8)

29.8% 

(14)

23.4% (11) 19.1% (9) 10.6% (5) 47

Household incomes 0.0% (0)

34.0% 

(16)

34.0% 

(16)

19.1% (9) 0.0% (0) 12.8% (6) 47

Poverty reduction 0.0% (0)

46.8% 

(22)

27.7% 

(13)

8.5% (4) 0.0% (0) 17.0% (8) 47

Population retention or growth 0.0% (0)

40.4% 

(19)

29.8% 

(14)

10.6% (5) 0.0% (0) 19.1% (9) 47

Resident health 4.3% (2)

57.4% 

(27)

6.4% (3) 2.1% (1) 2.1% (1)

27.7% 

13)

47

Preservation of local cultural 

heritage

0.0% (0) 2.1% (1)

21.3% 

(10)

21.3% (10)

42.6% 

(20)

12.8% (6) 47

Visual landscape 2.1% (1)

27.7% 

(13)

10.6% (5) 36.2% (17) 10.6% (5) 12.8% (6) 47

Energy conservation 4.3% (2)

53.2% 

(25)

14.9% (7) 2.1% (1) 2.1% (1)

23.4% 

11)

47

Water conservation 4.3% (2)

66.0% 

(31)

2.1% (1) 4.3% (2) 2.1% (1)

21.3% 

10)

47
8. In what way and how frequently have you measured these impacts?

Response

Count

47

answered question 47

skipped question 4

9. If you had additional funds, how would you measure these impacts differently?

Response

Count

47

answered question 47

skipped question 4

10. As part of this survey process, we would like to contact indviduals or organizations that were impacted by your work. In the 

following spaces, please identify and if possible, provide contact information for people outside your organization who either 

played a role in your project or were impacted by your work. For example, you could give the name of a local hotel operator who 

saw an increase in visitors or the name of a local community group that helped you publicize your organization.

Response

Percent

Response

Count

1st stakeholder (name, 

organization and contact info)

100.0% 38

 2nd stakeholder (name, 

organization and contact info)

89.5% 34

 3rd stakeholder (name, 

organization and contact info)

71.1% 27

answered question 38

skipped question 13
11. The answers you have given in this project will be confidential. However, in order to categorize responses we are asking 

individuals to give their name and the name of the organization they are representing. Please do so in the following space.

Response

Count

46

answered question 46

skipped question 5
Appendix D. Interview Protocols for Program Managers and

Stakeholders
Program Evaluation of ARC’s Tourism, Appendix D-1

Cultural Heritage and Natural Asset-Related Projects

Questions for Individual Program Managers at ARC

We might we want to start by asking questions about specific case study candidates. We 

can forward a list to each of the managers so the can offer specific comments.

1. What are the most common goals of tourism related projects?

2. How are these goals measured?

3. What measures are grantees required to report? How do these compare to project 

goals?

4. What is the internal ability of most project recipients to do self-evaluation?

5. What would help you better evaluate the impact of these projects?

6. Are there any projects that you have worked with those incorporate triple-bottomline goals? 

Broader group discussion

1. What are the most common challenges that these types of projects face?

2. Are there any unique approaches to addressing to these challenges? Or any things 

that have been particularly effective?

3. What are the most realistic measures for these grantees to be attempting to gather?

4. What might be mechanisms that could be put in place to facilitate realistic 

collection?

5. Are there qualitative measures that could be used in a systematic way?

6. What are your thoughts about using the triple bottom line measures in these type 

of projects?
Program Evaluation of ARC’s Tourism, Appendix D-2

Cultural Heritage and Natural Asset-Related Projects

Questions for Stakeholders

1. What was your role in the project?

2. Were you involved in setting the goals for the project? What was your 

understanding of the goals of the project?

3. What has been the impact of the project on your organization?

4. How have you measured that impact? What do you think the regional impact has 

been?

5. If the project has closed, what has happened since the project?

6. What do you think were the biggest challenges facing the project’s 

implementation? What do you think were the biggest impediments to success?

7. Has the project been successful? How would you measure that success? How 

would you suggest the project’s outcomes be measured?

8. Has the project had any triple-bottom line impacts? 

9. Are there organizations in your region that cover these triple-bottom line 

elements?
Appendix E. Case Studies
Program Evaluation of ARC’s Tourism, Appendix E-1

Cultural Heritage and Natural Asset-Related Projects Draft

ATWOOD LAKE RESORT RENOVATIONS 

PROJECT DESCRIPTION

ARC awarded a grant of $200,000 to the Muskingum Watershed Conservancy District in 

2003 to pay for a portion of the renovation of the Atwood Lake Resort and Conference 

Center in Carroll County, Ohio. The resort, Carroll County’s largest private employer 

and only major lodging facility, has 123 lodging units, meeting space for up to 350, an 

18-hole golf course, and other recreational facilities. The resort current employs 50 yearround workers and 100 seasonal workers. 

The Conservancy District, the owner of the resort, is a political subdivision of the state of 

Ohio established in 1933 to create a flood-reduction system for the Muskingum River 

Watershed in southeastern Ohio. The district covers 18 counties, 16 of them in the 

Appalachian region. The development of the flood-reduction system resulted in the 

creation of 10 permanent reservoirs, including the Atwood Reservoir. In 1940, the 

Conservancy District added recreation to its mission. In addition to the resort, the 

District has developed a large number of recreational areas and facilities throughout the 

Muskingum Watershed, including parks, campgrounds, and marinas. While owned by 

the District, the resort is managed through a contract with a hotel management firm. 

The ARC funding supported three components of the project: 1) bringing the facility into 

compliance with the Americans with Disabilities Act (ADA), primarily by installing an 

elevator to provide access to guest rooms and meeting space; 2) installing new windows 

to make the facility more energy efficient; and 3) improving the facility’s public spaces. 

The total cost of these three components was $2.5 million. 

The total cost of the renovations was $4.5 million. The additional $2 million was used 

for improvements to the resort’s overnight guest accommodations and administrative 

offices. Other sources of funding included a $250,000 grant from the state of Ohio and 

$4.1 million in general revenue bonds issued by the Conservancy District. 

The projected outcome of the project was the retention of the resort’s 50 full-time jobs 

and the creation of 10 additional jobs over a five-year period. 

PROJECT IMPLEMENTATION

During the 35 years after the resort’s initial construction, the Conservancy District 

upgraded its water and sewer systems, acquired an adjacent golf course and other 

adjacent property, and invested in additional recreation facilities. However, it did not 

undertake any substantial renovations of the facility’s guest rooms, meeting rooms, or 

public spaces. By the early 1990s, the facility had become rundown, outdated, and 

inefficient. In addition, the facility was out of compliance with the Americans with 

Disabilities Act because its guest rooms and meeting facilities were not handicapped 

accessible. This was particularly detrimental to its meeting bookings. As a result, the 
Program Evaluation of ARC’s Tourism, Appendix E-2

Cultural Heritage and Natural Asset-Related Projects Draft

resort experienced steadily declining patronage during the 1990s and early 2000s. A 

consultant’s report commissioned by the District in 2001 concluded that under its current 

conditions, the resort could potentially operate for only another three to five years before 

facing the need to make serious operating decisions regarding number of employees and 

level of services offered. 

The District responded by developing a renovation plan. It planned to cover most of the 

costs by issuing its own bonds, but sought grant funding from other sources to reduce its 

debt burden. In 2001, it received a $250,000 appropriation from the state’s capital 

budget. Then, in 2002, it approached ARC for additional grant funding. ARC had 

provided funding for the resort twice in the past: in 1965, during the resort’s initial 

construction, for an access road; and, in 1975, for an upgrade to the resort’s water 

treatment plant. 

Before formally requesting ARC funding, the District submitted the proposal for review 

to the Ohio Mid-East Government Associates (OMEGA), the local development district 

that rates grant proposals to ARC, and briefed tourism, economic development, and local 

government officials in the region. OMEGA strongly recommended the project to ARC, 

and all of the 10 counties represented by OMEGA submitted letters of support. 

According to OMEGA’s current chair, while the organization does not usually 

recommend tourism projects, it did so in this case because of the wide support the project 

received and its important regional impact. ARC funding was approved in July 2003 and 

the renovations were completed by the end of that year. 

The ARC funding comprised less than 5 percent of project costs and was probably not 

absolutely essential to project completion. However, provision of grant funding did 

reduce the Conservancy District’s financial burden. The District received no tax 

revenues at that time and, thus, had to rely almost entirely on earned revenues to fund its 

operations and debt service. Its revenue consisted primarily of fees charged for use of its 

recreational facilities and income from land leases for natural-resource-based activities 

such as forestry, agriculture, and oil and gas extraction. Moreover, the resort had run an 

operating deficit for most of its history. The grant funding from ARC and the state 

enabled the District to somewhat reduce the level of bond financing required for the 

project, lowering its debt service and, thus, the annual subsidy required to keep the 

facility in operation. 

PROJECT OUTCOMES

Regional economic development and tourism officials agree that the renovations have 

greatly improved the appearance, comfort, and accessibility of the resort. This is borne 

out by significant growth in bookings since the renovations were completed. Annual 

room occupancy rates have climbed from 38 percent in 2004, the year after the 

renovations were completed, to a projected 42 percent in 2008 (based on current booking 

trends). Use of meeting facilities has also increased significantly. Catering revenues 

from meetings have grown steadily from $480,000 in 2004 to $625,000 in 2007, an 
Program Evaluation of ARC’s Tourism, Appendix E-3

Cultural Heritage and Natural Asset-Related Projects Draft

increase of 30 percent. The improvements to the meeting facilities have enabled the 

resort to increase bookings for small corporate meetings and family events such as 

weddings and reunions. This has included expanding its off-season business when 

occupancy rates have traditionally been very low. As a result, group revenues have 

actually surpassed transient revenue (from unaffiliated visitors). Major feeder cities for 

group business are Cleveland, Akron, Youngstown, and Pittsburgh. 

Despite these improvements, the resort continues to run an operating deficit, requiring 

ongoing subsidization by the Conservancy District. The resort’s manager estimates that 

occupancy rates would have to reach 50 percent for the resort to break even on an 

operating basis. 

The resort is clearly a critical source of support for the tourism economy in Carroll 

County and the surrounding region. As noted, it is the only full-service resort in the 

county. In fact, the county has only six other lodging facilities other than campgrounds—

five bed and breakfasts and a recently constructed 43-room limited service motel with no 

recreational amenities. Surrounding counties have additional lodging facilities, but few 

full-service resorts. Tourism is a major component of the region’s economic base and 

second only to agriculture in Carroll County according to local tourism and economic 

development officials. 

In addition to attracting visitors to the region, the resort is a primary contributor to the 

region’s tourism marketing infrastructure. According to the director of the Carroll 

County Convention and Visitors Bureau (CVB), the resort generates approximately 75 

percent of the county’s room tax revenue. Half of this, currently approximately $60,000, 

is allocated to the Convention and Visitors Bureau for tourism marketing. In addition, 

the resort has its own four-person marketing and sales staff and a marketing budget that 

exceeds the CVB’s. It is responsible for virtually all of the county’s marketing to groups. 

Tourism officials in surrounding counties recognize the importance of the resort as a 

regional tourism attraction because of the limited number of resort properties with 

comparable features in the region. While there is no formal regional tourism marketing 

organization, local tourism officials view tourism as regional in scope, with different 

parts of the region offering their own unique attractions. In this context, more guests at 

the resort translate into more visitors to surrounding counties. This is why local tourism 

organizations such as the convention and visitors bureau in adjoining Tuscarawas County 

include the resort in their marketing packages. 

The resort is also an important source of support for local government functions. The 

other half of the county room tax revenues it generates are allocated to the Village of 

Dellroy, where the resort is located, to fund village operations. 

The renovations did not result in increased employment at the resort. Current 

employment is 75 year-round workers, 43 of whom work full-time, and 75 seasonal 

workers, about the same as before the renovations. However, the renovations ensured the 

retention of workers who may have lost their jobs had they not been undertaken. While 
Program Evaluation of ARC’s Tourism, Appendix E-4

Cultural Heritage and Natural Asset-Related Projects Draft

the jobs are not high paying, they are relatively high quality compared to other available 

jobs in the area. There is little turnover of year-round employees. Seasonal summer jobs 

are an important source of employment for high school and college students, and even 

attract some school teachers and retirees. 

While the project did not have any explicit social or environmental objectives, it did 

appear to generate positive outcomes with regard to these two elements of the “triple 

bottom line.” In terms of environmental outcomes, the project has helped to preserve 

open space by maintaining the operations of the resort, a critical source of support for the 

region’s recreational tourism base. This has reduced pressures to promote more intensive 

forms of development. The Conservancy District is highly sensitive to environmental 

sustainability and has established policies that maintain the environmental quality of the 

land under its jurisdiction. It promotes resource-based uses and low-impact recreational 

activities. Economic development officials note that maintaining open space is consistent 

with the desires of the majority of area residents, who highly value the region’s rural 

character. 

In terms of social outcomes, the project has maintained and enhanced meeting facilities 

that are heavily used by local and regional organizations. There are no other comparable 

spaces within Carroll County and a limited number in surrounding areas. By providing 

space for business, civic, and social organizations to meet and conduct business, the 

resort helps to maintain the organizational capacity that builds social capital. 

MEASUREMENT ISSUES

It can reasonably be argued that the project led to the retention of all of the jobs at the 

resort, since the resort’s survival appeared problematic had the renovations not been 

undertaken. The role of ARC funding is less clear, however. Given ARC’s small 

financial role in the project, it is likely that the renovations would have proceeded 

without ARC support, although possibly with some reduction in scope. 

While the ARC funding application indicated that the project would result in the creation 

of ten new jobs, direct employment at the resort did not increase. No effort has been 

made to estimate indirect job creation or retention from the project. This could be done 

by calculating the number of visitor-days spent at the resort, applying available data on 

non-lodging visitor spending, and using an economic impact model to estimate total 

indirect spending and employment. Expenditures resulting from an increase in visitordays after the renovations would translate to new job creation. The state of Ohio has 

developed a model to estimate direct and indirect spending and employment resulting 

from tourism at the county level, and this model could likely be adapted to estimate the 

project’s indirect impacts. 
Program Evaluation of ARC’s Tourism, Appendix E-5

Cultural Heritage and Natural Asset-Related Projects Draft

BROOME COUNTY AGRI-TOURISM AND MARKETING 

PROJECT 

PROJECT DESCRIPTION

ARC awarded a grant of $24,206 to the Cornell Cooperative Extension in Broome 

County, New York, in 2003 for an initiative to strengthen the county’s agriculture 

industry through diversification and the development of agri-tourism. The total cost of 

the project was $96,697. The $72,491 local match included $68,991of in-kind support 

from the grantee and $3,500 from the Broome County Chamber of Commerce. 

The initiative had several components: 

1. Providing training and technical assistance to farmers in agri-tourism, marketing, 

product diversification, and business planning and promotion. This would include a 

six-week training program to be attended by an estimated 200 farmers and one-onone technical assistance to an estimated 40 farmers.

2. Forming three producer purchasing and/or marketing groups. 

3. Developing agri-tourism and direct-to-consumer marketing tools. This would include 

creating and distributing an agri-tourism marketing brochure, promoting tour 

packages to motor coach tour owners resulting in an increase from 10 to 25 tours per 

year, creating a countywide logo for agricultural products, and installing agri-tourism 

signage. 

4. Creating and distributing a five-year comprehensive agri-tourism development plan. 

The initiative was expected to result in an increase of 50 percent in direct sales from 

farmers to consumers, the expansion of 50 percent of existing agri-businesses to include 

agri-tourism and/or an alternative enterprise, the creation of 25 new agri-businesses, and 

the creation of 20 new jobs. 

PROJECT IMPLEMENTATION

The Cooperative Extension had been working on agricultural diversification efforts since 

the mid-1990s. A major catalyst was consolidation in the dairy industry, which had long 

dominated Broome County agriculture. As the scale of investment and complexity of 

management required for a dairy farm to achieve profitability increased, some dairy 

producers expanded their operations and some others sold their holdings and left the 

industry. Others who wanted to remain in agriculture began to explore opportunities to 

diversify into other agricultural products. 

The ARC-funded project evolved out of discussions between the Extension and owners

and operators of farm markets about the need to better cross-promote each other and 

develop agri-tourism routes within the county. Those involved also wanted to build 

stronger relationships with their peers.
Program Evaluation of ARC’s Tourism, Appendix E-6

Cultural Heritage and Natural Asset-Related Projects Draft

To help steer the project, a Direct Marketing Advisory Committee was formed and met 

periodically during the grant period. The committee, composed of agri-tourism business 

owners, members of the Broome County Farm Bureau, and members of the Broome 

County Chamber of Commerce, prioritized tasks for implementing the project. 

The grant period extended from September 2003 to January 2005. Most of the elements 

described in the initial work plan were completed, although some after the official closeout date of the grant. 

Training and Technical Assistance

The Extension offered 19 workshops between January 2004 and April 2005. Workshop 

topics covered various aspects of business development and management, direct 

marketing to consumers, and agri-tourism market development. One hundred forty-seven 

new or existing farmers attended one or more workshops, and total workshop attendance 

was 295. The workshops appeared to be well-received. Responses to evaluation forms 

distributed at some of the workshops were generally very positive.

Individualized technical assistance was provided to 139 new or existing farmers. This 

included help with analysis of their existing resources and the current direction of their 

businesses, and referrals to appropriate agencies for additional assistance with business 

planning. 

Weekly sessions were also held for farmers in the Extension’s computer lab. These 

included training in basic computer literacy and help using the Internet for product and 

market research. Approximately three to five farmers used the computer lab on a weekly 

basis. 

The Extension also initiated its quarterly Market Basket newsletter that provides news 

and information to agri-businesses about marketing events, programs, and resources. The 

newsletter was still in publication as of mid-2008. 

Formation of Purchasing and Marketing Groups 

The Extension supported the development of a number of purchasing and marketing 

groups. This included: 

• Establishment of a livestock producers group with the objective of bringing a USDAcertified slaughter facility to the area and jointly marketing value-added meat 

products. This work of this group led to receipt by the Extension of a grant from the 

New York State Department of Agriculture and Markets to conduct a feasibility study 

for the slaughterhouse. 

• Initiation of efforts to establish a farmers’ marketing association. 

• Formation by the Susquehanna chapter of the Northeast Organic Farmers’ 

Association of a purchasing group to study whether costs savings could be achieved 
Program Evaluation of ARC’s Tourism, Appendix E-7

Cultural Heritage and Natural Asset-Related Projects Draft

through cooperative purchasing of organic grains. The group determined that, in fact, 

no cost savings could be achieved. 

• Formalization of the management and operations of three farmers markets.

Development of Agri-tourism and Direct-to-Consumer Marketing Tools 

The Cooperative Extension worked with area farms, the Broome County Farm Bureau, 

and the Broome County Chamber of Commerce to develop an agri-tourism marketing 

brochure. The brochure included information on 36 local agri-businesses. With in-kind 

project funding from the Greater Binghamton Convention and Visitors Bureau (CVB), 

20,000 brochures were initially distributed at various visitor tourism welcome centers and 

hotels in the area. The brochure was later redesigned, three businesses added, and an 

additional 20,000 distributed using ARC funds. 

The Extension also worked with participating business owners, the New York State 

Department of Agriculture and Markets, and the Advisory Committee to create a logo 

identifying agricultural products grown in the Southern Tier. Research, however, 

indicated that there was an insufficient volume of direct-market agricultural products 

produced in the county to make a recognizable consumer brand. While not used for 

regional product branding, the logo was used for branding the Market Basket newsletter 

and on flyers and posters. 

The Extension held discussions with motor coach tour operators about including agritourism stops on their tour itineraries, but did not have much success in this regard. It 

learned that physical improvements would likely have to be made at any stops to 

accommodate motor coaches and large groups. 

Other market development activities completed by the Extension during the project 

included creating a website with the agri-tourism brochure, and helping to plan and 

execute marketing campaigns to promote the local farmers’ markets through displays, 

brochures, and PowerPoint presentations. 

Completion of Agri-tourism Development Plan

The Direct Marketing Advisory Committee completed a five-year agri-tourism 

development plan. The plan included recommendations for marketing and promotions, 

attraction development, and product diversification. 

The Role of ARC Funding

The Extension’s executive director and agricultural economic development specialist 

believe that the ARC funding was essential to completing the project work plan. It 

enabled the Extension to hire a part-time program assistant to provide additional staffing 

for the project. The agricultural economic development specialist, who directed the 

project, would not have been able to undertake many of the tasks included in the work 

plan given her existing workload. They also believe that the ARC-funded project laid the 
Program Evaluation of ARC’s Tourism, Appendix E-8

Cultural Heritage and Natural Asset-Related Projects Draft

groundwork for subsequent initiatives by demonstrating the potential for agri-tourism and 

direct-to-consumer marketing, and by enabling them to build relationships with business 

organizations and political leaders and leverage additional resources. 

PROJECT OUTCOMES

Agri-tourism and agri-marketing efforts in Broome County have continued and have 

gained momentum since completion of the project. 

Farm Friendly Broome County is one important initiative that emerged from the activities 

undertaken and relationships developed during the project. A collaboration of the 

Cooperative Extension, the Farm Bureau, U.S. Department of Agriculture, Broome 

County, and the Broome County Soil and Water Conservation District, Farm Friendly 

Broome County seeks to advance the visibility, sustainability, and profitability of all 

aspects of agriculture in the county. Its website, launched by the Extension in late 2007 

with input from its partners, seeks to bring together in one place a range of information 

about agri-marketing and agri-tourism. The website includes information on developing 

an agricultural enterprise, agri-tourism, buying locally, and local events, as well as links 

to related sites. It provides a user-friendly tool to identify vendors of local agricultural 

products, sorted by product category. 

In another new regional marketing effort, the Extension developed the Buy from the 

Backyard initiative, which encompasses Broome County and two neighboring counties. 

Buy from the Backyard’s website, targeted to local consumers, has a buyer’s guide that 

enables users to identify vendors of local agricultural products along with other 

information about buying and preparing these products. The initiative is also marketed 

through radio and TV advertising. 

Marketing efforts initiated during the project remain in place, and some have been 

expanded. The Greater Binghamton Convention and Visitors Bureau, after funding the 

agri-tourism marketing brochure, has expanded its agri-tourism marketing efforts. The 

CVB now has a webpage on agri-tourism with individual businesses listed. It has also 

added an agri-tourism section to its I Love New York Travel Guide, which can be 

downloaded from its website and is distributed at local welcome centers and in response 

to information requests. 

With the Extension’s assistance, a new regional farmers’ marketing association is in the 

final stages of development. The association will promote the development of farmers’ 

markets and other marketing channels for agri-businesses throughout the southern tier of 

New York and northern tier of Pennsylvania, and assist agri-businesses that want to 

diversify and increase their sales in local markets. The Extension has committed funding 

for launching the new organization. Efforts are also continuing to develop a USDAcertified slaughterhouse in the region, which would facilitate more local sales of valueadded meat products. 
Program Evaluation of ARC’s Tourism, Appendix E-9

Cultural Heritage and Natural Asset-Related Projects Draft

The Extension and the CVB have continued efforts to market agri-tourism to motor coach 

tours. The Extension is planning to add facilities to accommodate tours when it 

renovates its Cutler Botanic Garden, a current stop for a small number of tours. The 

Extension continues to conduct research on this market to determine other ways to attract 

additional business.

As local political officials and economic development professionals have become more 

aware of the economic benefits of agri-tourism and local purchasing of agricultural 

products, new efforts have emerged to more fully capture these benefits. In mid-2008, 

the county commissioned a feasibility study for an all-weather farmers’ market that 

would operate year-round, or most of the year, and would include space for more vendors 

as well as complementary activities. The county also was planning to develop a shareduse commercial kitchen for new food product development and, possibly, full-scale 

production of some products. 

In follow-up to the completion of the five-year agri-tourism development plan by the 

ARC project’s Direct Marketing Advisory Committee, the county’s Agricultural and 

Farmland Protection Board incorporated agri-tourism into the county-wide Agricultural 

and Farmland Protection Plan and identified it as among the highest priorities. In 2008, 

the Agricultural and Farmland Protection Board and the Cooperative Extension’s 

Agricultural Profitability Committee completed an updated plan centered on creating 

more cohesive relationships with the county and tourism offices, updating skills and 

education around marketing and agri-tourism issues, and creating product mixes and 

packaging attractive to tourists.

While the project did not have any explicit social or environmental objectives, it did 

appear to generate positive outcomes with regard to these two elements of the “triple 

bottom line.” In terms of environmental outcomes, farmers who are able to diversify 

their products, market more effectively, and increase their profitability are less likely to 

face pressure to sell their land for more intensive residential or commercial development. 

The county’s Farmland Protection Board recognized this by making agri-tourism 

development a high priority in its Farmland Protection Plan. In addition, increased local 

sales of local farm products lessen transportation requirements, thus reducing carbon 

emissions. Small-scale, local production may also make less intensive use of agricultural 

chemicals. 

The social dimensions of the project are less tangible but still present. The agricultural 

economy has tended to function in a separate sphere from other segments of the local 

economy, with its own set of relationships and organizational structures. The project and 

ensuing activities have brought the agricultural community together with other local 

economic stakeholders in collaborative efforts that have identified common interests and 

built relationships. The project has also built relationships, organizational structures, and 

leadership among non-traditional agricultural producers (e.g., through the marketing and 

production groups). This has increased their ability to work together to address common 

challenges and opportunities. Finally, the project has promoted interchanges between 
Program Evaluation of ARC’s Tourism, Appendix E-10

Cultural Heritage and Natural Asset-Related Projects Draft

farmers and urban consumers, helping consumers better understand the value of 

supporting local food production. 

 Looking to the future, there is a general consensus among agricultural, economic 

development, tourism, and political representatives that agri-tourism and local direct-toconsumer sales have strong growth potential and will contribute more to the local 

economy in years to come, particularly if the efforts that have been initiated in recent 

years are maintained and expanded. 

MEASUREMENT ISSUES

There is considerable evidence that the project and ensuing activities have helped local 

agri-businesses to increase sales to visitors and local consumers. In a random sampling 

of business listed in the agri-tourism guide conducted in mid-2005 by the Cooperative 

Extension, half of the businesses attributed an increase in sales to the guide. Most others 

reported uncertainty because they had not tried to track the guide’s impact on patronage. 

The growth in patronage of the farmers’ markets is another general indicator. Finally, the 

level of engagement and enthusiasm among agri-businesses themselves indicates that 

they see the value of these activities in their bottom lines. 

That being said, the Extension has only been able to estimate through observation the 

degree to which the economic impacts projected in the ARC funding proposal were 

achieved. It estimated 12 new agri-businesses formed as of mid-2008 compared to an 

initial goal of 25, and an increase of 15 agri-business jobs compared to an initial goal of 

20. It also estimated that 30 percent of existing agri-businesses have expanded to include 

agri-tourism and/or an alternative enterprise compared to an initial goal of 50 percent. 

These figures may well increase as efforts progress. The Extension has not been able to 

document increases in direct sales from farmers to consumers. One reason is that farmers 

are reluctant to reveal sales data. Another is that mechanisms have not been established 

to obtain data on the sales impacts of specific marketing tools. For example, information 

is not systematically collected from consumers about how they learned about a business 

and why they buy its products. A third reason is that the Extension does not have the 

staff in place to conduct surveys or other forms of data collection on an ongoing basis. 

The Extension is exploring the development of practical methods to obtain sales data, but 

it is uncertain whether data collection barriers can ever be fully addressed. 

Data collection issues aside, the Extension staff have concluded that it was unrealistic to 

expect the magnitude of impacts projected in the ARC funding proposal within the 

timeframe of the project. While they see evidence of progress, they note that the 

development of new agricultural products and markets is a process that takes place over a 

period of several years, not one or two. Therefore, even if effective data collection 

methods can be developed, the timeframe for assessing the economic impacts of this type 

of project must extend far beyond the end of the ARC grant period. 
Program Evaluation of ARC’s Tourism, Appendix E-11

Cultural Heritage and Natural Asset-Related Projects Draft

CERAMIC GUILD OF APPALACHIA

Appalachia is often characterized by its isolation. Communities lie separate from urban 

areas by difficult terrain and infrastructure too often neglected. But that physical 

isolation cannot prevent global economic changes from being felt in even the most 

remote parts of the Appalachian region. In the case of rural Ohio, global changes in the 

ceramics industry ultimately scuttled what had the potential to be an innovative program 

that sought to build social networks among ceramics companies and to begin to help 

them build a niche for their products.

PROJECT DESCRIPTION

ARC awarded Hocking Technical College in Nelsonville, Ohio a $225,000 grant in 2003 

to create the Ceramic Guild for Appalachian Ohio. The grant built upon an initial 

investment made by the Commission, which created a Ceramics museum that highlighted 

the history of the Ceramics industry in the region and showcased work by regional 

artists.

Southern Ohio had a long tradition of ceramics with the abundance of clay in the area 

making a natural for producing this product used both as decorative pottery and as 

industrial ceramics. Hocking College, as the leading educational institution in the area, 

has a vested interest in helping industry within its service area prosper.

The college approached ARC to fund the Guild with several goals in mind. First, it 

believed that it could bring together ceramic artists in the region with the larger ceramics 

companies. This would allow the companies to develop more creativity-based product 

lines and allow the region to develop a market niche. Second, the guild hoped to create 

educational programs that would present new techniques and technology around ceramics 

product that would allow businesses to more effectively compete. Third, the Guild, along 

with the Ceramics museum, would help “develop and promote the national heritage in 

ceramic art that has been perpetuated for over 100 years in the region.” This would 

happen through acquisition of more permanent pottery displays for the region and 

encourage more pottery shows that highlight local artists and companies. Fourth, the 

college and the guild would encourage entrepreneurship for potters in the area to ensure 

that the industry would continue to grow.

PROJECT IMPLEMENTATION

The first months of the project’s implementation forecasted great success for the effort. 

During the first fall, more than 75 students signed up to receive training in decorative 

course work. These skills were seen as the first step to making the potters more 

competitive nationally. As the program continued, project staff actively were engaged in 

recruiting new students for the training program, reaching out into local educational 

providers to show how the effort could help pave the way for advanced careers in 

ceramics. “I think the primary goal was the educational program,” said the program 
Program Evaluation of ARC’s Tourism, Appendix E-12

Cultural Heritage and Natural Asset-Related Projects Draft

manger during the final years of the program. “I visited local schools and I acted as a 

pottery recruiter,” instead of the army recruiter that one might expect to see on a campus.

The goals of the guild itself were to offer an alternative to companies who were facing 

increased competition. The purpose of the guild was to affiliate an artist group and 

cultivate it. Over the time it would provide art and design alternatives for these potteries. 

An artist who came up with a design could mass produce it.

Getting companies together in terms of the guild proved more difficult. Companies, who 

had been used to stealing employees and designs from each other, were understandably 

reluctant to get together to swap ideas about increasing their general competitiveness. 

And then the bottom fell out of the ceramics industry.

The reasons for the industry’s collapse are similar to the decline in other manufacturing 

sectors. Cheaper imported goods, primarily from China, caused users of ceramics to look 

elsewhere. In 2000, Muskingum County, where the museum was located, had seven 

establishments with a total of 297 employees. By 2007, there were only two 

establishments with a total of around 50 employees. The results were devastating—to the 

firms, to the employees, and ultimately to the project itself.

“There was a need for skilled technicians in pottery but then economy hit—they all went 

out of business,” said the owner of one of the few ceramics shops left standing.

With the shutdowns of participating and potentially participating firms, the guild lost 

momentum and ultimately ceased being operational. The museum’s training programs 

tailed off although the museum itself continues to be operational. It has a strong web 

presence and works to continue to promote the ceramics heritage of the region.

Hocking College, which sponsored the project, still sees great potential in assisting the 

ceramics industry. The project manager sees the college, with its tradition of advanced 

fine arts having the ability to 1)help local artists gain the business skills they need to get 

their products to market 2) to finish the work of getting the ceramics producers that 

remain the help in adding design to allow them to become more competitive. 

PROJECT OUTCOMES AND MEASUREMENT ISSUES

As the above indicates, the outcomes of the project were ultimately very limited due to 

circumstances beyond the control of the project. At least this is the case in job creation 

and retention. The global forces at work in the case of the ceramics industry were too 

great and ultimately too unexpected to allow the guild strategy a chance to work. Other 

work on networking and on incorporating to design into manufacturing suggests that it 

was not necessarily a misguided strategy rather that it needed to be implemented several 

years before. The chance to incorporate design into the ceramics products industry in a 

greater way is clearly the right approach—market forces simply overwhelmed the project.
Program Evaluation of ARC’s Tourism, Appendix E-13

Cultural Heritage and Natural Asset-Related Projects Draft

Measurement of numbers of individuals trained appears to be a much more appropriate 

way to evaluate the project’s direct impact. Project staff tracked numbers of individuals 

taking courses at the museum for which the ARC provided funding. As in the case of 

most of these projects, there is not follow-up done to see what if anything these 

individuals do with the training they receive. Instead, the program simply tracked 

whether or not they received training—follow up work has not been done to date.

The program did report an extremely positive experience in working with ARC around 

evaluation, a point that is worth emphasizing. After the project was closed, ARC staff 

visited on a validation visit. “More than the measurement activities, the validation visit 

was helpful to letting us know more about the impact of our work,” the director said. The 

validation visit helped the program director formulate new ideas on how to incorporate 

design into future college programs and how to meet the project’s original goals. 

 
Program Evaluation of ARC’s Tourism, Appendix E-14

Cultural Heritage and Natural Asset-Related Projects Draft

HERITAGE TRAILS IN SOUTHEAST TENNESSEE AND

NORTHWEST GEORGIA

In some communities, tourism efforts are not focused on just one locale with places, 

towns, counties and even states collaborating to create heritage trails. These trails allow 

motorists and other tourists who are interested in a particular subject area to travel to 

more than one location during their visit. In so, the tourist would extend their stay in a 

region and potentially produce more impact on a communities region.

This case study profiles two heritage trails funded through the Appalachian Regional 

Commission’s Tourism Program: 1) The Religious Heritage Trail in Southeast Tennessee 

and the 2) Chickamauga Campaign Heritage Trail in Northwestern Georgia. Both of 

these trails are excellent examples of this type of tourism activities both in terms of their 

ability to attract visitors and in the difficulty in evaluating an activity with multiple 

visitors.

PROJECT DESCRIPTION

ARC awarded a $40,000 grant to the Southeast Tennessee Development District in 

Chattanooga to create the religious trail, known as the Glory Land Road. The grants 

proceeds were to be used to develop signage for 20 sites in Southeast Tennessee and to 

create promotional materials, including a web site to encourage visitors to these sites. 

The Glory Land Road is the first trail in the US to seek to take advantage of tourist’s 

interest in religious heritage. The Glory Land Road is ecumenical in nature featuring 

such diverse stops as small rural churches, a synagogue in Chattanooga, a holocaust 

memorial, a large Pentecostal university, and the site of the famous “Scopes Monkey” 

Trial that debated evolution.

The Chickamauga Campaign Heritage Trail documents a more standard historical 

event—the battle that many believe is one of the turning points of the Civil War. The 

$50,000 grant from ARC awarded to the City of Chickamauga, Georgia includes support 

for signage, developing of a marketing plan and creation of a web site to promote the 

trail. The goal of the project was to position Chickamauga and surrounding communities 

as a viable tourist attraction to take advantage of the hundreds of thousands of tourists 

who visit the National Battlefield Park just outside the city and who might be encouraged 

to stay longer if they knew of more sites. The ultimate goal is to position the region as 

the “Gettysburg of the South.”

PROJECT IMPLEMENTATION

Both of the projects have been implemented with both national and regional fanfare. The 

religious heritage project in particular garnered attention from such national publications 

as USA Today, CNN and ABC News, mainly focusing on the unusual nature of a 

heritage trail devoted to spiritual sites. In both cases, the implementation efforts required 

significant on-the-ground cooperation between the grantees and the sites participating in 
Program Evaluation of ARC’s Tourism, Appendix E-15

Cultural Heritage and Natural Asset-Related Projects Draft

the trail. In order to be included in the religious heritage trail, sites had to agree to 

contribute a certain amount towards the cost of the permanent market placed on or near 

the locale. For some, coming up with the amount of funds—less than $1,000—was not 

difficult. In other cases, such as in the case of the one small church, it was more 

challenging. For instance, this church is a very small historically African-American 

congregation. However, people in the community banded together to help put together 

funds for the marker.

In the case of the Chickamauga Campaign Heritage Trail, implementation has garnered 

attention as part of the regional run up to the sesquicentennial of the Civil War. Making 

sure that the region’s sites are well equipped for the expected onslaught of visitors over 

the next few years is a goal shared by many in the region. Accordingly, the city of 

Chickamauga garnered support both financial and in implementation from the fifteen 

counties which the trail crosses. Each county contributed $1,500 to the effort that 

assisted in the full-scale ramp up of the project from marketing materials to the signs 

themselves.

In both cases, the trails implementation was facilitated by the substantial collaboration 

across jurisdictional lines. In the case of Chickamauga’s effort, working with counties 

across the region represented a new way of doing business. “It was the first time we had 

all worked together,,” said one county tourism director. “The effort was really the first 

time we had been all around the same table.” The fact that not only did individuals come 

together to discuss the effort but make a financial contribution has been critical to the 

project getting off the ground.

PROJECT OUTCOMES AND MEASUREMENT ISSUES

More so than perhaps any type of tourism project, analyzing the direct outcomes from a 

heritage trail is extremely difficult. While one can look at increases in numbers of visitors 

at individual sites, it is difficult if not impossible to directly attribute that to the existence 

of the trails themselves. Visitor data was not directly gathered in either the Religious 

Heritage Trail or the Chickamauga Campaign Trail that would suggest a direct impact on 

visitation as a result of the trail. In many cases this would have been impossible, many of 

the sites were just signs on the side of a road making things like site-based surveys or 

visitor logs impossible. Tracking of the trail use was primarily done through distribution 

of brochures or through visits to web sites promoting the trails.

There is anecdotal evidence, however, that the trails have increased visitation. In the 

case of Chickamauga, those involved in the project report that the trail is encouraging 

visitors to spend more time in the region beyond just visiting the Battlefield. “We want 

people to spend time to potentially track the experience their ancestors had going through 

the campaign,” one person said. “We want this to be a multi-day experience.”
Program Evaluation of ARC’s Tourism, Appendix E-16

Cultural Heritage and Natural Asset-Related Projects Draft

Since the trail was put in place a new hotel opened in Chickamauga catering to these 

multi-day visitors. In addition, a family owned Mexican restaurant is now opened in 

downtown Chickamauga. There is no absolute evidence that these establishments or just 

as importantly the jobs associated with them came directly from the implementation of 

the project. Those interviewed for the case study did believe that increased visitation and 

the resulting increase in visitor spending did indirectly result in increased employment in 

the region. Indeed, the original proposal suggested that the ARC investment would lead 

to the creation or retention of 100 jobs. While this figure may have been reached, any 

evidence towards this would be simply anecdotal as would the suggested $30-40 million 

impact that the project was supposed to generate.

The Glory Land Road’s impact is similarly hard to measure. The project had similar 

goals—“to keep people here for longer time instead of just a day.” Impacts have tended 

to be seen in the number of brochures used. “We couldn’t keep it on the shelf.”

Determining actual visitation is very difficult. For instance, it would be difficult to 

measure how many individuals visited the Prayer Plaza one of the stops on the Trail. The 

Plaza is a public space on the campus of the International Church of God in Cleveland, 

Tennessee. There is no facility that can keep track of visitors to the site so it would be 

difficult if not impossible to gauge visitation in that area.

The Glory Land Road’s Impact’s therefore like Chickamauga almost exclusively 

anecdotal. One of the greatest outcomes is the energy that banding together to raise 

awareness about the unique sites in Southeast Tennessee brought forth. For example the 

community of Charleston, Tennessee became so interested in being represented on the 

trail that people became interested in forming a historical society and are working 

towards increasing historical preservation in that small community. “Charleston has a 

new attitude about history,” said one program participant.

Another impact that may have come through the Glory Road is the emphasis on 

regionalism. Much like Chickamauga, the Religious Heritage Trail relied on many 

different communities coming together to create a joint planning effort. The fact that in

Southeast Tennessee most of the counties worked through the Southeast Tennessee 

Development Association made the effort easier but the extension of partnerships built 

through the Trail can be seen as a critical outcome.

In terms of job creation, it is difficult to attribute any direct impact on employment on the 

region as a result of the Glory Road. Again increased visitation to the area can be 

assumed to increase tourism spending which could increase employment at hotels and 

restaurants and other establishments catering to visitors. Estimating what percentage of 

those jobs is attributable to the trail specifically was beyond the ability of the granteee.
Program Evaluation of ARC’s Tourism, Appendix E-17

Cultural Heritage and Natural Asset-Related Projects Draft

TRIPLE BOTTOM LINE IMPACTS

The concept of heritage trails does bring into play some very interesting issues around the 

Triple Bottom Line. Certainly, anecdotal evidence suggests that the trails improved the 

economies of the regions that put the trails into place. However, as in the case of most 

tourism projects, the jobs that may have been created are in retail and are not generally 

that well paying. 

In terms of the environmental impact, one could make the case that the trails encourage 

driving which could have an adverse impact on the environment in these regions. In 

many cases, however, these trails may be traversed by buses and may not actually 

encourage new trips to the region in automobiles but instead promote the idea of 

staycations where individuals in the region opt to stay closer to home rather than driving 

long distances to out-of-state locations.

The most positive impact in terms of the Triple Bottom Line appears to have been in 

increasing the idea of social capital among region residents. The increases in 

collaboration among organizations and individuals can be seen as a direct result of these 

two projects. In both cases, individuals were brought together who had not collaborated 

on other projects. In rural areas where here are issues of scale it is critical that 

communities are able to band resources together. In the case of the Glory Road Trail, 

social interaction also went down to the community level, with individuals raising money 

and thus awareness about projects that were near and dear to them, encouraging 

involvement and empowering them in the economic development of their community.
Program Evaluation of ARC’s Tourism, Appendix E-18

Cultural Heritage and Natural Asset-Related Projects Draft

MOUNTAIN HERITAGE CRAFT INCUBATOR: BURNSVILLE, 

NORTH CAROLINA

ARC awarded $200,000 to the Yancey County Cultural Resources Commission (CRC), a 

county agency, though the Regional Council of Governments in Boone to convert an 

historic high school boys dorm (“the old Brown Dorm”) in Burnsville that was originally 

built in 1914 into an incubator for arts, craft, a recording studio, wholesale and retail 

showroom/store for traditional mountain crafts, e-commerce web site for artisans. It was 

to become a centerpiece for Burnsville’s “Heritage Circle.”

The stated goal of the project, which was matched by $223,500 in CDBG and state funds, 

was to generate at least 8 new or expanding businesses (resident craftsmen), stimulate or 

encourage at least 100 other local artisans, 30 increased sales, and leverage $443,500 in 

private investment within one year after the facility was completed. The target was to be 

local traditional folk artists, not those associated with Penland. In fact, there was a clear 

intent to focus only within the borders of Yancey County. 

CONTEXT

Yancey County, located in the western region of the state on the Tennessee border, is one 

of North Carolina’s poorest counties. Its population is less than 19,000, with about 1,800 

living in Burnsville, the county seat. Once dependent on traditional manufacturing, most 

of the plants have shut down and now sit empty at both ends of Burnsville. The 

population is officially less than two percent non-white although that would be greater if 

it included undocumented Latinos. In 2000, before the manufacturing left, the poverty 

rare was 15.8 percent and 13.7 percent received food stamps, both well above state 

average for the state and for all rural counties. Only 13 percent of residents have a 

baccalaureate degree. 

Although the county continues to pursue branch plants it also happens to be home to one 

of the highest concentrations of craftspeople in the U.S.—proclaimed to be the highest 

concentration by the Chamber of Commerce but without any hard evidence. The high 

concentration of artists is largely due to its Appalachian folk art heritage, its proximity to 

the world-renowned Penland School of Craft in the adjacent Mitchell County, and to the 

Celo Community, the largest intentional community in the state, which has a strong arts 

culture and has attracted a large number of artists from across the U.S. 

Due to distance to Asheville (38 miles), Mount Mitchell and the Blue Ridge Parkway, 

there has been increasing development of new expensive new home developments along 

Highways 19E and 80 that are being purchased largely as summer homes for Florida 

residents. The most established is a gated mountain top community with its own airstrip 

and golf courses. Although not really integrated into the community, they do create a 

potential market for art, crafts, and handcrafted architectural elements. The current 

recession, however, is threatening some of the new developments
Program Evaluation of ARC’s Tourism, Appendix E-19

Cultural Heritage and Natural Asset-Related Projects Draft

PROJECT IMPLEMENTATION

The building was completed and opened in October 2007 and is being used by a variety 

of artists. It sits in a hill overlooking the town across from the Parkway Playhouse, near a 

historic building they plan to turn into the new county library, and near a historic stone 

building now housing the school system offices. The long-term plan is to turn that area 

of town into a “cultural district” and initiate cultural events. 

The project has quickly altered its initial direction, away from the original plan of 

incubating new businesses to become a low-cost shared artists’ and arts business space 

and resource center. The building shortened its name from the original “Mountain 

Heritage Crafts Incubator” to the “Mountain Heritage Center.” It expects Yancey County 

to eventually become a major tourist destination and that developing and marketing a 

“Yancey County” brand is the answer, with the Mountain Heritage Center a significant 

piece of it. But it intends to develop this completely independent of its surrounding 

assets, including Penland, which is less than miles outside the county border. 

The facility has 12 small studios, an office for director1, a conference room, artists’ 

lounge, small show room, an attractive front porch with potential for holding small events 

or coffees. Nine studios rent for $100 per month, two slightly larger studios at $125, and 

the recording studio at $150. Limited parking is available on site but more nearby on the 

street. Although the building has no capability to house crafts that require kilns or ovens, 

there is one place in the basement that could possibly house a kiln. Yancey County pays 

the taxes, insurance, and director’s salary while the rental income covers utilities and 

upkeep. The Center has a strong 22-member Board if Directors that includes local 

officials, arts council, CC, local businesses. Its new chairman is a local developer. 

At the time of the initial visits in January 2008, seven offices were rented by: 

• a full-time photographer

• a part-time weaver who also manages an art gallery in town

• a part-time jeweler who expects to become full time 

• the owner of a B&B who does art restoration on the side

• the high school principal learning caning as a retirement craft, 

• the offices of the Parkway Playhouse

• a recording studio that moved there from a home-based studio in nearby 

Bakersville

Its most promising businesses were the photographer, who said sales had tripled sine 

moving from her home to the studio, and the recording studio. 

The facility also has meeting space used by the community plus a small gallery area. 

According to the Center director, in January 2008 initial goals were to:

• Rent all space but at higher rates

 1

 Jeanne Ray, 682-9654 Jeanne@yanceycountyCRC.com
Program Evaluation of ARC’s Tourism, Appendix E-20

Cultural Heritage and Natural Asset-Related Projects Draft

• Make the Center a community center and have at least one special event a month

• Develop a relationship with Mayland Community College and hold photography 

classes in the Center.

• Raise more financial support

• Establish a retail store for resident artists and others

• Develop stronger social atmosphere, including turning the front porch into a 

coffee shop 

• Attract more tourists and customers

The CRC received at least one related grant, from the Golden Leaf Foundation to create a 

Tile Center in Burnsville where they would train and hire local people to learn how to 

manufacture ceramic artistic tiles. The tile were displayed and sold at the MHC. They 

hired and later fired a marketing director and the only full-time employees were the 

project directors, who also was the original Chairman of the MHC Board. The Tile 

Center project was abandoned in 2008 and the facility is still looking for new owners. 

The Center also developed a web site (Yanceyarts.com) on a Golden Leaf Foundation 

grant that was turned over to the CRC.

The real breakthrough for the MHC as a cultural resource for the County came in 2008 

when the dedicated the recording studio at the Mountain Heritage Center to the County’s 

best known musicians, Leslie Riddle. Riddle, an African American musician in an 

almost entirely white county, had played with the Carter family and Cashes and helped 

shape country music. On February 22, 2008, the ribbon cutting took place with a special 

concert featuring Leslie Riddle’s friend, internationally known musician, and folklorist 

Mike Seeger. The Leslie Riddle Recording Studio was featured in an article in the 

January/February issue of Blue Ridge Magazine, citing the Mountain Heritage Center. A 

second “RiddleFest” highlighting his music and the recording studio is scheduled for 

February 21, 2009 at the town center, also featuring Mike Seeger. 

But the local newspaper was still describing the Center as “a self sustaining business 

incubator” with “apprenticeship training, small business incubation and trade-show 

exhibitions”2 although it was clearly becoming a cultural space and not an incubator. 

By the summer of 2008 all but one space were rented, by

• a full-time photographer

• a part-time traditional voice group

• a three quarter time writer

• half-time by a teacher/artist

• an office for the area’s Quilt Trail

• a full-time upholsterer and chair caner

• a part-time fine art restorer

• a mapmaker

• Parkway Playhouse office

• the Leslie Riddle Recording Studio

 2

 Yancey County Time, July 21 2008.
Program Evaluation of ARC’s Tourism, Appendix E-21

Cultural Heritage and Natural Asset-Related Projects Draft

OUTCOMES

The Center cannot be termed successful based on its original economic goals of 

developing and marketing local crafts. It has not incubated any business that has left the 

facility as a successful enterprise, and has not created any new jobs with the exception of 

the Center director. There is another very successful crafts incubator in the County called 

the EnergyXchange (built on a landfill and powered by its methane) that also has 

received ARC funds, but there is no connections between the two. 

The Center is not operating a crafts gallery (although it does display art), and it has not 

been on the semi-annual open studio tour of the Toe River Arts Council, which includes 

artists and galleries. The county has a well-established art community and marketing 

structures, but the one dedicated folk art gallery closed about five years earlier (not for 

lack of business). 

The project also has not raised the private sector match as proposed. Its connection to the 

Toe River Arts Council, located just blocks away, is weak. 

The MTC is, however a success as a cultural and community resource and attraction and 

a success in terms of its unstated social goals. It has a dedicated director intent on 

making it a success. It has continuing support from the town council. And plans are 

progressing, albeit slowly, on developing the rest of te Heritage Circle. 

The project also achieves its implicit social goals by serving the local community. 

Community organizations meet there regularly, Mayland Community College holds some 

continuing education classes in the Center in handbuilding with clay, video, video 

recording and editing. The visibility given to a native African American artist Leslie 

Riddle also helps overcome stereotypes and is a source of pride for the small African 

American population. 

The only environmental outcomes are that the project restores a historic building and 

does not consume many physical resources. 

BARRIERS TO GREATER IMPACT

The major barriers to long-term success are the unwillingness of the Board and city 

council to think or act regionally or collaboratively and largely artificially imposed (not 

recognized by craftspeople) distinctions between the “outsider” Penland-type artists and 

local folk artists. It based on a fear that non-local artists are negatively affecting local 

culture and believe that local crafts traditions have not been adequately valued because so 

many very good artisans have moved there from other regions. 
Program Evaluation of ARC’s Tourism, Appendix E-22

Cultural Heritage and Natural Asset-Related Projects Draft

The focus on Yancey County as a brand instead of the two-county area covered by the 

Toe River Arts Council that includes Penland School of Craft (which began as a way to 

develop local Appalachian crafts) restricts the market and arbitrarily bisects the cluster. 

Another barrier to the Heritage Center becoming a major tourist destination is lack of 

accommodations and restaurants. The County does not allow sales of alcoholic 

beverages, making it difficult to support high-end restaurants, and there are no high 

quality local or chain motels. Residents are more apt to go to Asheville for entertainment 

and culture than others are to come to Yancey County. Arts Caravans from Asheville 

have been tried, but without success/ 

The community college, which could be an important partner, has not shown strong 

support for crafts.

MEASUREMENT ISSUES

Because the goals changed form the original proposal, no data are being collected and no 

one on the county is evaluating impacts or outcomes other that space rented and direct

revenues generated. 
Program Evaluation of ARC’s Tourism, Appendix E-23

Cultural Heritage and Natural Asset-Related Projects Draft

CROOKED ROAD VIRGINIA PROJECTS

PROJECT DESCRIPTION

Within the ARC states Virginia has been most strategic in focusing on tourism, cultural 

heritage and natural asset-related projects. In particular the state has used the Crooked 

Road: Southwest Virginia’s Heritage Music Trail initiative to build a portfolio of assets, 

programs and marketing tools to build the region. Much of the funding has gone to 

projects that built on the music-related cultural heritage of the region. These projects 

have led to additional efforts to build a more diverse tourism portfolio using trails, 

farmer’s markets, artisan development and other tourist features.

Some of the projects that were used, directly or indirectly, to build the Crooked Road 

include:

Project Title

Amount -

ARC

Amount -

Total

Blue Ridge Music Trail $50,000 $107,000 

Ralph Stanley Museum & Visitor Center $500,000 $1,200,000 

Carter Family Memorial Music Center Expansion $500,000 $1,025,268 

Southwest VA Heritage Music Trail Network (3 

projects)

 $109,000 $170,965 

VA Crooked Road Music Project $45,000 $93,000 

William King Regional Arts Center $17,500 $31,250 

BCMA Cultural Heritage Center Planning & Design $50,000 $259,374 

William King Regional Arts Center $50,000 $100,000 

Tourist Information Center & Southwest VA Farmers" 

Market Expansion

 $100,000 $200,000 

Blue Ridge Music Center Interpretive Exhibits $500,000 $1,585,000 

The Crooked Road VA Heritage Music Trail 

Executive Director

 $41,400 $76,667 

William King Regional Arts Center - Artisans 

Courtyard

 $310,000 $1,693,785 

Blue Ridge Music Heritage Tourism Marketing 

Campaign, VA

 $50,000 $100,000 

Heart of Appalachia Heritage Tourism Marketing 

Campaign, VA

 $50,000 $75,000 

Appalachian VA Artisan Network $100,000 $185,185 

Total $2,472,900 $6,902,494 

Not all of these projects explicitly targeted the Crooked Road but they served to build a 

tourism portfolio where the whole was greater than the sum of the parts. The ARC 
Program Evaluation of ARC’s Tourism, Appendix E-24

Cultural Heritage and Natural Asset-Related Projects Draft

investment for the projects listed here leveraged a larger amount than the funding that 

ARC provided. Most of the projects were Virginia ARC funded projects but the last 

three were Commission funded projects.

PROJECT IMPLEMENTATION

As can be seen above, much of the Crooked Road focus has been on building physical 

assets to present the musical and cultural heritage of southwest Virginia. These assets 

provide the infrastructure to present music and cultural heritage items such as the Blue 

Ridge Music Center Interpretive Exhibits or the Birth of Country Music Cultural 

Heritage Center. These are the bedrock of the strategy that provides a base for a broader 

range of tourism developments. We describe some of the individual projects below based 

on our discussions with project managers and stakeholders within the communities.

The Virginia Artisan Network “project evolved from the Crooked Road project that was 

started in 2001. This was a regional tourism base initiative that highlighted the music in 

the area and elevated natural resources. All towns located along the Crooked Road have 

become cultural and artistic centers. Virginia Governor Warner in 2002 mandated an arts 

and crafts initiative akin to the Crooked Road that came to be called Round the 

Mountain: Southwest Virginia’s Artisan Network. This initiative serves artisans and 

crafters in Virginia by bringing tourist into the region. The efforts work in unison.”

The project was designed and implemented to be more than an “end destination.” It was 

designed to send tourists to crafter’s shops and studios and to be not just a place where 

sales are made but something to give tourists the desire to explore. The artisan network 

was seen as an alternative to the declining economic base and to highlight the region’s 

assets. The network has grown to include more than 240 artists and has generated jobs 

and income to the region. It has led to a greater focus on the creative economy and 

generated conferences and other high profile events. It has also led to a series of 

networked retail sales outlets.

From a TBL standpoint the project is seen as bringing a stronger sense of community 

with more interaction with artists and citizens. It has also created mutually beneficial 

relationships that have led to more agri-tourism and a stronger local food system.

The William King Regional Art Center Cultural Product Development project was 

designed to create another marketing component to complement the Crooked Road 

initiative. It was begun as a way of preserving some of regional artisanal products by 

providing a market for artifacts made using traditional techniques. These traditional 

techniques were in danger of being lost. The project identified 25 unique crafts products.

The project manager feels the project has been a great success, with high quality goods 

that present what was a dying way of life. They hope to increase both local as well as 

online sales. The products include both those typically produced by men such as wood 

products and quilting products produced by women.
Program Evaluation of ARC’s Tourism, Appendix E-25

Cultural Heritage and Natural Asset-Related Projects Draft

The Birth of Country Music Cultural Heritage Center was envisioned to document and 

build on the historical role of Bristol, Virginia and Tennessee on the development of 

country music in the early and middle decades of the 20th century. It was seen as an 

attraction that could build on the tourist influx from the Bristol Motor Speedway and 

broaden how people viewed the community. It has both tourism development and local 

community goals. With the construction of the Center, the Birth of Country Music 

Alliance and the local Chamber hope that visitors will see Bristol as a destination rather 

than a town near Bristol Motor Speedway. The interviewees also feel that the unique 

nature of this asset combined with the interest in country music nationally leaves Bristol 

poised to see impressive economic gains.

From a TBL standpoint the Center sees itself as providing a social connection between 

generations and a means to build community pride. The project manager noted that “We 

might serve as a destination and a starting point for a journey. The last exhibit we’re 

developing is about going down the Crooked Road. We have a moral obligation to let 

them know what the region offers. Our biggest challenge in this region is to get people 

here in the first place.”

He also noted, with regard to TBL impacts: “We want to show the interrelationships 

between the land, the culture and the people. Things like talking about the traditional 

instruments, wood from Gibson guitars for cabinetry. Those kind of relationships to the 

land, it is a part of our process. The building we’re in is a contributing structure to the 

historical district. We see a financial reward and doing the right thing at the same time.”

The Crab Orchard Museum Expansion partially funded by ARC was another project that 

was seen as symbiotic with the Crooked Road. The ARC funding was seen as critical as 

small museums like Crab Orchard have difficulty attracting foundation dollars. The 

project was designed to add features but also to increase other capabilities such as 

providing more office space and improving visitor services such as a gift shop. It is seen 

as increasing the numbers of tours, hosted functions (an asset for the local community) 

and improving the care of the artifacts. The improvements have led to more local 

income, more grants to artists and also increased donors improving long-term prospects. 

It has also led to 7,000 more annual visitors, added 9 jobs, increased overall impact on 

area's economy and expanded overall tourism related efforts.

From a TBL standpoint the project has “built new relationships and the project has 

boosted pride of people in the local community – they are very proud of the exhibit and 

they will bring family and guests” to the museum when they visit. “The Museum is 

improving the quality of life which is a key component to selling the area” and keeping 

residents and encouraging new residents to the area.

PROJECT OUTCOMES AND MEASUREMENT ISSUES

There are often un-measured spillover and synergy impacts from projects. This is 

particularly evident in the Crooked Road region of southwest Virginia where a series of 

related projects funded by ARC have helped create a creative cluster that builds on the 
Program Evaluation of ARC’s Tourism, Appendix E-26

Cultural Heritage and Natural Asset-Related Projects Draft

relationship between projects. The project discussions above provide a feel for the ways 

that the Crooked Road interacts with other economic development efforts within 

southwest Virginia.

To summarize we found that synergy elements included:

• Musical heritage brings tourists to region and employs musicians and other artists. 

Much of the impact comes from evening concerts that draw visitors and provides 

jobs and income.

• The Blue Ridge Parkway provides both a means for access as well as a tourist 

draw. Trips on the parkway are longer when there are additional activities off the 

parkway that lead to extended stays within the region. For example, the 

community in the town of Floyd has developed music, food and wine festivals at 

a venue right off the parkway that markets and brings tourists to the town. 

• Agri-tourism provides additional options for activities during the day, for example 

the Abingdon farmer’s market. In addition, the availability of local foods 

provides other cultural and environmental benefits for tourists who are drawn to 

restaurants that market the use of local foodstuffs including local wines.

• Development of artisan networks provides daytime shopping options, provides 

avenues for increasing artists’ income and serves to preserve much of the artisanal 

heritage of the region.

• Hiking and biking trails including the Virginia Creeper trail provide outdoor 

recreational options and increase the length of visitor stays.

• The development of Crooked Road projects had positive impacts on the local 

community. For example the museums provide additional opportunities for 

education and bringing students to the facilities. Facility space is used for 

training programs for farmers and others. 

• The projects have helped build a feeling of pride and connectedness to the 

community.

A recent study was completed of the Crooked Road portfolio of projects. Since ARC has 

provided funding for virtually all of the Crooked Road projects (mostly in tandem with 

other Federal, state, or local funds) it provides a test of the value of using a more strategic 

project funding that could be encouraged in other ARC states.

The 2008 study was conducted by Sustainable Development Consulting (SDC) and 

entitled Economic Impact Assessment of the Crooked Road: Virginia’s Heritage Music 

Trail. The analysis was based on surveys and other data sources that estimated visitation, 

dollars spent, length of stay and other direct impacts. These data were then examined 

using an economic input-output model that tracks how economic impacts flow through a 

regional economy.

They estimated that the Crooked Road Music Trail generated $22.8 million in annual 

revenue to the region and accounted for 445 full-time equivalent (FTE) jobs. How does 

this compare to the estimated impacts as reported by ARC grantees for the same portfolio 

of projects? The grantees reported that their projects generated 80 jobs or 18% of the 

estimated total reported by SDC. On the other hand the 445 FTE estimate is quite close 
Program Evaluation of ARC’s Tourism, Appendix E-27

Cultural Heritage and Natural Asset-Related Projects Draft

to the initial projections made by the ARC grantees of 416. The use of such a model is a 

needed tool to be able to fully document a strategic portfolio of projects where there is 

significant synergy between the various elements.

Clearly the Crooked Road Music Trail has, as designed, created a brand for southwest 

Virginia that can be used to market individual projects and assets as well as the overall 

region. This identity has had both internal impacts on how the region sees itself and on 

outside marketing of the full range of Crooked Road region activities. 

We recommend that ARC look into ways to encourage states to become strategic in their 

investments. The consistent and planned method of investment through a strategically 

designed program provides a model for using projects to build a stronger regional 

economy, brick by brick and asset by asset.

Project by project measurement of impacts serves only to examine the direct impacts of 

the project within the project area. Examining projects in a vacuum can underestimate 

the impact of an individual project. When one project increases the likelihood of 

lengthening stays within a region it has a multiplying impact on other projects. 

This is true not only for projects funded by ARC but for other developments outside of 

ARC’s project range of efforts. For example, the burgeoning wine industry plays an 

additional role in attracting tourists, lengthening stays and increasing the viability of the 

local and regional food systems. 
Program Evaluation of ARC’s Tourism, Appendix E-28

Cultural Heritage and Natural Asset-Related Projects Draft

CREATING ASSET-BASED ECONOMIES IN WESTERN 

NORTH CAROLINA

PROJECT DESCRIPTION

ARC awarded $200,000 to The Conservation Fund (TCF) to support economic 

development in Western North Carolina. To achieve this goal, TCF served as a fiduciary 

agent for a number of smaller grantees from Western North Carolina. This project is 

related to similar efforts that TCF is taking within other areas of North Carolina under 

their Creating New Economies Fund (CNEF). The project had an additional element that 

expanded an existing asset-based lending program. Within this case study we focus on 

the small grants program.

Unlike the other case studies within this report, this project was designed and conducted 

based explicitly on triple bottom line philosophies. All grantees were required to 

demonstrate a dedication to the triple bottom line from project application through 

execution. In addition to providing funding, TCF also provided technical support and 

training. Many of these grantees are small non-profits or agencies that would be unable 

to take the time to apply for and manage a full ARC grant. TCF describes the program:

“TCF’s Resourceful Communities Program (RCP) proposed to expand the 

Creating New Economies Fund (CNEF), providing small grants of up to 

$15,000 to local agencies and/or nonprofit organizations for communitydriven, asset-based development initiatives that generate economic, social 

and environmental (“triple bottom line”) returns-on-investment.” 

PROJECT IMPLEMENTATION

The TBL small grants program has two features that implement a TBL approach.

The first is that it is intentional in nature; organizations must demonstrate that their

proposed project integrates TBL as a basic operational goal. The process begins with 

the project’s mission and vision and then follows with goals, objectives, outputs and 

outcomes – and all reflect a commitment to TBL. 

Second, the project requires and instills a collaborative, continual training and 

learning process that integrates evaluation. Grant applicants and recipients are 

required to attend workshops (the Fund pays basic travel and per diem expenses) in 

which they learn the basic elements of TBL as an explicitly integrated goal of action and 

go through a rigorous but straightforward group exercise that takes them through the 

steps of building an evaluation system. 

The workshops stress that the organization must be intentional, participatory, and explicit 

about evaluation. The workshop also instills the idea that TBL evaluation is a valuable 

management tool that actually increases project impacts and organizational effectiveness. 
Program Evaluation of ARC’s Tourism, Appendix E-29

Cultural Heritage and Natural Asset-Related Projects Draft

One of the key parts of this program is that recipients are required (and funded) to return 

for workshops during the project funding cycle. The Fund staff report that these sessions 

become great methods to demonstrate what is working, what is problematic and how 

others have dealt with unforeseen issues. It becomes a collaborative learning process that 

builds and transfers expertise.

PROJECT OUTCOMES

Three of the grantees and their TBL impacts are profiled below. 

Stecoah Valley Food Ventures

The Stecoah Valley Arts, Crafts and Educational Center, which hosts the Stecoah Valley 

Food Ventures project, is a refurbished school facility including a gymnasium and school 

building. Initial grants from various regional and state sources covered reconstruction of 

the facilities. The $7,000 grant from TCF was intended to make the kitchen sustainable 

and “empower local growers and emerging food entrepreneurs across the four county 

area of far western North Carolina.” 

Economic: Jobs created: servers, kitchen manager, and maintenance. Cultural Heritage 

Tourism, products created in kitchen for sale, Appalachian Dinners, use of space as 

meeting place, reception hall.

Social: Education of community on heritage cooking, reinvestment in community 

specific assets, gathering place

Environmental: As we educate people about agriculture, we encourage people to 

preserve natural heritage. 

Growing Minds Growing Roots

Growing Minds, Growing Roots is part of the Appalachian Sustainable Agriculture 

Project and ties the schools in Henderson County, NC to area farms. Students engage in 

a number of activities both in school and at the farms to help them better understand food

systems, introduce them to fresh food and teach them basic cooking and food preparation 

skills. Farms establish relationships with school cafeterias, which then use the farms as 

sources of produce. The initial grant funding from CNEF was used to organize and 

facilitate a community meeting about Farm to School in Henderson County. The major 

outcomes from this meeting included stakeholder identification and stakeholder 

collaboration. 

Economic Outcomes: supplemental income for local farms. 

Social Outcomes: Collaboration among community groups, introducing children to local 

farms, connecting the community. Tying schools to the community. 

Environmental Outcomes: Sustaining local farms, community investing in natural 

environment, children appreciating use of natural resources
Program Evaluation of ARC’s Tourism, Appendix E-30

Cultural Heritage and Natural Asset-Related Projects Draft

Smoky Mountain Native Plants Association

The funds were used to establish an economic incubator. SMNPA focuses on native 

plant preservation and economic development, primarily through the harvesting, 

processing and marketing of native ramps. Ramps are a pungent plant grown in Western 

North Carolina that require special harvesting and processing. By using the facilities at 

the Stecoah Valley Arts, Crafts and Educational Center, SMNPA has created a lucrative 

local business that cannot be outsourced. 

Economic: 64 seasonal jobs, marketing and sales of ramp powder

Social: diverse group of people (age, race, length of residency)- supplemental income in 

a region with a very low median income, protected part of Cherokee heritage

Environmental: teaches sustainable harvesting techniques, processing techniques that 

waste very little of the ramp.

The program ultimately funded 37 projects, all of which included economic, social and 

environmental impact goals. A key impact finding is that the ARC funding generated 

leveraged funding of $1.28 million from foundations, the private sector and other federal 

sources.

According to TCF staff the typical projects have the following types of impacts:

Economic: 

• Development of entrepreneurial programs

• Focus on secondary incomes

• Focus on green technology businesses and workforce

Social:

• Target entrepreneurial programs on non-traditional populations such as 

immigrants and minorities

• Bringing minorities, woman, low income and hard-to-employ (e.g., welfare 

recipients) into the workforce

• Including non-traditional populations into community, business and workforce 

decision-making 

Environmental:

• Bring entrepreneurial skills to green businesses

• Target economic projects that are respectful of the environment such as 

sustainable agriculture and community gardens

• Focus on projects that serve environmental health issues such as sewer and water, 

lead prevention, healthy eating and energy efficiency

They found that grantees had the most difficulty in identifying the environmental aspects 

of projects and developing the measurement tools for the environmental parameters. 

Economic and social aspects were more straightforward to identify and measure. This 

finding has implications for ARC if it adopts a TBL focus.
Program Evaluation of ARC’s Tourism, Appendix E-31

Cultural Heritage and Natural Asset-Related Projects Draft

MEASUREMENT ISSUES

We spoke with the Fund staff to get a more nuanced view on how the process works. We 

asked whether it was difficult for the organizations to modify or push their proposed 

projects to mold them to the TBL framework. Unexpectedly the staff reported that not 

only were individuals able to quickly articulate TBL aspects and goals but that the giveand-take within the workshop process helped others to think about how their projects 

could impact their communities in economic, social and environmental ways they had not 

realized.

From the standpoint of ARC, a project of this type creates an additional layer through 

which impacts are measured and reported. None of the impacts are directly the result of 

TCF activities but of their grantees. ARC’s traditional oversight role remains with TCF 

and not with the ultimate grantees. This has both advantages and disadvantages. TCF 

has a close relationship with grantees, provides technical assistance and provides methods 

for collaboration between the grantees. It would be difficult for ARC staff to work 

directly with nearly 40 small grantees so the “out-sourcing” of administration and 

technical assistance appears to be an effective tool for ARC to make small grants to 

emerging organizations. Nevertheless, ARC must rely on TCF to accurately collect and 

report the impacts of the various organizations.

In addition, the TCF project has provided ARC with a low cost “pilot program” for a 

broader adoption of TBL with ARC. 
Appendix F. The Triple Bottom Line: Overview and Application to 

Tourism
The Triple Bottom Line: Overview and Application to Tourism

Dr. Michael R. Evans

Professor and Director of Hospitality and Tourism Management

The John A. Walker College of Business

Appalachian State University

828‐262‐6222

evansmr@appstate.edu

Dr. Carol E. Pollard

Associate Professor of Computer Information Systems

Executive Director of Center for Applied Research on Emerging Technologies

The John A. Walker College of Business

Appalachian State University

828‐262‐6185

pollardce@appstate.edu

Dr. James E. Stoddard

Associate Professor of Marketing

The John A. Walker College of Business

Appalachian State University

828‐262‐6191

Boone, NC 28608
Program Evaluation of ARC’s Tourism, Page F-2

Cultural Heritage and Natural Asset-Related Projects

TABLE OF CONTENTS

Introduction................................................................................ 3

Sustainability and Sustainable Development........................................ 4

The Concept of the Triple Bottom Line............................................... 6

     Triple Bottom Line Defined......................................................... 6

     The Economic Dimension of TBL................................................. 9  

     The Social Dimension of TBL...................................................... 9

     The Environmental Dimension of TBL........................................... 10

Critique of TBL........................................................................... 10

Linking TBL, Sustainability and Sustainable Tourism.............................. 13

     Sustainable Tourism Defined....................................................... 14

     Ecotourism............................................................................ 14

     Heritage/Cultural Tourism.......................................................... 14

     Agritourism............................................................................ 15

Developing Measures of TBL for Sustainable Tourism............................ 17

     Economic Indicators................................................................. 18

     Social Indicators..................................................................... 18

     Environmental Indicators........................................................... 19

Utility of TBL for Evaluation of Tourism Development Projects................. 20

Measurement Development Procedure................................................ 22

Preliminary Survey Design............................................................. 23

Assessment of the Results............................................................... 24

References................................................................................. 25

Table 1: Environmental Drivers of the Triple Bottom Line........................ 29

Table 2: Historical Pressure Waves Leading to the Development of TBL....... 30

Table 3: Review of Economic Indicators of Tourism Impact...................... 31

Table 4: Review of Social indicators of Tourism Impact......................... 32

Table 5: Review of Environmental Indicators of Tourism Impact................ 33

Table 6: Initial Triple Bottom Line Indicators....................................... 34

Table 7: Final Triple Bottom Line Indicators........................................ 38

Figure 1: Global Resource Initiative G3 Reporting Framework................... 41

Figure 2: Evaluation of Tourism Development Projects........................... 42

Appendix 1: Sample Survey of TBL Assessment of TD Projects................. 43
Program Evaluation of ARC’s Tourism, Page F-3

Cultural Heritage and Natural Asset-Related Projects

Introduction

Over the past few years, there has been growing interest in the need for organizations to be more 

sensitive to their corporate impact vis-à-vis the sustainability of environmental and human 

capital. The Brundtland Commission report (World Commission on Environment and 

Development, 1987) described sustainable development as being that which “meets the needs of 

the present without compromising the needs of future generations.” While there is no single 

agreed upon definition of the term “sustainable business,” for purposes of this paper a 

sustainable business is one that operates in an environmentally responsible way such that its 

products, services and business processes produce minimal negative environmental impact.

Building on the concept of sustainability, a new corporate philosophy and accounting form has 

emerged that takes into consideration not only the traditional economic “bottom line,” but also 

considers less quantifiable indicators that measure social and environmental impact. This new 

approach to measuring corporate performance is called the Triple Bottom Line (TBL) 

(Elkington, 1998). However, some organizations have presented themselves as being socially 

responsible through the use of various marketing and public relations techniques, but are, in 

reality, not necessarily “sustainable.” This half-hearted approach to corporate social 

responsibility has led to the emergence of proponents and opponents of TBL.

The purpose of this paper is to assess the usefulness of TBL as a tool to evaluate the 

sustainability of tourism projects. First, the idea of sustainability and sustainable development 

are presented. Next, the underlying concepts of TBL – both as a philosophy and as an accounting 

measure are presented with an examination of the pros and cons of TBL. Then, an overview of 

the emerging measures associated with the three dimensions of TBL: the economic dimension, 

the social dimension and the environmental is presented. Finally, TBL is integrated into the 

concept of “sustainable tourism” and a case is presented for TBL’s usefulness in evaluating the 

success and sustainability of tourism projects. The paper begins with a discussion of 

sustainability.

Sustainability and Sustainable Development

The concept of the triple bottom line originated from the notion of sustainability and sustainable 

development. Ecologically sustainable development (ESD) thinking was first espoused in the 

Brundtland Report (World Commission on Environment and Development, 1987) and reiterated 

during Agenda 21 and the Rio Declaration on Environment and Development (1992).

The Brundtland Report defined development as sustainable if it meets “...the needs and 

aspirations of the present without compromising the ability to meet those of the future (The 

concept of sustainable development, paragraph 1). Manning and Dougherty (1995) further 

clarified the term defining sustainable development as “the use of natural resources to support 

economic activity without compromising the environment’s carrying capacity, which is its ability 

to continue producing those economic goods and services (p. 30).”
Program Evaluation of ARC’s Tourism, Page F-4

Cultural Heritage and Natural Asset-Related Projects

One explanation for the increased attention to sustainability by organizations is provided by 

normative institutional theory. Normative institutional theory is a sociological perspective of 

organizational behavior which suggests that the “logic of appropriateness” guides the behavior of 

organizations. This “logic of appropriateness” means that organizations will react to 

environmental changes (such as community dissatisfaction or adversity) by initiating reforms 

(i.e., changing behaviors) and welcoming greater complexity (such as initiating more rules of 

conduct). In this way, the organization appears to be taking “appropriate actions” and as the 

marketplace becomes aware of these actions, benefits accrue to the organization (such as 

increased market share) as suggested by Kimmet & Boyd (2004).

In their review, Marshall and Toffel (2005) concluded that sustainability has been defined in 

many ways over the years. Based on that review, they see sustainability as falling along a 

continuum. At one end of this continuum (micro-sustainability), sustainability can be thought of 

as a business approach that creates long-term shareholder value by embracing opportunities and 

managing risks deriving from economic, environmental and social developments. The other end 

of the continuum (macro-sustainability), positions sustainability as a diverse set of societal goals 

such as poverty elimination and fair and transparent governance. Based on this rationale and 

Maslow’s Hierarchy of Needs, a sustainability hierarchy was developed. Like Maslow’s 

Hierarchy of Needs, the sustainability hierarchy levels build upon one another from the most 

pressing to the least pressing. The resulting four-level sustainability hierarchy includes 

sustainability of human survival (level 1 – the sustainability floor), sustainability of basic human 

health (level 2), sustainability of fundamental human rights and species survival (level 3) and 

sustainability of quality of life (level 4). The point of defining sustainability as a hierarchy was 

to show that virtually all organizational activities can be thought of as being sustainable (i.e., at 

least conforming to level 1), so it is important to identify the level of sustainability one is 

referring to when setting sustainability objectives. Their conclusion was that sustainability 

referred to:

“...transforming our ways of living to maximize the chances that environmental and social 

conditions will indefinitely support human security, well-being, and health.”

Accordingly, the degree to which an organizational endeavor is sustainable depends on a number 

of factors such as local environmental conditions, the local culture, the sustainability hierarchy 

level, and the spatial and temporal scale of the endeavor. The triple bottom line is one 

framework that organizations can implement to assess the degree to which their operations are 

sustainable.

The Concept of the Triple Bottom Line

Triple Bottom Line (TBL) is a term used to describe the economic, social and environmental

accountability of a firm. TBL is directly tied to the concepts and goals of sustainable 

development and is a relatively new measure of corporate performance that requires the public 

disclosure of social, economic and environmental indicators of organizational performance and is 

a concept that is closely related to “social responsibility.” A key element of TBL requires that 

corporate performance should be geared not only to benefit its “shareholders” but all of its 
Program Evaluation of ARC’s Tourism, Page F-5

Cultural Heritage and Natural Asset-Related Projects

“stakeholders” including groups such as the local community within which business operations 

are conducted.

TBL Defined. The term “Triple Bottom Line” evolved in the mid 1990’s when the consulting 

group AccountAbility coined the term which was later popularized by John Elkington (1998) in 

his book Cannibals With Forks: The Triple Bottom Line of 21st Century Business. The triple 

bottom line is a concept that expands the notion of organizational performance evaluation to 

include not only the traditional financial bottom line to one that also includes environmental 

quality and social justice (Elkington, 1999). Marshall and Toffel (2005) contend that the 

elements of the triple bottom line can be combined in pairs resulting in some of the popular 

terms used to describe sustainable projects. For example, eco-efficiency involves optimizing 

economic and environmental goals, fair trade is concerned with conducting economic activity 

with attention to social consequences, and environmental justice involves achieving social equity 

while respecting environmental protection.

It is important to note that TBL was intended to be a way of thinking about corporate social 

responsibility, not just an accounting methodology. Elkington (1994, 1998) stressed the 

integration of philosophy and accounting measures in implementing TBL and suggested that 

there are seven environmental drivers that have emerged to support its use (Table 1).

Table 2

Environmental Drivers of the Triple Bottom Line

Drivers Characteristics

Markets Unlike the past when business used competition as an excuse not to incur additional 

costs (for environmental and socially responsible behavior), future business will use 

TBL thinking to build a case for action and investment in environmental and social 

infrastructure.

Values Society’s changing values towards more environmental and social consciousness will 

influence corporate culture.

Transparency Organizations are facing growing international transparency about their environmental 

and social policies which is being facilitated by new societal value systems and radically 

different information technologies.

Life‐Cycle 

Technology

There is a new emphasis on cradle‐grave performance of products that examines 

product performance from extraction of raw materials through recycling or disposal.

Partners New forms of partnerships are evolving among business organizations and 

campaigning groups (e.g., Greenpeace).

Time Organizations are realizing that they must pay attention to long term rather than short 

term performance.

Corporate 

Governance

Top level organizational managers will be forced to pay attention to new issues such as 

the design of corporations and boards and their value chains of ‘business ecosystems.’

(adapted from Elkington, J. (2004) “Enter the Triple Bottom Line”)

As Table 1 suggests, first-rate TBL uses relevant, common indicators to make it easier to 

compare performance or “value added” across organizations and requires honest, open and 

transparent disclosure. Ideally, it has been suggested that TBL should lead to improvements in 
Program Evaluation of ARC’s Tourism, Page F-6

Cultural Heritage and Natural Asset-Related Projects

corporate performance. TBL reports should include key goals for improving organizational 

performance into the future – preferably quantitative and time-bound goals. The Global 

Reporting Initiative, a program developed under the auspices of nearly 20 agencies including the 

World Business Council for Sustainable Development, the United National Environment 

Program and the World Resources Institute, standardizes TBL measures and areas to be reported 

and currently has more than 1,300 participating organizations. The GRI suggests five categories 

of TBL indicators (GRI 2008). These include:

1.Economic performance

2.Environmental performance

3.Social performance

4.Labor practices and “decent” work

5.Public policies and implementation measures

The GRI also provides a reporting framework of how and what to report and measure (Figure 1).

Figure 1 – Global Resource Initiative G3 Reporting Framework

From an historical perspective, Elkington identified three “pressure waves” that led to the need 

for corporate sustainability and subsequently to the development of TBL (see Table 2).

Table 2

Historical Pressure Waves Leading to the Development of TBL

(Source: Elkington 2001)
Program Evaluation of ARC’s Tourism, Page F-7

Cultural Heritage and Natural Asset-Related Projects

Wave Time Characteristics

LIMITS Early 

60’s

Environmental impacts and natural resources demands have to be limited –

environmental protection

GREEN 1988 New kinds of production technologies and products are needed and have to 

be sustainable

GLOBALIZATION 1999 Growing recognition that sustainable development requires profound change 

in corporate governance and globalization increasing focus on government 

and on “civil society”.

Elkington proposed that these and one or two more pressure waves will result in a “Chrysalis 

Economy” whose key driver will be the “unsustainability of current patterns of wealth creation 

and distribution.” These unsustainable patterns of wealth creation generate worsening 

environmental and social problems which themselves motivate society to transition to 

sustainable development and will require environmental policies for different types of 

organizations.

In addition, new government policies and new government roles will be needed to change 

existing organizations into more flexible and fluid entities that will support sustainability. This 

will result in major “silo-busting” campaigns [where silos are systems (i.e., industries) that 

cannot integrate with other systems (e.g., society)] and will require a comprehensive approach 

that involves a wide range of stakeholders across many areas of government policy. Developing 

this comprehensive approach will be a “central governance challenge – and even more critically, 

a market challenge” (Elkington, 2001).

Although TBL was intended to be a philosophy – a way of doing business that is socially and 

environmentally responsible – many organizations have adopted TBL as an accounting method 

or a slick public relations campaign and have overlooked the need to establish and follow a 

corporate philosophy in keeping with its intentions. However, when TBL is embraced to its 

fullest extent, it forces organizations to engage with their full complement of external 

stakeholders (e.g., customers, vendors, non-profit activists) to discuss particularly “thorny issues 

and seek advice,” (Dahle, 2007).

One caveat to the aforementioned discussion, however, is that the philosophy of TLB is not well 

understood and needs to be better explicated through the media. For example, Vandenburg 

(2002) found a great deal of confusion in the organizations they surveyed on the topic of TBL. In 

their empirical study of 32 organizations in Australia, they found many different interpretations 

and understanding of the TBL definition and philosophy. Therefore, it is necessary to understand 

the multidimensional nature of the triple bottom line to fully appreciate the scope of its coverage. 

This section begins by defining and discussing its three dimensions.

The Economic Dimension of TBL 

Perhaps the easiest dimension to capture during triple bottom line evaluation is the economic 

dimension. The economic dimension can be assessed using traditional financial performance 

indicators such as sales revenue, profit, return on investment or shareholder value models. Other 

industry-specific measures are also readily available to assess the economic performance of an 
Program Evaluation of ARC’s Tourism, Page F-8

Cultural Heritage and Natural Asset-Related Projects

operation. For example, in the tourism industry, indicators such as heads in beds (for hotels) or 

visitation (for attractions) are popular measures. More difficult to assess are the social and 

environmental dimensions of the triple bottom line. These dimensions are discussed next.

The Social Dimension of TBL

It has been suggested that the social dimension of TBL, sometimes referred to as social capital

consists of two components: human capital (employees, contractors, suppliers and advisors) and 

investment by the social systems that support the business (Dwyer, 2005; Sauvante, 2001). A 

leading authority on social capital, Robert Putnam (1993) describes social capital as the “trust, 

norms and networks needed to facilitate cooperation.” Others such as Bullen and Onyx (1998) 

and Cox (1998) suggest that social capital is “created from a myriad of everyday interactions 

between people” and identify key dimensions such as valuing self and others; trust (interpersonal 

and generic); connection (community participation and social networks); multiple relationships; 

and reciprocity in relationships. Some social indicators suggested by the GRI include: employee 

retention rates; job satisfaction levels and investment per employee in illness and injury 

prevention (Goldsworthy, 2000). 

Miller, Buys and Summerville (2007) developed a framework and indicators to assess the social 

dimension of the TBL for the Australian dairy industry. In their view, the social dimension 

involves social well-being, working with employees, their families, the local community, and 

society to improve their quality of life. Organizations with social awareness better understand 

and account for the consequences of their operations on the social well-being of the communities 

affected by those operations. Their framework specifies four dimensions of an operation’s social 

impact, including an individual’s well-being, community well-being, employment experiences 

and satisfaction, and organizational impact.

The Environmental Dimension of TBL

The environmental dimension is referred to by some as natural capital. For example, Onyx and 

Bullen (2000) and Schnake (1991) suggest that the availability of natural resources such as 

forests, minerals, fish and soil have long been taken for granted despite the fact that this is the 

“capital” upon which our existence depends (i.e., clean air and water). As Onyx (2000) so 

succinctly put it “wealth that destroys the basis of life is no wealth at all.” Environmental 

indicators suggested by the GRI include items such as total energy use, use of recycled materials 

and water sources significantly affected by an organization’s use of water.

Critique of TBL

TBL is regarded by some as a “useful tool for integrating sustainability into the business agenda” 

(McDonough and Braungart, 2002, p. 251). However, it focuses principally on controlling for the 

bad effects and not necessarily enhancing the positive effects of products. McDonough and 

Braungart (2002) proposed a focus on the triple TOP line where the emphasis is on increasing 

the good effects and thus, by definition, eliminating the bad.
Program Evaluation of ARC’s Tourism, Page F-9

Cultural Heritage and Natural Asset-Related Projects

On the other hand, others are skeptical about the virtues of TBL. For example, Vanclay (2004) 

criticized TBL proponents and through that they had much to learn from the social impact 

assessment proponents. He pointed out that while the economic and environmental indicators of 

TBL are easy to identify, the social indicators are difficult to assess.

Norman and MacDonald (2003) also criticized TBL. Their criticisms can be divided into those 

that (1) center on whether there are indeed additional bottom lines that can be measured as 

assessed (i.e., beyond the economic bottom line) and if they can, (2) whether organizations 

should measure and report those additional bottom lines.

For brevity’s sake, Norman and MacDonald (2003) discussed their criticisms with respect to the 

social dimension, although they claim that the same arguments can be made for the 

environmental dimension. Regarding whether or not additional bottom lines can be measured 

and assessed (point 1) they invoke their “measurement claim” and their “aggregation claim.”

The argument was that neither the social nor the environmental bottom lines can be measured in 

objective ways (as can the economic bottom line) (i.e., the measurement claim) and therefore, it 

is not possible to compute something similar to a social or environmental net profit (i.e., the 

aggregation claim). In essence then, there is no common currency to allow for the expression of 

good and bad magnitudes with respect to the social and environmental bottom lines.

Their second argument addresses the issue of why an organization should measure, assess and 

report its additional bottom lines (point 2). Here they raise their “convergence claim” (i.e., that 

measuring and improving social and environmental performance tends to be more profitable in 

the long run), their “strong social-obligation claim” (i.e., that firms have an obligation to 

maximize their social and environmental lines) and their “transparency claims” (i.e., that 

organizations have an obligation to stakeholders to disclose information about their performance 

on their social and environmental bottom lines. Essentially, their argument is that these issues are 

nothing new (they are corporate social responsibility) and have been incorporated into 

organizational strategic plans for years; however, maintaining the financial bottom line (e.g., 

shareholder value) is the only thing that really counts.

In addition to the aforementioned criticisms, Norman and MacDonald (2003) suggest that on 

both a conceptual and practical level the language of TBL promises more than it can ever 

deliver. For example, organizations may believe that by committing themselves to the principle 

of TBL, they are making a verifiable commitment to sustainability. However, TBL reporting 

gives and organization an opportunity to display only their clean laundry in public. On the flip 

side, it also allows firms not to provide data that could cause public embarrassment.

On the other hand, Pava (2007) responded to the criticisms of Norman and MacDonald (2003) by 

arguing that their “aggregation claim” was false. In essence, Pava (2007) suggested 

organizations that support the use of triple bottom line metrics do not have to assess their social 

and environmental bottom lines the same way that their financial or economic bottom lines are 

assessed. In addition, Pava (2007) pointed out that even the popular net income aggregate 

measure of financial performance was deemed a deficient indicator of financial performance 

since return on investment drew on more sources of information (i.e., the income statement and 
Program Evaluation of ARC’s Tourism, Page F-10

Cultural Heritage and Natural Asset-Related Projects

the balance sheet). Since financial performance cannot be summarized with a single objective 

number, one should not expect to assess social and environmental performance in this manner. 

Finally, Pava (2007) reminded the reader that organizational performance is multidimensional.

To summarize, some suggest that TBL is nothing new. However, TBL thinking may produce 

some useful and dependable indicators of societal and environmental indicators of “value added” 

in relation to corporate responsibility and accountability. Furthermore, an organization’s 

motivation to use TBL might also include its attempt to differentiate itself from its competition 

to attract investors and customers who might be willing to pay more to support ethical firms.

Clearly, the main challenge associated with TBL is measurement. The social and environmental 

measurement and comparability of organizations and industrial sectors over time is proving to be 

a very difficult task. As Norman and MacDonald (2003) point out, it is commonly understood 

that in modern management theory; “if you can’t measure it, you can’t manage it.” Current 

TBL measurement efforts are time-consuming, vague, badly misleading (i.e., public relations 

rhetoric) and provide a smokescreen for poor performance. This raises the issue of industrial 

sector variables. This is one of the components of “what to report” discussed in the Global 

Resource Initiative G3 Reporting Framework. It suggests there are unique sets of sustainability 

issues faced by different industrial sectors such as the automotive, banking and tourism sectors, 

among others. For these reasons, to use TBL to evaluate sustainable tourism projects, it is 

necessary to explore TBL in the particular context of sustainable tourism and the corresponding 

variables of interest. The next section ties together TBL with sustainable tourism.

Linking TBL, Sustainability and Sustainable Tourism

The concepts of TBL, sustainability, sustainable development and sustainable tourism are closely 

linked. As Manning and Dougherty (1995) suggest, sustainable tourism is the best way to 

preserve the “golden goose” of tourism. They feel it is a viable tool to maintain and enhance a 

destination’s competitiveness.

Similar to other types of industries, Isaksson and Garvare (2003) identified sustainable tourism 

and development as that which finds a balance between economic prosperity, environmental 

protection and social equity. While there is still some debate surrounding a clear definition of 

sustainable tourism, there is wide agreement that it should facilitate the social, economic and 

environmental well-being of a region (Gilmore and Simmons, 2007).

Sustainable Tourism Defined. For the purpose of this paper, sustainable tourism refers to a level 

of tourism activity that can be maintained over the long term because it results in a net benefit for 

the social, economic, natural and cultural environments of the area in which it takes place.

For now, the term sustainable tourism (encouraged by both the WTO & UN) is used as an 

umbrella concept, under which terms, such as eco-tourism, heritage and cultural tourism, geotourism, may fall. These categories are discussed next.

Ecotourism. In a review of the literature, Hvenegaard (1994) identified ecotourism as 

“responsible travel to natural areas which conserves the environment and improves the welfare 
Program Evaluation of ARC’s Tourism, Page F-11

Cultural Heritage and Natural Asset-Related Projects

of local people (p. 25).” Sometimes referred to as ‘nature tourism,’ ecotourism is nature-based 

tourism designed to minimize ecological impact to a tourism area and includes amenities such as 

landscape sightseeing, bicycling, hiking, canoeing, etc. (Mehmetoglu, 2007).

Heritage/Cultural Tourism. Various definitions have been offered for heritage/cultural tourism. 

For example, Caton and Santos (2007) see heritage tourism as tourism that engages with the 

cultural tradition of a particular location. Included in heritage and cultural tourism are tangible 

remains of the past such as artifacts, as well as intangible cultural assets such as folk traditions. 

Others have identified cultural tourism as historical and heritage sites, arts and craft fairs and 

festivals, museums, and the performing and visual arts which tourists visit in pursuit of their 

cultural experiences (Hall and Zeppel, 1990). Kerstetter, Confer and Bricker (1998) identified 

heritage tourism as being about the cultural traditions, places and values that groups around the 

world are proud to preserve. Included are such traditions as family patterns, religious practices, 

folklore traditions, arts and crafts, ethno-history, and social customs. Heritage/cultural tourism 

provides visitors the opportunity to understand and appreciate the character of an area and 

includes its history, archeology, people and their lifestyle, cultural diversity, arts and 

architecture, and social and political structure.

Agritourism. Finally, adopting a definition from Weaver and Fennell (1997), McGhee (2007), 

and McGhee and Kim (2004) identified agritourism as rural enterprises that incorporate both a 

working farm and commercial tourism component. Agritourism is based on attracting visitors to 

farm operations including crop and animal farms, U-pick operations, wineries, for-fee fishing 

operations, Christmas tree farms, herb farms and greenhouses, maple syrup and cheese 

producers, and farm stands.

In discussing sustainable tourism, Lansing and De Vries (2006) seem to be in the same “TBL 

critical camp” as Norman and MacDonald (2003) and further suggest that sustainable tourism is 

only a marketing tool to attract morally conscious tourists. Despite the apparent evidence for the 

positive effects of sustainable tourism, they also argue that the concept is vaguely defined, 

misused and lacks real content, thus reducing it to nothing more than a clever marketing ploy. 

Milne, Ball and Gray (20) also suggest that many organizations seem to confuse narrow and 

often incomplete reporting practices with claims of moving towards sustainability. They suggest 

the current TBL practice leads to a dangerous confusion and a “change-but-no-change policy.”

They also point out the need for more clarity to make sustainable tourism a practical reality as 

well as the need for a governance instrument (like a rating system). This might include a private 

sector approach to integrate sustainability governance with existing ratings such as AAA 

Diamonds, Mobile Stars and Michelin Stars.

According to the National Geographic Center for Sustainable Destinations (National Geographic, 

2008), sustainable and geo-tourism (do no harm policies and practices) seek to avoid the “loved 

to death” syndrome and develop wise destination stewardship. In principle, business, government 

and other stakeholders can anticipate development pressures and apply limits and management 

that sustain natural resources, cultural/heritage resources, and balanced quality of life issues for 

local residents.
Program Evaluation of ARC’s Tourism, Page F-12

Cultural Heritage and Natural Asset-Related Projects

The sustainable tourism movement seems to be growing around the world. Several high profile 

international organizations are now developing policy, management principles, certification 

guidelines, etc. for sustainable tourism development. These include the U.N. World Tourism 

Organization (UNWTO), UNESCO World Heritage Center, The World Bank, World Travel & 

Tourism Council (WTTC), Pacific Asia Travel Association (PATA), and the European Network 

for Sustainable Development (ECOTIP), and many others. Furthermore, the governments of 

many countries have developed strong policies to encourage sustainable tourism including 

Australia, New Zealand, England, Costa Rica, Brazil, Norway, Peru, Romania, Canada and the 

Caribbean Alliance for Sustainable Tourism (CAST). In the United States, supported efforts 

through the National Park Service and the National Heritage Area Programs have emerged.

Other U.S. based non-profits like the Nature Conservancy, the National Trust for Historic 

Preservation, and the National Geographic Conservation Trust have supported and promoted

research, conferences, and news related to sustainable tourism efforts. In the private sector, 

efforts or organizations include “Green Seal” with support from the American Hotel & Lodging 

Association and the “Green” Hotels Association. American Express Company has also been a 

leader and provided financial support of sustainable projects around the U.S. Hawaii, Vermont, 

Wisconsin, Rhode Island and Arizona are among the states leading the way in sustainable 

tourism (Sheldon, Knox and Lowry, 2005).

McCool, Moisey and Nickerson (2001) reported that from a list of 20 predetermined items that 

focused on what Montana’s tourism industry should sustain, the items ranked highest were 

“Montana’s natural and cultural heritage, community economic stability, quality of life and 

unique natural environment (p. 127).” Further, they point out the need for shared definitions to 

prevent the negative aspects of social discourse. They also suggest there is “somewhat of a 

disconnect between preferences for what should be sustained by tourism and indicators that 

might measure progress toward this goal,” (p. 127). Clearly there seems to be a monitoring 

dilemma that currently plagues sustainable tourism development. This dilemma indicates a need 

for developing useful measures for sustainable tourism as discussed in the next section.

Developing Measures of TBL for Sustainable Tourism

As stated earlier, one of the most challenging aspects of implementing TBL is the difficulty 

associated with developing meaningful economic, social and environmental indicators. Of these, 

the greatest challenge is associated with attempting to quantify social and environmental 

impacts. McCool, Moisey and Nickerson (2001) suggest that, in assessing sustainable tourism 

impact, the initial question when considering appropriate indicators must be “what should the 

tourism and recreation industry sustain?” followed by a need to identify indicators that can be 

monitored to determine if current policies are facilitating sustainability taking into consideration

the context in which they exist. In developing appropriate indicators, it has been generally 

recognized that there is a need to tailor these indicators for specific industry sectors. Within 

tourism, it is also important to tailor indicators since tourists in different market segments can 

generate varying social and environmental impacts on destinations (Dwyer and Forsyth, 2008). 

A review of the existing literature reveals some early attempts to identify meaningful measures 

of sustainable tourism impact (WTO, 1996; Dymond, 1997). More recent progress using various 

approaches (such as the Delphi technique, surveys, interviews and focus groups) to develop 
Program Evaluation of ARC’s Tourism, Page F-13

Cultural Heritage and Natural Asset-Related Projects

indicators of the economic, environmental and social impact of sustainable tourism based on data 

collected from a wide variety of constituent groups is discussed next and the results are 

compared and contrasted.

Economic Indicators. Economic measures of tourism are the most straightforward to establish 

and have long been in place. Typically, economic impact is measured by hotel occupancy rates, 

number of nonresident visitors, per capita tourist expenditures and lodging revenues, number of 

tourism employees and labor income from tourism (McCool, Moisey and Nickerson, 2001). 

Others have developed indicators focused on employment issues, destination economic benefits 

(Miller, 2001; WTO, 2004; Choi and Sirikaya, 2006), seasonality and poverty alleviation (WTO, 

2004; Choi and Sirikaya, 2006). A comparison of the economic indicators that emerged from 

previous research is shown in Table 3.

Table 3

Review of Economic Indicators of Tourism Impact

Theme Indicators Study

McCool, Moisey 

and Nickerson 

(2001)

Miller 

(2001)

WorldTourism 

Organization 

(2004)

Choi and 

Sirikaya 

(2006)

Degree of seasonality X X X

Strengthening Shoulder 

Seasons and Low Seasons

X

Provision of sufficient 

infrastructure year‐round

X

Seasonality

Short‐term and seasonal 

employment

X

Number and quality of 

employment in tourism sector

X X X X

Professional and personal 

development

X

Contentment from work X

Lack of skilled labor X

Employment

Labor income X

Business investment in 

tourism

X

Tourism revenue X X X X

Community Expenditures X X X

Net Economic Benefits X X X

Changes in cost of living X

financial rate of return to 

operators

gross operating surplus of 

different tourism industry 

sectors

X X

Destination 

Economic 

Benefits

hotel occupancy rates X
Program Evaluation of ARC’s Tourism, Page F-14

Cultural Heritage and Natural Asset-Related Projects

number of nonresident visitors X

Amount of money leaving 

tourism locality

X X

Stabilizing and improving 

community income

X X

Improving local employment 

opportunities

X X

Operation and support of 

MSMEs

X X

Achieving equitable 

distribution of tourism 

funds/benefits across the 

community

X X

Number and type of 

development programs in 

place

X

Community survey assessment 

of usefulness and success of 

various development 

programs

X

Poverty 

Alleviation

Evaluating less tangible, non‐

economic, livelihood priorities

X

Social Indicators. As discussed previously, social impact is a term that is closely linked with the 

concept of social capital, i.e., improving trust, encouraging cooperation and collaboration, 

recognizing and enhancing individual and organizational networks and fostering life-long 

learning (Rogers, 2003).

To date, research has suggested a number of social impacts of tourism on communities. These 

include state of the local economy (Gursoy, Jurowski & Uysal, 2002), maturity of the tourism 

destination and level of community attachment (Fedline, 2001). More recently, in cooperation 

with the Sustainable Tourism Cooperative Research Centre (STCRC) and based on the Green 

Globe 21 Standard, Deery, Jago and Fredline (2005), developed a potential list of indicators to 

measure the social and socio-economic impacts of tourism on communities. Through the use of 

focus groups and in-depth interviews with community residents of a tourist community in 

Victoria, Australia, they identified a number of social and socio-economic indicators. Other 

suggested social indicators fall into broad categories of support for access and equity, pressure 

on services, pride and sense of belonging to local area, support for cultural and artistic 

endeavors, regional showcase and community health and safety issues (WTO, 2004; Choi and 

Sirikaya, 2006). A detailed comparison of the social indicators that have emerged from previous 

literature is shown in Table 4.

Table 4
Program Evaluation of ARC’s Tourism, Page F-15

Cultural Heritage and Natural Asset-Related Projects

Review of Social Indicators of Tourism Impact

Theme Indicators Study

McCool, 

Moisey 

and 

Nickerson 

(2001)

Miller 

(2001)

World 

Tourism 

Organization 

(2004)

Deery, 

Jago and 

Fredline 

(2005)

Choi and 

Sirikaya 

(2006)

Resident population X X

Visitors to the Area X X

Background

Land use, Particularly 

Tourism Development

X

Access, especially parking X* X X

Highway traffic count X X* X

Housing Affordability X X X

Business Success X

Safety in the Community X X X

Crowding X X

Tourism Development X

Resident attitudes towards 

tourism

X X X

Pressure on Health and 

Social Services

X X X

Pressure on Police X X

Number of Complaints by 

local residents

X

Positive Participation in 

Community Activities

X X X

Change in Character of Local 

Community Such as 

Development of Local 

Community Groups

X* X X

Maintenance of Cultural 

Heritage Through 

Enhancement of Attractions 

X* X X

Resident perceptions of 

quality of life

X X X

Cultural Development: 

events (number and type)

X X

Increased Awareness of 

Destination (Increased 

Visitation)

X X

Increased Awareness of 

Destination ( New 

Investment/Business 

Opportunity in the Region)

X X

Social 

Environment

Change in Crime Patterns X X X X
Program Evaluation of ARC’s Tourism, Page F-16

Cultural Heritage and Natural Asset-Related Projects

Change in Social Problems 

(e.g., alcohol, drugs)

X X X

Malnutrition X

Family Support X X

Education and Training X* X

*Classified as “environmental indicators” by Miller (2001)

It is interesting to note, however, that to date no widely accepted method of measuring social 

capital in tourism has been developed.

Environmental Indicators. Environmental indicators suggested by researchers are varied. Miller 

(2001) conducted a study to develop indicators in the UK and in Australia, Lundie, Dwyer and 

Forsyth (2007) suggested measures focused on energy use, water use, greenhouse gas emissions 

and ecological footprint. Others have generated a more complete list, for example, the most 

recent list generated by the WTO (2004) who polled 62 experts from more than 20 countries, is 

the most comprehensive resource on this topic. Their list includes management of natural 

resources (waste, water, energy, etc.), climate change, visual impact of tourism and measuring 

the impact of tourism on the natural environment. More recently, Choi and Sirakaya (2006) 

reinforced the efficacy of this list when a panel of 38 academics reached consensus on a total of 

25 environmental (ecological) indicators, the majority of which duplicate those of the WTO 

(2004). A comparison of environmental indicators that have emerged from previous research are 

listed in Table 5.

Table 5

Review of Environmental Indicators of Tourism Impact

Theme Indicators Study

McCool, 

Moisey & 

Nickerson 

(2001)

Miller 

(2001)

World 

Tourism 

Organization 

(2004)

Choi and 

Sirikaya 

(2006)

Lundie, 

Dwyer & 

Forsyth 

(2007)

Measuring impact 

of tourism on 

natural 

environment 

X X X

Financing for 

biodiversity 

conservation and 

maintenance of 

protected areas

X

Tourism as a 

Contributor to 

Nature 

Conservation

Site‐specific 

regulations

X X X
Program Evaluation of ARC’s Tourism, Page F-17

Cultural Heritage and Natural Asset-Related Projects

Provision of 

opportunities for 

participation by 

tourists in 

conservation

X X

Intensity of use X X

Managing scarce 

resources

X X X

Greenhouse gas 

emissions

X X

Water availability 

and conservation

X X X X

Sewage treatment X X X

Solid waste 

management

X X

water pollution X X X

Air pollution X X X

Controlling Noise 

Levels

X X X

Limiting 

Environmental 

Impacts of Tourism 

Activity

Managing Visual 

impacts of 

Tourism

X X X

A review of Tables 3 through 5 suggests that the list of economic, social and environmental 

indicators of tourism developed by the World Tourism Organization (2004) and published in a 

guidebook of indicators of sustainable development for tourism destinations is still the most 

comprehensive and useful set of measures (See Appendix A). Consequently, it is suggested that 

this set of measures appears to offer the greatest potential in developing a widely accepted, 

comprehensive list of sustainable tourism indicators. It also appears that it is appropriate to 

develop destination-specific indicators and a comparison across destination types (e.g., urban, 

rural, coastal, etc.) and at different levels (state, region, community) to properly document 

sustainable tourism impact across the different tourism sectors (WTO, 2004; Lundie, Dwyer and 

Forsyth, 2007; Dwyer and Forsyth, 2008). To date, however, it appears that limited research has 

been undertaken to measure sustainable tourism impact on a segment-by-segment basis and that 

there is still much work to be done in this important area.

Utility of TBL for Evaluation of Tourism Development Projects

One question that must be addressed when applying triple bottom line thinking to tourism 

development projects is whether synergies can be achieved between the tourism development 

organization and the community within which the tourism organization operates. The previous 

sections delineated the benefits of the TBL approach for the local communities, but what benefits 

accrue to the tourism organization for implementing the TBL?
Program Evaluation of ARC’s Tourism, Page F-18

Cultural Heritage and Natural Asset-Related Projects

Dwyer (2005) provided insight into the benefits that accrue to the tourism development 

organization resulting from the adoption of the TBL approach. Included in these benefits are 

efficiencies and cost savings, improved market positioning, better stakeholder relations, 

improved strategic decision making along with benefits to the wider community.

With regard to efficiencies and cost savings, TBL reporting can identify potential cost savings 

such as reduced operating costs via the reduction of materials and energy use, enhanced 

operational and design efficiencies, recycling and reusing wastes, as well as reduced 

transportation storage and packaging costs. Lower compliance costs may result if regulators have 

a better understanding of an organization’s operations. Human resource costs may be reduced 

when employees are attracted and retained by an organization that focuses on community and 

environmental values. Finally, capital costs may be reduced as the organization has improved 

access to “green” and “ethical” investment funds (Dwyer, 2005).

From a marketing perspective, tourism development organizations can benefit from adopting the 

TBL approach through improved market positioning. Improved market positioning can result 

when the organization’s brand image is enhanced when consumers become aware of the 

environmental and social sensitivity of the organization. In addition, the competitive 

differentiation afforded by the TBL approach can assist the organization in appealing to new 

markets, and encouraging repeat visitation (Dwyer, 2005).

Enhanced tourism development organization stakeholder relationships can result as the 

organization’s beneficial activities permeate throughout the community fostered by increased 

transparency. These enhanced relationships can also provide a competitive advantage as the 

organization’s positive reputation is communicated among stakeholders strengthening the 

organization’s brand (Dwyer, 2005).

Finally, adopting the TBL approach can improve a tourism development organization’s strategic 

decision making. The TBL approach forces the organization to focus on managing those tasks 

that get measured, systematize the best practices and benchmarks, improve risk management 

activities, improve the quality of information for decision makers, and facilitate information 

sharing throughout the organization which promotes integrated decision making within the 

organization and with other organizations in the community (Dwyer, 2005).

Given the usefulness of the triple bottom line approach for tourism development organizations, 

measures were developed that were designed to assess the degree to which tourism development 

projects achieved the three triple bottom line objectives (economic, social and environmental). 

The following section discusses the development of these measures.

Measurement Development Procedure

Based upon the previous literature review, a set of 115 items were adapted from Indicators of 

Sustainable Development for Tourism Destinations: A Guidebook (2004). Of the 115 items 

adopted, 53 items were thought to measure the social bottom line; 23 items were thought to 

measure the environmental bottom line, and 39 items were thought to measure the economic 

bottom line (see Table 6).
Program Evaluation of ARC’s Tourism, Page F-19

Cultural Heritage and Natural Asset-Related Projects

Table 6

 Initial Triple Bottom Line Indicators

Social Indicators

Degree to which this tourism development project impacts local satisfaction with tourism

Degree to which this tourism development project impacts the number of tourism complaints by local 

residents

Degree to which this tourism development project impacts community social services

Degree to which this tourism development project impacts local infrastructure

Degree to which this tourism development project impacts community participation rate in tourism

Degree to which this tourism development project impacts the number of tourists per day

Degree to which this tourism development project impacts the ratio of number of tourists to locals

Degree to which this tourism development project impacts the number of tourists per square mile

Degree to which this tourism development project impacts the preservation of the local culture

Degree to which this tourism development project impacts traditional occupations in the local community

Degree to which this tourism development project impacts residents continuing with local customs

Degree to which this tourism development project impacts residents continuing with local language

Degree to which this tourism development project impacts locals continuing with local music

Degree to which this tourism development project impacts locals continuing with local cuisine

Degree to which this tourism development project impacts cultural activities

Degree to which this tourism development project impacts affordable housing for local residents

Degree to which this tourism development project impacts the distance locals must travel to work

Degree to which this tourism development project impacts the number of local residents that left the 

community last year

Degree to which this tourism development project impacts the number of people from outside the local 

community taking tourism jobs in the past year

Degree to which this tourism development project impacts net migration in/out of the local community

Degree to which this tourism development project impacts the costs of local assets to community residents

Degree to which this tourism development project impacts the number of complaints by community 

residents regarding tourism

Degree to which this tourism development project impacts conserving local cultural sites

Degree to which this tourism development project impacts the local community in conserving local 

monuments

Degree to which this tourism development project impacts the local community by minimizing damage to 

local heritage sites

Degree to which this tourism development project impacts the maintenance and preservation of local 

cultural/heritage sites

Degree to which this tourism development project impacts local effluent treatment facilities

Degree to which this tourism development project impacts the local community by supporting local noise 

regulations

Degree to which this tourism development project impacts the local community by supporting local 

congestion regulations
Program Evaluation of ARC’s Tourism, Page F-20

Cultural Heritage and Natural Asset-Related Projects

Degree to which this tourism development project impacts the local community by enforcing local alcohol 

regulations

Degree to which this tourism development project impacts tourist behavior near residential communities

Degree to which this tourism development project impacts tourist behavior near children's play areas

Degree to which this tourism development project impacts local education programs on substance abuse

Degree to which this tourism development project impacts local health programs for substance abuse

Degree to which this tourism development project impacts a smoke free environment

Degree to which this tourism development project impacts childcare facilities for workers

Degree to which this tourism development project impacts family‐friendly work shifts

Social Indicators (Continued)

Degree to which this tourism development project impacts the number of scholarships for local residents

Degree to which this tourism development project impacts training programs for local residents

Degree to which this tourism development project impacts work programs for local residents

Degree to which this tourism development project impacts local community's ability to provide drinking 

water

The degree to which this tourism development project impacts local community's ability to provide waste 

disposal

The degree to which this tourism development project impacts local community's ability to provide sewage 

treatment

Degree to which this tourism development project impacts the number of crimes involving tourists

Degree to which this tourism development project impacts community needs development

Degree to which this tourism development project impacts local parking

Degree to which this tourism development project impacts pressure on local health services

Degree to which this tourism development project impacts pressure on police

Degree to which this tourism development project impacts the number of local community groups

Degree to which this tourism development project impacts local senior citizens

The degree to which this tourism development project impacts community sports programs

The degree to which this tourism development project impacts community festivals and events

The degree to which this tourism development project impacts community wellness centers

Environmental Indicators

Degree to which this tourism development project impacts the number of local environmental assessment 

projects

Degree to which this tourism development project impacts the number of local conservation projects

Degree to which this tourism development project impacts the number of local protected areas

Degree to which this tourism development project impacts local tourist group size

Degree to which this tourism development project impacts local environmentally friendly modes of 

transport

Degree to which this tourism development project impacts local environmentally friendly waste disposal 

methods

Degree to which this tourism development project impacts local noise pollution

Degree to which this tourism development project impacts opportunities for tourists to practice 
Program Evaluation of ARC’s Tourism, Page F-21

Cultural Heritage and Natural Asset-Related Projects

conservation (recycling, etc.)

Degree to which this tourism development project impacts local ordinances to minimize environmental 

impacts

Degree to which this tourism development project impacts the number of tourists to the local community

Degree to which this tourism development project impacts energy consumption

Degree to which this tourism development project impacts local use of renewable energy sources

Degree to which this tourism development project impacts local greenhouse gas emissions

Degree to which this tourism development project impacts local water consumption

Degree to which this tourism development project impacts local recapturing and reuse of water

Degree to which this tourism development project impacts sewage treatment in the local community

Degree to which this tourism development project impacts solid waste disposal in the local community

Degree to which this tourism development project impacts air pollution in the local community

Degree to which this tourism development project impacts noise pollution in the local community

Degree to which this tourism development project impacts the local visual landscape

Degree to which this tourism development project impacts the loss of open land in the local communty

Environmental Indicators (Continued)

Degree to which this tourism development project impacts public open spaces such as parks

Degree to which this tourism development project impacts suppliers' waste management activities

Economic Indicators

Degree to which this tourism development project impacts local community development programs

Degree to which this tourism development project impacts the local community's cultural values

Degree of local skilled labor for this tourism development project

Degree to which this tourism development project impacts local community natural resource management 

programs

Satisfaction of employees of this tourism development project

Percent of jobs provided by this tourism development project that are less than six months per year

Degree to which this tourism development project impacts local community cash needs

Degree to which this tourism development project impacts the local community's physical security

Degree to which this tourism development project impacts local community training programs

Degree to which this tourism development project impacts the local community's educational programs

Degree to which this tourism development project impacts the local community via tourism taxes

Opportunities for promotion for employees of this tourism development project

Degree to which this tourism development project impacts low seasons

Degree to which this tourism development project impacts the local community's income

Percentage of jobs provided by this tourism development project that are full‐time jobs

Total number of jobs provided by this tourism development project

Months per year this tourism development project operates

Percent of jobs provided by this tourism development project that are full‐year

Yearly revenue of this tourism development project
Program Evaluation of ARC’s Tourism, Page F-22

Cultural Heritage and Natural Asset-Related Projects

Degree to which this tourism development project impacts the local community's health management 

programs

Asset value of this tourism development project

Degree to which this tourism development project impacts empolyee decision making

Frequency of employee training programs for this tourism development project

Number of workplace accidents for this touism development project

Degree to which this tourism development project impacts local community conservation programs

Employee retention rate of this tourism development project

Average total local tourist spending at this tourism development project

Degree to which this tourism development project impacts local community food security

Degree to which this tourism development project impacts suppliers' waste management activities

Percentage of employees of this tourism development project that are qualified/certified/degreed

Degree to which this tourism development project impacts the local community's poor

Degree to which this tourism development project impacts local community infrastructure

Training funds spent per employee of this tourism development project

Degree to which this tourism development project impacts employee family income

Degree to which this tourism development project impacts shoulder seasons

Economic Indicators (Continued)

Degree to which this tourism development project impacts local employment opportunities

Per person fees charged for this tourism development project

Income levels of employees of this tourism development project

Degree to which this tourism development project impacts the operation and support of micro, small, and 

medium sized enterprises (MSMEs)

After the items were identified, they were each placed on individual cards for a sorting task that 

was completed by two experts from the RTS Advisory Board. For purposes of sorting the items 

the board members were provided with the following definitions for each of the TBL 

dimensions:

The Economic Dimension: Refers to the financial 'revenue' shared by all commerce but also 

includes the additional economic benefits enjoyed by the host society.

The Social Dimension: Sometimes called social capital or community well-being, refers to the 

human capital provided by employees, contractors, suppliers, and advisors as well as investment 

in the social and cultural systems that support an organization.

The Environmental Dimension: Sometimes called natural capital, refers to the availability of 

natural resources upon which society’s existence depends.

The sorting task involved categorizing each of the 115 items into one of four categories, the

economic dimension, the social dimension, the environmental dimension, and a fourth “none of 

the above” category. 
Program Evaluation of ARC’s Tourism, Page F-23

Cultural Heritage and Natural Asset-Related Projects

After the sorting task was completed, an examination for consensus was undertaken. Items were 

eliminated if there was no category consensus between the researchers and the two experts. 

Items were re-categorized by majority decision. As a result of this process, 92 items remained in 

the scales, 44 that measure the social dimension, 20 that measure the economic dimension, and 

28 that measure the environmental dimension. Table 7 presents the final triple bottom line 

indicators.

Table 7

Final Triple Bottom Line Indicators

Social Indicators

1 Degree to which this tourism development project impacts local satisfaction with tourism

2

Degree to which this tourism development project impacts the number of tourism complaints by 

local residents

3 Degree to which this tourism development project impacts community social services

4 Degree to which this tourism development project impacts local community development programs

5 Degree to which this tourism development project impacts community participation rate in tourism

6 Degree to which this tourism development project impacts the local community's cultural values

7 Degree of local skilled labor for this tourism development project

8 Degree to which this tourism development project impacts the local community's physical security

9 Degree to which this tourism development project impacts the preservation of the local culture

10

Degree to which this tourism development project impacts the local community's educational 

programs

11 Degree to which this tourism development project impacts residents continuing with local customs

12 Degree to which this tourism development project impacts residents continuing with local language

13 Degree to which this tourism development project impacts locals continuing with local music

14 Degree to which this tourism development project impacts locals continuing with local cuisine

15 Degree to which this tourism development project impacts cultural activities

16 Degree to which this tourism development project impacts affordable housing for local residents

17

Degree to which this tourism development project impacts the local community's health 

management programs

18

Degree to which this tourism development project impacts the number of local residents that left 

the community last year

19 Frequency of employee training programs for this tourism development project

20

Degree to which this tourism development project impacts net migration in/out of the local 

community

21 Degree to which this tourism development project impacts local community food security

22

Degree to which this tourism development project impacts the number of complaints by community 

residents regarding tourism

23 Degree to which this tourism development project impacts conserving local cultural sites

24

Degree to which this tourism development project impacts the local community in conserving local 

monuments
Program Evaluation of ARC’s Tourism, Page F-24

Cultural Heritage and Natural Asset-Related Projects

25

Degree to which this tourism development project impacts the local community by minimizing 

damage to local heritage sites

26

Degree to which this tourism development project impacts the maintenance and preservation of 

local cultural/heritage sites

27 Percentage of employees of this tourism development project that are qualified/certified/degreed

28 Degree to which this tourism development project impacts the local community's poor

29

Degree to which this tourism development project impacts the local community by enforcing local

alcohol regulations

30

Degree to which this tourism development project impacts tourist behavior near children's play 

areas

31

Degree to which this tourism development project impacts local education programs on substance 

abuse

32

Degree to which this tourism development project impacts local health programs for substance 

abuse

33 Degree to which this tourism development project impacts childcare facilities for workers

34 Degree to which this tourism development project impacts family‐friendly work shifts

35 Degree to which this tourism development project impacts training programs for local residents

36 Degree to which this tourism development project impacts work programs for local residents

37 Degree to which this tourism development project impacts the number of crimes involving tourists

38 Degree to which this tourism development project impacts pressure on local health services

39 Degree to which this tourism development project impacts pressure on police

40 Degree to which this tourism development project impacts the number of local community groups

41 Degree to which this tourism development project impacts local senior citizens

42 The degree to which this tourism development project impacts community sports programs

43 The degree to which this tourism development project impacts community festivals and events

44 The degree to which this tourism development project impacts community wellness centers

Economic Indicators

1

Degree to which this tourism development project impacts the number of people from outside the 

local community taking tourism jobs in the past year

2

Degree to which this tourism development project impacts local community natural resource 

management programs

3 Percent of jobs provided by this tourism development project that are less than six months per year

4 Degree to which this tourism development project impacts the local community via tourism taxes

5 Opportunities for promotion for employees of this tourism development project

6 Degree to which this tourism development project impacts low seasons

7 Degree to which this tourism development project impacts the local community's income

8 Percentage of jobs provided by this tourism development project that are full‐time jobs

9 Total number of jobs provided by this tourism development project

10 Months per year this tourism development project operates

11 Percent of jobs provided by this tourism development project that are full‐year

12 Yearly revenue of this tourism development project
Program Evaluation of ARC’s Tourism, Page F-25

Cultural Heritage and Natural Asset-Related Projects

13 Asset value of this tourism development project

14 Employee retention rate of this tourism development project

15 Average total local tourist spending at this tourism development project

16 Degree to which this tourism development project impacts employee family income

17 Degree to which this tourism development project impacts local employment opportunities

18 Per person fees charged for this tourism development project

19 Income levels of employees of this tourism development project

20

Degree to which this tourism development project impacts the operation and support of micro, 

small, and medium sized enterprises (MSMEs)

Environmental Indicators

1 Degree to which this tourism development project impacts local effluent treatment facilities

2

Degree to which this tourism development project impacts the local community by supporting local 

noise regulations

3

Degree to which this tourism development project impacts the local community by supporting local 

congestion regulations

4 Degree to which this tourism development project impacts a smoke free environment

5

Degree to which this tourism development project impacts local community's ability to provide 

drinking water

6 Degree to which this tourism development project impacts local community conservation programs

7 Degree to which this tourism development project impacts suppliers' waste management activities

8

Degree to which this tourism development project impacts the number of local environmental 

assessment projects

9 Degree to which this tourism development project impacts the number of local conservation projects

10 Degree to which this tourism development project impacts the number of local protected areas

11

Degree to which this tourism development project impacts local environmentally friendly modes of 

transport

12

Degree to which this tourism development project impacts local environmentally friendly waste 

disposal methods

13 Degree to which this tourism development project impacts local noise pollution

14

Degree to which this tourism development project impacts opportunities for tourists to practice 

conservation (recycling, etc.)

15

Degree to which this tourism development project impacts local ordinances to minimize 

environmental impacts

16 Degree to which this tourism development project impacts energy consumption

17 Degree to which this tourism development project impacts local use of renewable energy sources

18 Degree to which this tourism development project impacts local greenhouse gas emissions

19 Degree to which this tourism development project impacts local water consumption

20 Degree to which this tourism development project impacts local recapturing and reuse of water

21 Degree to which this tourism development project impacts sewage treatment in the local community

22

Degree to which this tourism development project impacts solid waste disposal in the local 

community

23 Degree to which this tourism development project impacts air pollution in the local community
Program Evaluation of ARC’s Tourism, Page F-26

Cultural Heritage and Natural Asset-Related Projects

24 Degree to which this tourism development project impacts noise pollution in the local community

25 Degree to which this tourism development project impacts the local visual landscape

26

Degree to which this tourism development project impacts the loss of open land in the local 

communty

27 Degree to which this tourism development project impacts public open spaces such as parks

28 Degree to which this tourism development project impacts suppliers' waste management activities

Preliminary Survey Design Ideas

After identifying the measures of the triple bottom line for tourism development organizations, a 

survey instrument was developed to collect data from each tourism development organization. 

The sample survey instrument is presented in Appendix 1. For each tourism development 

project it is envisioned that multiple respondents will respond to the triple bottom line 

questionnaire. Means and standard deviations will be computed for each questionnaire item and 

grand means will be computed for each of the social, economic and environmental dimension of 

the triple bottom line for each tourism development project. The grand means for the social and 

environmental dimensions can then be used as input for assessment of the extent to which each 

tourism development project pursues a social and environmental agenda. The following section 

discusses the assessment technique that is being contemplated for the tourism development 

projects. 

Assessment of the Results

Subsequent to calculating the grand means for the social and environmental dimensions, each 

tourism development project can be plotted on a graph where the X-axis represents the mean of 

the social dimension and the Y-axis represents the mean of the environmental dimension (for 

each tourism development project). Figure 2 shows such a graph which we label the socialenvironmental tourism project evaluation matrix.
Program Evaluation of ARC’s Tourism, Page F-27

Cultural Heritage and Natural Asset-Related Projects

Figure 2: The Social-Environmental Tourism Project Evaluation Matrix.

 As Figure 2 shows, plotting each tourism development project will allow for the assessment of 

projects in both an absolute and relative sense. From the absolute perspective projects can be 

assessed on a scale of “goodness.” Here, a bad project might be one whose grand environmental 

mean and grand social mean is 1 to 3 on the 9-point scale. Moderate projects might be those 

whose grand environmental and social means fall within the range of 4 - 6 on the scale. Good 

projects might be those whose grand environmental and social means fall within the range of 7 -

9 on the scale. Finally, with regard to the relative perspective, plotting tourism development 

projects relative to each other might allow for generalizations as to which types of projects might 

be more likely to be environmentally and socially cognizant.
Program Evaluation of ARC’s Tourism, Page F-28

Cultural Heritage and Natural Asset-Related Projects

References

Agenda 21, the Rio Declaration on Environment and Development, (1992). Available at: 

http://www.un.org/esa/sustdev/documents/agenda21/english/agenda21toc.htm (Accessed on 

January 29 2008)

Caton, K. and Almeida Santos, C. (2007), Heritage Tourism on Route 66: Deconstructing 

Nostalgia, Journal of Travel Research, 45 (4), 371-386.

Economic measures of tourism yield: what markets to target? Larry Dwyer, Peter Forsyth

International Journal of Tourism Research

Volume 10, Issue 2 , Pages155 - 168

Choi, H.C. and Sirakaya, E. (2006). Sustainability Indicators for Managing Community 

Tourism. Tourism Management, 27, 1274-1289.

Dahle, C. (2007). TBL reporting: Taking Stock of More Than Share Price. Online resource: 

http://www.blueegg.com/hot_topics/2007/8/29/triple-bottom-line-dahle/print.go (Accessed 

February 6 2008)

Deery, M, Jago, L. and Fredline, L. (2005). A Framework for the Development of Social and 

Socioeconomic Indicators for Sustainable Tourism in Communities, Tourism Review 

International, 9, 1-9.

Dwyer, L. (2005). Relevance of Triple Bottom Line Reporting to Achievement of Sustainable 

Tourism: A Scoping Study, Tourism Review International 9,79-93.

Dwyer, L. and Forsyth, P. (2008). Economic measures of tourism yield: what markets to target?

International Journal of Tourism Research, 10(2), 155 – 168.

Dymond, S. 1997. Indicators of sustainable tourism in New Zealand: a local government 

perspective. Journal of Sustainable Tourism. 5(4); 279-293.

Elkington, J. (2004). Enter the Triple Bottom Line. In A. Henriques and J. Richardson (Eds.) 

The Triple Bottom Line, Does It All Add Up? Assessing the Sustainability of Business and CSR, 

London: Earthscan Publications Ltd., 1-16

__________ (1999), Triple Bottom Line Revolution -- Reporting for the Third Millennium, 

Australian CPA, 69(1), 75 - 77.

__________ (1998). Cannibals with Forks: The Triple Bottom Line of 21st Century Business. 

New Society Publishers: Gabriola Island, BC and Stony Creek, CT

__________ (1994). Towards the Sustainable Corporation: Win-Win-Win Business Strategies 

for Sustainable Development, California Management Review, 36(2): 90-100.
Program Evaluation of ARC’s Tourism, Page F-29

Cultural Heritage and Natural Asset-Related Projects

Fredline, L. (2001). Social Impacts of Tourism on the Gold Coast: Host Community 

Perceptions of the Impacts of Tourism on the Gold Coast. Griffith University, Gold Coast, QLD: 

CRC for Sustainable Tourism.

Gilmore, A. and Simmons, G. (2007), Integrating Sustainable Tourism and Marketing 

Management: Can National Parks Provide the Framework for Strategic Change? Strategic 

Change, 16 (5), 191-200.

Global Resource Initiative (2008). 

http://www.globalreporting.org/ReportingFramework/G3Guidelines/. (Accessed January 29 

2008)

Goldsworthy, A. (2000). The Triple Bottom Line: Shareholders Society Sustainability. BHERT 

News, Issue 9: 4-6.

Gursoy, D., Jurowski, C. and Uysal, M. (2002). Resident Attitudes: A structural Modeling 

Approach. Annals of Tourism Research, 29(10), 79-105.

Hill, R. (2000). The Triple Bottom Line: Shareholders, Society, Sustainability. BHERT News. 

Issue 9: 1-44.

Hvenegaard, G. T. (1994), Ecotourism: A Status Report and Conceptual Framework, The

Journal of Tourism Studies, 5 (2), 24-35.

Isaksson, R. and Garvare, R. (2003), Measuring Sustainable Development Using Process 

Models, Managerial Auditing Journal, 18 (8), 649-656.

Kerstetter, D., Confer, J. and Bricker, K. (1998), Industrial Heritage Attractions: Types and 

Tourists, Journal of Travel and Tourism Marketing, 7 (2), 91-104.

Kimmet, P. and Boyd. T. (2004), An Institutional Understanding of Triple Bottom Line 

Evaluations and the use of Social and Environmental Metrics, PPRES Conference, Pacific Rim 

Estate Society Press, Thailand, January 25-28.

Lansing, P. and De Vries, P. (2007). Sustainable Tourism: Ethical Alternative or Marketing 

Ploy, Journal of Business Ethics (2007) 72:75-85 

Lundie, S., Dwyer, L. and Forsyth, P. (2007). Environmental-Economic Measures of Tourism 

Yield, Journal of Sustainable Tourism, (15) 503-519.

McCool, S, Moisey, N. and Nickerson, N. (2001). What Should Tourism Sustain? The 

Disconnect with Industry Perceptions of Useful Indicators, Journal of Travel Research, 40 (2): 

124-131.

McDonough, W. and Braungart, M. (2002). Design for the Triple Top Line: New Tools for 

Sustainable Commerce. Corporate Environmental Strategy, 9(3): 251-258.
Program Evaluation of ARC’s Tourism, Page F-30

Cultural Heritage and Natural Asset-Related Projects

McGehee, N. G. (2007), An Agritourism Systems Model: A Weberian Perspective, Journal of 

Sustainable Tourism, 15 (2), 111-124.

____________ and Kim, K. (2004), Motivation for Agri-Tourism Entrepreneurship, Journal of 

Travel Research, 43 (2), 161-170.

Manning, E. W. and Dougherty T.D. ( 1995). Sustainable Tourism: Preserving the Golden

Goose. Cornell Hotel and Restaurant Administration Quarterly, 36(2): 29-42.

Marshall, J. D. and Toffel, M.W. (2005), Framing the Elusive Concept of Sustainability: A 

Sustainability Hierarchy, Environmental Science & Technology, 39 (3), 673-682.

Mehmetoglu, M. (2007), Nature-Based Tourists: The Relationship Between Their Trip 

Expenditures and Activities, Journal of Sustainable Tourism, 15 (2), 200-215.

Miller, G. (2001). The Development of Indicators for Sustainable Tourism: Results of a Delphi 

survey of tourism researchers. Tourism Management 22 (4) 351-362.

Miller, E., Buys, L. and Summerville, J. (2007), Quantifying the Social Dimension of Triple 

Bottom Line: Development of a Framework and Indicators to Assess the Social Impact of 

Organizations, International Journal of Business Governance and Ethics, 3 (3), 223-237.

Milne, M.J., Ball, A., and Gray, R. (2005). From Soothing Palliatives and Towards Ecological 

Literacy: A Critique of the Triple Bottom Line. Department of Accountancy and Business Law. 

Working Paper Series, University of Otago, New Zealand. Accessed at: 

http://www.commerce.otago.ac.nz/acty/research/pdf/Milne%20et%20al%20-

palliatives%20paper.pdf on January 26 2008.

National Geographic (2008). The Geotourism Charter. Online resource accessed at 

www.nationalgeographic.com/travel/sustainable/ on February 7 2008.

Norman, W. and MacDonald, C. (2004). Getting to the Bottom of Triple Bottom Line, Business 

Ethics Quarterly, 14 (2), 243-262.

Onyx, J. and Bullen, P. (2000) Measuring Social Capital in Five Communities. Journal of 

Applied Behavioral Science, 36(10): 23-42.

_________ (2000). Social Capital Is Good Business. BHERT NEWS, Issue 9: 11-13.

Rogers, M. (2003). Small Towns: Big Picture – Social Sustainability Indicators and the Arts. 

Social Dimensions of the Triple Bottom Line in Rural Australia. Bureau of Rural Sciences.

Sauvante, M. (2001). The ‘Triple Bottom Line: A Boardroom Guide. Director’s Monthly 

25(11): 1-6.

Schnake, M. (1991). Organizational Citizenship: A Review, Proposed Model and Research 

Agenda. Human Relations, 44(7): 735-759.
Program Evaluation of ARC’s Tourism, Page F-31

Cultural Heritage and Natural Asset-Related Projects

Sheldon, P., Knox, J. and Lowry, K. (2005). Sustainability in a Mature Mass-Tourism 

Destination: The Case of Hawai’i. Tourism Review International, 9: 47-59.

Vanclay, F. (2004). The Triple Bottom Line and Impact Assessment: How do TBL, EIA, SIA, 

SEA AND EMS Relate to Each Other? Journal of Environmental Assessment Policy and 

Management, 6(3): 265-288.

Vandenburg, M. (2002). TBL Victoria Scoping Study: How Victorian Businesses, 

Governments and Non-government Organisations are Taking the Journey toward the Triple 

Bottom Line. Report for the Victorian State Government, Department of Premier and Cabinet, 

Melbourne: Encompass. (on-line resource). 

http://www.ethyka.com/tbl/resources/TBL%20Scoping%20Study.pdf (Accessed 

Weaver, D. B. and Fennell, D. A. (1997), “The Vacation Farm Sector in Saskatchewan: A Profile 

of Operations,” Tourism Management, 18 (6), 357-365.

World Commission on Environment and Development (Brundtland Commission) (1987), Our 

Common Future, New York: Oxford University Press.

World Tourism Organization (WTO) (1996). What Tourism Managers Need to Know: A 

Practical Guide to the Development and Use of Indicators of Sustainable Tourism. Madrid, 

Spain: WTO Publications.

_____________________________ (2004). Indicators of Sustainable Development for Tourism 

Destinations: A Guidebook. Madrid, Spain: WTO Publications.
Appendix G. Sustainable Development / Triple Bottom Line 

Literature Review
Formulating a Sustainable Economic Development 

Process for Rural America

Sustainable Development / Triple Bottom Line Literature Review

DRAFT March 27, 2008 DRAFT

Prepared for:

Regional Technology Strategies, Inc.

Ford Foundation

Prepared by:

Scott Sawyer, PhD
1

Draft March 27, 2008 Draft

Formulating a Sustainable Economic Development Process 

for Rural America

Task: Sustainable Development / Triple Bottom Line Literature Review

The goals of the rural economic development portfolio in the Economic Development unit of the 

Ford Foundation are to figure out how “to increase the ownership of wealth by residents of rural 

America through the employment of triple bottom line (TBL) development practices” and “how 

to measure the results”. The Ford Foundation proposal suggests that a “semantic transition”, from 

sustainability to triple bottom line, is necessary to make sustainability a practical concept for alleviating 

rural poverty. The overall hypothesis being tested is whether cluster-based development, value chain 

connections, entrepreneurship training, and increasing the availability of financial services can be 

combined into a single development model incorporating a TBL economic development and evaluation process. 

A sub-task of the Ford Foundation proposal, and the purpose of this report, is to conduct a literature 

review for Regional Technology Strategies, Inc., that addresses “how triple bottom line approaches 

have been integrated into project activities and goals and how they have been used in evaluating 

tourism, cultural heritage and natural resource based projects both in the U.S. and overseas. The 

review, in the course, of this task will also address how different definitions of triple bottom line approaches impact evaluation metrics.” 

The rest of this report: 

► Defines the theory and practice of sustainable development (e.g., why is there a perception that 

 sustainable development is not a practical concept?) 

► Defines the triple bottom line and provides examples of its uses (e.g., what is a triple bottom line 

 development practice?)

► Describes how sustainable development and triple bottom line are different (i.e., is a semantic 

 transition from sustainability to triple bottom line advisable?)

► Uses sustainable tourism as an example of applying principles of sustainability and triple bottom 

 line measurement

► Concludes by providing caveats/warnings/suggestions about the use of indicators

A tally of sustainable development indicators assembled to demonstrate the wide variety of potential 

indicators is attached as an Excel spreadsheet in the Appendix. 

What is Sustainable Development? 

Human societies necessarily exploit surrounding ecosystems in order to survive, but societies that 

flourish to the extent of overexploiting their ecosystems may destroy the basis of their own survival. 

It was in the context of continuing environmental degradation, growing inequality, and persistent 

poverty that the most commonly used definition of sustainable development was put forward by the 
2

Draft March 27, 2008 Draft

World Commission on Environment and Development (the Brundtland Report) in Our Common 

Future:

“Sustainable development is development that meets the needs of the present without compromising the 

ability of future generations to meet their own needs” (1987: 43). 

How can development meet the needs of the present without compromising the ability of future 

generations to meet their own needs? How can societies determine if they are practicing sustainable 

development or unsustainable development? A substantial body of theory development and research 

in the natural sciences and social sciences has illuminated the social-ecological dynamics that lead 

to the collapse or sustainability of human societies. In particular, the theoretical work of Lee Freese 

(1988a, 1988b, 1994, 1995, 1997a, 1997b) and the body of knowledge accumulated by Lance Gunderson, C.S. Holling and colleagues (2002) have set the stage for answering these questions. 

Figure 1. A Qualitative Model for the Evolution of Biosociocultural Regimes

Figure 1 shows Lee Freese’s model for what he calls “the evolution of biosociocultural regimes.” The 

phrase “biosociocultural regime” is meant to encapsulate the fact that biophysical aspects of life (e.g., 

photosynthesis) and sociocultural aspects of life (e.g. urban development) continuously interact. 

Freese’s model is grouped into five interacting processes that describe how societies sustain themselves 

through time- or face collapse. For the purposes of demonstration, energy will be used as the variable moving through this model to indicate how a society can meet the needs of the present without 

compromising the ability of future generations to meet their needs. The example could have just as 

easily referred to water or forests or some other critical ecosystem service.

3

1

Ecosystem 

Energy 

Production

Human 

Energy 

Expropriation

Biosociocultural

Reorganization

Human 

Subsistence

Organization

Societal

Development - 

Dissolution

2

Human 

Subsistence

Reorganization

4

5

Ecosystem

Sustainability -

Disorganization

Source: Freese, 1997a.
3

Draft March 27, 2008 Draft

Freese starts with the proposition that “energy flows through ecosystems into sociocultural systems as 

the fundamental stuff of life support” (1997b: 84). Process 1 in his model describes the relationship 

between ecosystem energy production to variable rates of human energy expropriation. Depending 

on the level of energy expropriation, ecosystem resilience is either sustainably maintained or disorganized.

Process 2 describes the relationship between energy expropriation, subsistence organization, and development or collapse. As Smil (1994) and many others have documented, human energy expropriation 

lays the foundation for increasing complexity in human subsistence organization (i.e., the transition 

from hunting and gathering to agricultural production to industrialization). In other words, increases 

in energy expropriation are interrelated with increased technological complexity, rising population 

(e.g., due to increased food availability and decreased mortality), and divisions of labor, leading to 

more complex societies. Conversely, and in line with Tainter’s (1988) theory of societal collapse, societies that reduce energy expropriation–for whatever reason–face the possibility of dissolution.

Process 3 generally comes into play when a society develops beyond the hunting and gathering phase, 

when the intensification of production for subsistence “tests” the carrying capacity of the particular 

ecosystems sustaining them. Disorganized ecosystems can reduce the amount of energy available to 

societies and make the likelihood of their collapse more likely. 

Processes 4 and 5 reflect a reorganization of social systems and ecosystems in tandem, based on levels 

of ecosystem energy production, energy expropriation, societal development, and ecosystem disorganization. Freese summarizes our modern predicament in the context of his model as follows: “To 

not live within real biophysical carry capacity in effect is to not live within sustainable ecological 

means, which is to say, to live in ways that nature’s evolutionary design does not permanently enable” 

(1997b: 214). 

Before going further, a model called “panarchy” adds a layer of dynamism and helps to explain the 

status of each of the boxes in Figure 1. A panarchy is a set of “adaptive cycles” (Figure 2) nested or 

linked at progressively larger scales. Adaptive cycles are models of how social systems and ecosystems 

develop, collapse, and reorganize (Gunderson and Holling, 2002). 

Figure 2. Adaptive Cycle

Source: Gunderson and Holling, 2002.
4

Draft March 27, 2008 Draft

In the panarchy model, the “r phase” or exploitation phase refers, for example, to the colonization 

of recently disturbed areas in ecosystems or open competition in markets in social systems. The 

front-loop stage, from r to K, represents the slow, incremental phase of growth, accumulation and 

development (e.g., of forests, Microsoft, and bureaucracies). In this stage, winners expand, grow, and 

accumulate potential from resources acquired (e.g., the Republican party) or ecosystems accumulate 

and store energy and material. As the system evolves to the “K phase” or conservation phase, connectivity among the survivors intensifies, and new species find it difficult to enter the ecosystem or 

entrepreneurs find it difficult to enter existing markets. The tightly bound accumulation of biomass 

and nutrients or social relations in the K phase becomes increasingly fragile until suddenly released 

by agents (Ω) such as forest fires, insect pests, wars or social movements. It is in the K to Ω phase 

that the panarchy theorists say that all systems become accidents waiting to happen. The back-loop 

phase, from Ω to α, represents an increase in uncertainty and danger, as well as opportunity, renewal 

and surprise. 

Panarchy scholars are concerned with the resiliency of social-ecological systems. Resiliency refers to 

maintaining system integrity in the midst of disturbance, and the ultimate goal of resilience management is to “prevent the system from moving to undesired system configurations” and to “nurture 

and preserve the elements that enable the system to renew and reorganize itself following a massive 

change” (Walker et al., 2002). Identifying “thresholds”, particularly prior to the turbulent K to Ω 

phase, is key to preparing for, mitigating against, and adapting to disturbances. 

Replacing each of the boxes in Freese’s model with adaptive cycles (Figure 3) gives a fuller sense of 

what it means to be practicing sustainable development or unsustainable development.

Figure 3. Freese’s Model with Adaptive Cycles

 

Withdrawals Additions

Human 

Subsistence

Organization

Human Energy

Expropriation

Ecosystem Energy Production Ecosystem Sustainability /

Disorganization

Societal Development /

Collapse

Biosociocultural Reorganization

Human 

Subsistence 

Reorganization

1

2

4

3 5

Peak oil Climate Change

Overshoot

(Increasing temperature, altered weather patterns, 

population dispersal, etc.) 

(Crisis) (Crisis)

(Crisis)

Adaptive 

Cycle
5

Draft March 27, 2008 Draft

For example, many scholars have converged on three major causal factors undermining past civilizations: societies can overshoot the carrying capacity of their ecosystems by (1) depleting natural 

resources while simultaneously (2) degrading ecosystems (e.g., deforestation). Past societies experiencing sudden and prolonged (3) climatic events, such as droughts, face the possibility of collapse, 

and climate changes that intersect with resource depletion and ecosystem degradation increase the 

possibility of collapse (Good and Reuveny, 2006; Diamond, 2005; Fagan, 2004; Janssen and Scheffer, 2004; Thompson, 2004; Wilkinson, 2004; Weiss and Bradley, 2002; Chew, 2001; De Menocal, 

2001; Fernandez-Armesto, 2001; Tainter, 2000, 1996, 1988; Eisenberg, 1998; Ponting, 1991; Catton, 1982; Hughes, 1975). That is, past societal collapses have been triggered by the crossing of K 

to Ω thresholds, either as the result of human behavior (e.g., deforestation or damaged soils due to 

agricultural practices) or ecological transitions (e.g., droughts). 

Returning now to the example of energy as an indicator of sustainable or unsustainable development, 

it would appear that K to Ω thresholds have been crossed or are in the process of being crossed 

throughout Freese’s model: 

► Ecosystem Energy Production: Peak Oil. Energy in the form of sunlight drives photosynthesis 

in plants, which purifies the air, helps to regulate climate and provides food for life forms on Earth. 

Human societies convert ecosystem energy flows into resources such as food, shelter and energy, and 

to power a wide range of other activities. Human societies depended on renewable, solar powered 

biomass for fuel, shelter, tools, and other items for most of human history. 

In contrast to sunlight and biomass, fossil fuels—oil, natural gas and coal—are a onetime endowment 

of nonrenewable resources. The conditions under which oil and natural gas were formed were quite 

specific and “Oilfields cover less than 0.1 percent of the continents and continental shelves” (Deffeyes, 2005: 17). Oil reservoirs formed over hundreds of millions of years. The first oil boom began 

in 1859 in Pennsylvania. In the 148 years since, societies have become utterly dependent on fossil 

fuels, particularly oil, for energy and a large range of other uses. 

Researchers such as M. King Hubbert (1976), Kenneth Deffeyes (2005) and Colin Campbell (2002) 

think Earth’s total endowment of oil is slightly more than 2 trillion barrels. Furthermore, these researchers believe that world oil production will soon cross the K to Ω threshold, it will “peak” in the 

near future, if it hasn’t already (worldwide oil production appears to have hit a plateau of 85 million 

barrels produced per day in 2005. It has not crossed above that level since). Peak oil, which refers to 

the moment when half of the oil that ever existed is removed from the ground, means that the supply 

of readily available oil and oil-related products will decrease, while demand around the world continues to increase. Peak oil has implications for how we live, where we live, what we can do and where 

we can go. For example, the price of oil recently reached an all-time high of $110 a barrel and the 

implications of this fact are ratcheting throughout the world.

► Human Energy Expropriation: Overshot Carrying Capacity. For the majority of human 

existence we lived in small bands and subsisted by gathering plants and hunting animals, that is, our 

ancestors depended on renewable biomass for fuel, food, tools, and other items for most of history. 

With the domestication of certain plants and animals between 10,000 and 12,000 years ago, our 

population grew. However, it took the harnessing of nonrenewable energy power—first coal and then 
6

Draft March 27, 2008 Draft

oil—for world population to top 1 billion around 1900. Since then, world population has skyrocketed with an additional 5 billion people. 

The harnessing of fossil energy power, especially oil, radically changed the trajectories of social systems and ecosystems. The West, and particularly the United States, secured its power and influence 

throughout the 20th Century by securing increasing amounts of energy. At the same time, human 

impacts on the environment have amplified. For example, the United Nations recent Millennium 

Ecosystem Assessment Synthesis Report, subtitled Our Human Planet (2006), revealed that people are 

transforming ecosystems throughout the world at a faster and more extensive pace than any other 

time in human history. Of particular significance, this compilation of expertise from more than 

2,000 authors and reviewers finds that 60 percent (15 out of 24) of the major ecosystems examined 

are being used unsustainably; the changes being made to these ecosystems are increasing the likelihood of “nonlinear” changes (e.g., the emergence of diseases and unpredictable events); and poor 

people are disproportionately being impacted by the harmful effects of ecosystem degradation. The 

litany of problems identified by the United Nations and many other sources provide evidence that a 

series of thresholds have been passed or have the potential of being passed, culminating with the notion 

that humans have exceeded the carrying capacity of the planet (Catton, 1982).

► Ecosystem Sustainability/Disorganization: Climate Change. Ecosystems such as oceans 

and forests, Costanza et al (1997: 254) explain, provide the full range of support functions necessary for life on Earth. Costanza et al., place the value of Earth’s services at somewhere between U.S. 

$16–54 trillion per year, while global gross national product total is around U.S. $18 trillion per year 

(1997: 253). At the same, time, as the Millennium Ecosystem Assessment and other sources have made 

clear, human disturbances are increasingly damaging and disorganizing the services that ecosystems 

provide. 

Ecosystem sustainability in this example of energy use has less to do with the impacts of oil extraction or the familiar instances of oil spills. Rather, the combustion of fossil fuels has released stored 

carbon and other gases into the atmosphere, warming the planet and upsetting the balance of ecosystems. For most of human evolution, Earth’s climate was generally unfavorable to widespread dispersal 

outside of Africa- ice sheets covered large expanses of the continents. For the past 10,000 years, however, Earth has experienced an interglacial known as the Holocene, or “long summer”, with a global 

average surface temperature of about 57°F (Fagan, 2004; Flannery 2005). It is during this temperate 

window that every continent except Antarctica was settled and everything we count as human civilization developed: domestication of major plants and animals, agriculture, cities, written language, 

etc. (Kolbert, 2006; Flannery, 2005; Diamond, 1997). That is, long summer conditions have been 

very favorable to energy expropriation (e.g., a warmer climate and longer growing seasons), subsistence organization, and societal development. 

With the combustion of fossil fuels to power development since the Industrial Revolution, another 

K to Ω threshold is in the process of being crossed. Atmospheric carbon dioxide levels have gone 

from approximately 270 parts per million prior to industrialization to nearly 400 parts per million 

currently and rising (Dyson, 2005). In 1998, scientists in Antarctica drilled the deepest ice core ever 

recorded and found that carbon dioxide and methane levels are higher now than they have ever been 

in the past 420,000 years, going back farther than the Holocene (Petit et al., 1999: 433).
7

Draft March 27, 2008 Draft

This increase in greenhouse gases is transforming ecosystems by changing Earth’s climate: melting 

glaciers and ice sheets are raising ocean levels, altering weather patterns (e.g., increasing the frequency 

and severity of hurricanes, see Emmanuel, 2005), and changing the composition of local plants and 

animals (i.e., pushing many species to extinction, see Flannery, 2005). The 1990s were the hottest 

decade on record. 2005 was the warmest year on record. Global mean surface temperature is projected 

to increase over the next century, exacerbating all of these transformations.

► Subsistence Organization: For all of human history until the late 1690s, humans used solar 

power, biomass, wind power, animal power, and muscle for warmth, lighting, cooking, food production, travel (including sailing), and other activities. Then, in the late 1690s steam power was tinkered 

with. From there, a steady onslaught of technologies for energy conversion developed: high pressure 

steam engines, batteries, electricity, combustion engines, dynamite, refrigerators, rockets, nuclear 

reactors, etc. (Smil, 1994: 259-269). The hard path described by Amory Lovins has materialized: 

the energy industry, with support from governments, has built an “enormous network of oil wells, 

supertankers, pipelines, coal mines, power plants, transmission lines, cars, trucks, trains, and ships” 

(Roberts, 2004: 2) that supplies homes, farms, stores, buildings, factories, and vehicles with power. 

Subsistence in the United States has also been predicated on converting forests, prairies, and other 

landscapes into farms, orchards, grazing range, and other food producing areas. The use of oil to 

power mechanized equipment (e.g., tractors), the development of refrigeration, and a transportation 

infrastructure meant that fewer farmers were required to grow more food that could reach dinner 

tables at farther and farther locations. The implication, again, is that a house of cards has been constructed on the foundation of a nonrenewable resource. As the other processes in Freese’s model start 

to enter a K to Ω phase, it follows that human subsistence patterns will have to change.

► Societal Development: An amplification in the amount of energy expropriated both enables and is a requirement of more complex forms of political, urban, and economic organizations. 

According to Freese, in sedentary societies political organization became a means to “domesticate 

people”. With increased food surpluses and parcels of land brought into the fold, political organization 

became a “vessel for socially organizing and institutionalizing in human society, by means of force if 

not genuine authority supported by cultural values, systems of social inequality based on the control 

of valued resources and the means to produce, protect, and distribute them” (1997a: 171). From their 

origins as ceremonial meeting places, cities began to take hold wherever Neolithic agriculture, which 

is to say, intensifications in energy expropriation, became successful. Beyond organizing for survival, 

the city became a focal point for economic, political, and cultural activity, drawing in functions that 

had previously been scattered (Mumford, 1989 [1961). Finally, larger and more complex economic 

organizations developed as humans provisioned themselves for subsistence and survival with greater 

amounts of energy. Toman and Jemelkova assert that “Expanded availability and use of energy services is strongly associated with economic development” (2003: 94). 

Natural and geographical inequalities (the origins of which were outside of human control) laid 

the foundation for political, urban, and economic developments–including agents of conquest 

such as “steel swords, guns, ocean-going ships, political organization, writing, and epidemic diseases” 

(Diamond, 1997)–that Europeans used to secure and enhance land, resources, and other rewards. 

How did they do this? Tilly (1999) theorizes that rewards from resources are secured and enhanced 
8

Draft March 27, 2008 Draft

through exploitation and opportunity hoarding. Exploitation operates when powerful, connected 

people command resources (natural and human-made) from which they draw significantly increased 

returns by coordinating the efforts of outsiders whom they exclude from the full value added by that 

effort (1999: 10). Europeans and Americans systematically expropriated resources from Native Americans, while simultaneously committing genocide. Africans were enslaved in order to boost ecosystem 

energy production in the form of crops, which Europeans and Americans converted into economic 

gains. It is no coincidence that Mohai (1995) reports that “Nearly all of the studies over the past two 

decades have found that environmental hazards under investigation are inequitably distributed according to race or income or both” (615). 

Contemporary industrial societies and their economic systems have been built on “a central belief 

that they can progress by conquering nature and expanding production” (Schnaiberg and Gould 

1994: v). Over the past two decades, Allan Schnaiberg and his colleagues have elaborated and refined 

the “treadmill of production” model, characterizing the society-environment relation as an inexorable 

process in which economic actors (primarily corporations) use and degrade ecological resources in 

order to increase their accumulation of capital. Failure to do this would threaten profitability and 

the survival of firms (who are also driven to enhance profits by cutting labor costs and investing in 

capital-intensive technologies that, in turn, further increase the withdrawals of resources and energy 

from, and increased additions of waste and pollution to, ecosystems). Because corporate managers 

are constrained by the demand for expanding profits, there is enduring pressure to externalize true 

ecological and social costs. 

When corporations are prodded to account for these costs (e.g., by governments, social movement 

organizations, public opinion) they tend to use their considerable influence (e.g., political lobbying 

and campaign contributions) to resist change (Schnaiberg and Gould 1994: 46). At the same time, 

workers (in order to have and maintain jobs) and governments (in order to provide for “national 

development” and “social security”) are also dependent on the treadmill of production and, therefore, must work to facilitate its expansion (Schnaiberg and Gould 1994: 69). Since the treadmill of 

production is strongly anti-ecological and societies around the planet are caught up in its continued 

operation, Schnaiberg and colleagues foresee enduring conflict between the economy and the environment at all levels: international, national, regional, local, and interpersonal.

Oil literally and figuratively greases the wheels of the treadmill of production. The ten largest corporations in the world, including six of the ten most profitable corporations in the world, and consultants paid by the oil industry such as Daniel Yergin, have a vested interest in debunking peak oil and 

climate change.

► Collapse: The geographer and historian Joseph Tainter defines collapse as “a rapid, significant loss of an established level of sociopolitical complexity” (1988: 4). Biogeographer Jared Diamond 

characterizes collapse as “a drastic decrease in human population size and/or political/economic/

social complexity, over a considerable area, for an extended time” (2005: 3). In the run-up to and 

aftermath of collapse, societies experience a breakdown of authority and governance (including law 

and order), reduced trade and construction, population dispersal (from greater to lower density), and 

a general return to local self-sufficiency to meet the needs of a smaller population. In other words, 

the “overarching structure that provides support services to the population loses capability or disappears entirely” (Tainter, 1988: 20). 
9

Draft March 27, 2008 Draft

With the emergence of global climate change and global peak oil production, the old set of factors–

resource depletion, ecosystem degradation, and climate change–have the potential to wreak havoc 

in new ways, as they impact every society on the planet at the same time, in ways that amplify other 

problems, and that realistically pose the threat of global societal collapse. In The Long Emergency, for 

example, James Kunstler predicts a future with severe losses “in population, in life expectancies, in 

standards of living, in the retention of knowledge and technology, and in decent behavior” (2005: 5).

► Reorganization: The preceding tour of Freese’s model of sustainable development described 

a perfect storm brewing: oil is the key energy resource used throughout the world, but ecosystems 

produced a nonrenewable amount that is rapidly being depleted by a growing human population. 

Human energy expropriation, particularly the combustion of fossil fuels, is warming the planet and 

consequently impairing ecosystem integrity. 

The implications of this are of no small consequence since complex, technologically advanced societies have depended on ever increasing energy expropriation to power societal development, and a temperate window–the long summer–in which to develop. Nevertheless, biosociocultural reorganizations 

are already taking place in Freese’s model and the real world: ice sheets are melting, Alaskan villages 

are being relocated, coral reefs are dying, island nations are eyeing rising ocean levels, etc. But we do 

not know how human subsistence will ultimately be reorganized. 

Sustainable development advocates believe that a less apocalyptic future is possible. In the specific 

example of energy, societies can only meet the needs of the present without compromising the 

ability of future generations to meet their needs by conserving energy, using it more efficiently, 

and through a switchover to renewable energy (e.g., wind, solar, geothermal, hydro, biomass, 

biofuels, ocean tidal, waste, etc.).

Unsustainable development, then, refers to societal dependency on an economic process–the 

“treadmill of production”–that requires increasing withdrawals of energy and material from 

ecosystems (and the generation of waste and pollution) in order to promote societal progress 

for some segments of society while further impoverishing other segments of society (Schnaiberg 

and Gould, 1994). 

Whether referring to energy, water, forests, fish, minerals, or any other material or ecosystem service 

or flow, meeting the needs of the present without compromising the ability of future generations to 

meet their needs requires avoiding crossing K to Ω thresholds. Literature reviews in the fields of environmental sociology, ecological economics, environmental history, industrial ecology, and the emerging field of ‘sustainability science’, along with reports from governments, nongovernmental organizations, and other sustainable development advocates, reveals an interconnected core of principles 

that provide a kind of game plan for avoiding thresholds while defining the contours of sustainable 

development: 

► Social and Environmental Justice: Advocates of social and environmental justice stress the 

need to promote opportunity and equity within our society, between societies, between generations, 

and between species.
10

Draft March 27, 2008 Draft

► Stewardship of Built and Natural Environment: Stewardship advocates invoke Aldo Leopold’s well known concept of the land ethic to stress the need to maintain the resiliency of life support systems such as forests in perpetuity (Hawken et al., 1999), while historic preservation advocates 

and others stress the need to maintain the fabric of communities.

► “Ecological Modernization”: Advocates of ecological modernization, or natural capitalism (Hawken et al., 1999) stress the need to green societal institutions (e.g., economic institutions, 

political institutions) through changes in values, as well as changes in technology, such as biomimicry 

(Benyus, 1997), renewable energy, and “cradle-to-cradle” production processes (Mol and Sonnenfeld, 

2000).

Sustainable Development Economic Development

Renewable Energy: wind, solar, geothermal, 

hydro, biomass, biofuels, ocean tidal, or waste

Nonrenewable Energy: Fossil fuels, nuclear, 

CO2 emissions

Energy Efficiency / Conservation Inefficiency / Waste

Ecological Design: green building, smart growth, 

New Urbanism, historic preservation

Sprawl: low density, car and oil dependent, 

Sustainable Agriculture: locally produced, 

organic, CSAs 

Industrial Agricultural: 3,000 mile salad, pesticides, synthetic fertilizers, consolidations

Sustainable forestry: FSC certification Unsustainable forestry: deforestation, illegal 

logging

Localism: Employee ownership, livable wages, 

fair trade

Globalization: sweatshops, fossil fuel dependent

Environmental Studies: Ecological economics 

/ Environmental Education / Environmental 

Sociology, etc.

Economics: “fallacies of misplaced concreteness” 

in “the market”, “invisible hand”, and indicators 

such as GDP

Ecotourism / Sustainable Tourism: local ownership, conservation, low visitor impactsTourism: corporate ownership, adverse social, 

cultural and, environmental impacts

For more examples, visit Conservation Economy (http://www.conservationeconomy.net)
11

Draft March 27, 2008 Draft

Combining these three principles, the Natural Research Council says that the primary goals of a transition toward sustainability over the next two generations should be to meet the needs—food, water, 

energy, clothing, shelter, security, employment, education, health care, belonging, worth, community, etc.—of a much larger but stabilizing human population, to sustain the life support systems of 

the planet (e.g., ensuring the quality and supply of fresh water, controlling emissions into the atmosphere, protecting the oceans, and maintaining species and ecosystems), and to substantially reduce 

hunger and poverty by ensuring income growth, employment opportunities, and essential safety net 

services (1999: 31; see also Holdren, Daily, and Ehrlich, 1995). 

Criticisms of Sustainable Development

Since its debut, the Brundtland Report definition has been criticized for being too vague while, at 

the same time, sustainable development has been accused of suffering from too many definitions and 

a lack of consistency (Hoffman and Bazerman, 2007; Lele, 1991). For example, Moffatt et al. (2001) 

report that over 100 definitions can be found in the literature on sustainable development. The vague 

/ lack of definition critiques, however, have the unmistakable odor of a red herring. As the preceding 

example of energy demonstrated, there is a substantial body of knowledge on the meaning of sustainable development. At the same time, most definitions are variants of the conceptualization used by 

the Brundtland Report: they repeat the same concepts (i.e., avoid critical thresholds by integrating 

ethical, economic, social, and ecological concerns). For example: 

► Hawken defines sustainability as “an economic state where the demands placed upon the environment by people and commerce can be met without reducing the capacity of the environment to 

provide for future generations” (1993: 139). 

► Common writes that “To sustain is to support without collapse...the sustainability problem is 

taken to be: how to address problems of inequality and poverty in ways that do not affect the environment so as to reduce humanity’s future prospects” (1995: 1). 

► Elkington says, “It’s the principle of ensuring that our actions today do not limit the range of 

economic, social, and environmental options open to future generations” (1998: 20). 

► Meadows suggests that “sustainable development is a social construct, referring to the long-term 

evolution of a hugely complex system—the human population and economy embedded within the 

ecosystems and biogeochemical flows of the planet” (1998: 7). 

► Berke and Conroy describe sustainable development as “a dynamic process in which communities 

anticipate and accommodate the needs of current and future generations in ways that reproduce and 

balance local social, economic, and ecological systems, and link local actions to global concerns” 

(2000: 23).

► Dryzek astutely points out that sustainable development is premised on the notion that “the 

legitimate development aspirations of the world’s peoples cannot be met by all countries following 

the growth path already taken by the industrialized countries, for such action would over-burden the 

world’s ecosystems;” however, since improving the conditions of the world’s poor is a desired inter-
12

Draft March 27, 2008 Draft

national goal, economic growth should be promoted “in ways that are both environmentally benign 

and socially just. Justice here refers not only to distribution within the present generation, but also to 

distribution across future generations” (1997: 129). 

Another type of criticism / observation reflects on the fact that “Sustainable development is nowhere 

an accomplished fact” (Dryzek, 1997: 123). That is, “the problem of envisioning a living sustainable 

community” is the “absence of any concrete examples” (Hempel, 1999: 44). But why is this really the 

case? 

Schnaiberg and Gould correctly assert that just “espousing sustained development as a goal is insufficient to set in motion the societal processes that will lead to its dominance over competing goals” 

(1994: 3-4). What competing goals? Sustainable development fundamentally challenges the profit 

maximizing paradigm of the treadmill of production: there is “a contest between the tenets of capitalism and the tenets of sustainability” (Hoffman and Bazerman, 2007: 90, my emphasis).

Seen in this light, it was not vagueness that led the “World Bank and other economic development 

agencies” to misinterpret “sustainable development as perpetual growth, which is an extreme perversion of the original sense of the phrase” (Ayres, 1998: 2). Rather, the sustainability discourse is 

contested “by a variety of interests in a variety of ways as a means of supporting and enhancing their 

basis of power” (van der Duim, 2004: 368). As will be referenced in the sustainable tourism section, 

economic development interests, to date, have trumped sustainable development interests in every 

case cited.

Additionally, “Organizational arrangements and cultural beliefs tend to perpetuate unsustainable behavior” (Hoffman and Bazerman, 2007: 97). Within organizations, “authority to make and implement decisions is fragmented among a large number of agencies and organizations at multiple governance levels (municipality, state, federal)...Commonly, coordination among departments, agencies, 

and levels is limited or absent” (Beratan et al. 2004: 180). Even if managers, planners, and decisionmakers wanted to move toward sustainability, many are constrained by a “lack the resources—time, 

staff, training, and money—required to change established procedures” (Beratan et al. 2004: 181).

What is the Triple Bottom Line?

The idea of sustainable development has been broadly applied, from agriculture, forestry, and fishing, to businesses, product design, and educational curricula. A corresponding “development of 

indicators has emerged as a “best practice” in moving towards sustainability” (Beratan et al., 2004: 

184). Sustainability indicators have been used at the international, national, community, and business level. In the business community, the “triple bottom line” concept is generally used to refer to 

the measurement of a business’s performance based on three criteria: profitability, social equity, and 

environmental health. It is meant to be used to assess the complete impact of a business. A growing 

number of business success stories (e.g. Seventh Generation, http://www.seventhgeneration.com) 

demonstrate that such integration offers a significant competitive advantage. This advantage takes 

place through innovative product and service designs that eliminate inefficiencies and waste disposal 

costs inherent in many conventional business operations. 
13

Draft March 27, 2008 Draft

The gist of the triple bottom line argument is that social and environmental performance “can be 

measured in fairly objective ways, and that firms should use these results in order to improve their 

social (and environmental) performance. Moreover, they should report these results as a matter of 

principle, and in using and reporting on these additional “bottom lines” firms can be expected to do 

better by their financial bottom line in the long run” (Norman and MacDonald, 2004). The triple 

bottom line is seen as “a metaphor to remind us that corporate performance is multi-dimensional” 

(Pava, 2007: 108).

The phrase ‘triple bottom line” is credited to John Elkington, particularly with the publication of 

Cannibals with Forks in 1998. Elkington was looking for a term that resonated with “business 

brains”. But the idea of a switchover in the way that corporations do business is not a new one. There 

are several published instances of ‘coming to the light’ moments, where CEOs describe how and why 

they came to the conclusion that they needed to change their businesses. The most famous example 

is probably Ray Anderson’s Mid-Course Correction, in which he describes how his “haunting role” in 

the devastation of the planet led him to radically change Interface (http://www.interface.org).

There is also a growing literature about how industrialized countries (or segments within these 

countries) such as Germany, Japan, Sweden, Norway, the Netherlands, and the United States appear 

to be going through a process of “ecological modernization.” Here it is suggested that high energy efficiency of national income (in terms of the amount of energy required to produce a unit of national 

income); low per capita emissions of pollutants; low per capita generation of household garbage and 

other solid wastes (Dryzek, 1997: 137) indicate that ‘free market capitalism’ can be transformed and 

redirected in such a way “that it less and less obstructs, and increasingly contributes to, the preservation of society’s sustenance base” (Mol and Spaargaren, 2000: 23).

For ecological modernization proponents, the “ecologization of the economy”, and society more 

generally, makes it possible for “an ecological switchover to take place which marks the end of a 

period of unspecified economic growth and this will result in nothing less than an ecological reconstruction of modern society’s institutional organization” (Mol and Spaargaren, 1993: 437). Elements 

in this switchover include the development and use of cleaner, more efficient technologies, the use of 

anticipatory (and participatory) planning practices (the precautionary principle), the internalization 

of externalities, greening of the marketplace and consumption, a broad revolutionizing of popular 

(environmental) participation in every aspect of social life, and strict government regulations (Cohen, 

1997: 109). Examples of ecological modernization in the United States can be found in Natural 

Capitalism, by Paul Hawken, Amory Lovins, Hunter Lovins (http://www.natcap.org), as well as McDonough and Braungart’s Cradle-to-Cradle (http://www.mcdonough.com/cradle_to_cradle.htm). 

In line with the theory of ecological modernization, Elkington (1998) identifies seven revolutions 

that he thinks are making businesses think differently about sustainability: 

The first revolution is in markets, where Elkington believes “business will shift from using competition as an excuse not to address the triple bottom line agenda to a new approach, using the triple 

bottom line as part of the business case for action and investment”(1998: 4-5). 
14

Draft March 27, 2008 Draft

A second revolution is said to be taking place in values (i.e., a worldwide shift in human and societal 

values towards sustainability). Elkington such shifts in values are among the most powerful influences faced by politicians and business leaders alike (5-7). 

A third revolution, increasing scrutiny worldwide, is forcing organizations to respond to environmental and social concerns (Elkington, 1998: 8). These pressures include: 

•	 compliance; 

•	 punitive fines and costs; 

•	 personal culpability and imprisonment; 

•	 environmental activist organizations; 

•	 other coalitions; 

•	 international codes for environmental performance; 

•	 environmentally conscious investors; 

•	 consumer preference; 

•	 global markets; 

•	 global politics and international organizations; 

•	 competition; 

•	 attracting good workforce (Kinlaw, 1993: 24-26) 

Like the compelling examples provided in Natural Capitalism or Interface’s recyclable carpet tiles, 

Elkington says that the fourth revolution is that companies are increasingly focusing on “life-cycle 

technology”. 

Increased partnerships between corporations, government and non-governmental organizations, 

and thinking in terms of longer time horizons are the basis for the fifth and sixth revolutions (1998: 

11-12). 

Finally, the seventh revolution that Elkington sees is improved corporate governance: “the better the 

system of corporate governance, the greater the chance that we can build towards genuinely sustainable capitalism” (1998: 12). 

Willard writes that the “trick” to turning these revolutions into action “is to focus on the “selfish” 

bottom-line benefits, not the seemingly altruistic society and environmental results” (2002: 12). But, 

there are, in fact, a variety of reasons that firms might behave in this way. Some embrace environmental concerns and stewardship as a basic principle (e.g., Ben & Jerry’s Ice Cream, Seventh Generation paper products, Interface). Some adjust along with or ahead of changing environmental and 

technological conditions (e.g., British Petroleum, 3M, the Pew Business Environmental Leadership 

Council). Others profit from new paradigms and rethinking business practices (see the cases cited 

in Hawken et al. 1999). Some organizations respond to changing government, professional and 

movement-induced standards (e.g., ISO standards, the U.S. Green Building Council’s LEED rating 

system, the certified production of wood products), and others respond to market pressures and opportunities (e.g., pollution trading, demand for renewable power). Case studies reported in Changing Course (1992), also indicate that senior management commitment to sustainable development is 

essential in setting priorities for the company. A similar commitment must be made at each facility 

owned by the company and each employee must be involved in the accomplishment of environmen-
15

Draft March 27, 2008 Draft

tal goals (Schmidheiny, 1992: 192).

The start of a more thorough list of reasons for why a business might embrace triple bottom line 

thinking include:

1. Anticipating the inevitable (changes in business climate, regulation, etc., resulting from environmental harm, new legislation/regulation, changing global standards of performance/best practices, 

ISO certification, etc.) helps corporations get ahead of the competition (Kinlaw, 1993; Elkington, 

1998; Nitkin and Brooks, 1998; Willard, 2002).

2. Better “reputation management” is believed to lead to easier hiring of the best talent and higher 

retention of top talent (Willard, 2002). Better employee morale, in turn, is said to lead to: 

3. Increased employee productivity, improved productivity, and worker creativity (World Conservation Union, 1991; Willard, 2002). Employee productivity, morale, and creativity also improve when 

the indoor environment they work in is healthy and comfortable (e.g., improved indoor air quality, 

daylighting). Green buildings themselves typically sell or lease faster than conventional developments 

(Hawken et al., 1999). 

4. Reducing compliance (e.g. fines), litigation, and cleanup costs (Kinlaw, 1993: 5). Reduced liability 

insurance and costs, as well as maintaining eligibility for less expensive insurance (World Conservation Union, 1991). Reducing risks of product liability; reducing risks of major environmental 

disasters; lower employee, public, and environmental risks and expenses, both present and future, 

and a better reputation leads to easier financing and greater credibility with banks and other financial 

institutions (World Conservation Union, 1991; Kinlaw, 1993; Nitkin and Brooks, 1998; Willard, 

2002). 

5. Reduced expenses in manufacturing and at commercial sites (Willard, 2002). Reducing the 

amounts of materials used (Kinlaw, 1993) leads to lower costs of raw materials (World Conservation 

Union, 1991). Replace nationally and internationally produced items with products created locally 

and regionally. (Hawken, 1993: 144) can cut transportation costs. And stronger locally/regionally 

based supply chains and trade ally relationships create greater predictability, greater resiliency (less 

threat of disruption), and increased network efficiencies. 

6. Reducing (and recapturing) energy expenditures (World Conservation Union, 1991). Creating 

earnings from energy peak demand reduction contracts and/or local electricity/heat generation (i.e., 

selling energy saved to third party customers or back into the grid. 

7. Making money from prevention (design, consulting, green architecture, etc.) as well as making 

money from cutting-edge environmentally-relevant research and development. 

8. Reducing the costs of waste handling and disposal costs; turning wastes into outputs (i.e., capturing value now discarded) (World Conservation Union, 1991; Kinlaw, 1993; Nitkin and Brooks, 1998; 

Hawken et al., 1999).
16

Draft March 27, 2008 Draft

9. Making money from clean-up (e.g., environmental mitigation, restoration, and research and 

development on new environmental technologies). 

10. Maintaining market shares with old customers who want more environmentally friendly products as well as creating new distribution opportunities to new customers who are more environmentally sensitive.

11. Building new centers of technical expertise in the firm and network (e.g., expertise in green 

building, waste reduction, etc.) that can turn into new profit centers (e.g., providing consulting/design and/or outsourced services to other firms). 

12. Ability to diffuse learning and efficiencies (via energy savings, waste reduction, new supply chain 

connections, process innovations, etc.) within the firm. 

13. Synergies that develop, with new products, production processes, trade alliances, supply chain 

expansion, etc., once a critical mass of “sustainable” business activities are underway and linked with 

one another. 

Criticisms of the Triple Bottom Line

The literature on the triple bottom line is sparse in comparison to sustainable development. As one 

indication, a Google search for “sustainable development” yielded 17,200,000 hits, while a search for 

“triple bottom line” generated 463,000 hits, or 2.7% of the hits received by sustainable development. 

Significantly, John Elkington’s company is called SustainAbility (http://www.sustainability.com). In 

one interview, Elkington even says, in reference to the Global Reporting Initiative, “To be honest, 

I don’t terribly like standards; I don’t like that whole game” (http://www.johnelkington.com/downloads/twentyyearsafter.pdf).

The main criticism of the triple bottom line is that “it is difficult to find anything that looks like 

a careful definition of the concept, let alone a methodology or formula (analogous to the calculations on a corporate income statement) for calculating” the social and environmental bottom lines 

(Norman and MacDonald, 2004: 245). Norman and MacDonald say “Probably the most curious 

fact about the 3BL movement–certainly the one that surprised us most as we researched it–is that 

none of the advocates of so-called 3BL accounting ever actually proposes, presents or even sketches 

a methodology of the sort implied” by the idea that we can arrive at an aggregated triple bottom line 

(2004: 248 ). They conclude that the concept of the triple bottom line is nothing more than “Good 

old-fashioned Single Bottom Line plus Vague Commitments to Social and Environmental Concerns” 

(2004: 254).

The literature analyzing triple bottom line pronouncements generally supports this conclusion, 

although results are mixed since there is no agreed upon triple bottom line accounting system, triple 

bottom line indicators are frequently called sustainability indicators, and most triple bottom line 

documents look like corporate social responsibility reports. For example, comparing 652 U.S. manufacturing firms between 1987-1996, King and Lenox (2001) found evidence “of a real association 

between lower pollution and higher financial performance. We also show that a firm’s environmental 
17

Draft March 27, 2008 Draft

performance relative to its industry is associated with higher financial performance” (2001: 106). 

A review of the experience of 174 of Canada’s largest 1500 public and private sector corporations 

that have begun to incorporate sustainable development management and reporting found that:

“Sustainable auditing is neither mandated nor common practice in Canada. There are a relatively large 

number of environmental progress reports in Canada. Some are very promising in terms of stakeholder inclusiveness and attempts to deal with sustainable development issues. There is little standardization of such 

reports and little independent opportunity to correlate external reports with what is audited internally”

(Nitkin and Brooks, 1998: 1506).

An analysis of 84 assessments of special events (e.g., the Olympics) found that “Notwithstanding 

the numerous calls for a broader evaluation framework, there have been few attempts to make the 

conceptual link between the evaluation of special events and the concept of the TBL” (Sherwood et 

al.). This analysis found that “the predominate method of evaluating events was from an economic 

standpoint” and the overall tendency was to “include social costs as a counterbalance to the mostly 

positive economic impacts rather than as an integration of impacts.” 

Governments and municipalities around the world have also adopted the triple bottom line as a reporting mechanism, but it is unclear what the end result has been. For example, the Australian Government’s Department of Family and Community Services made a commitment to the triple bottom 

line in 2002 (http://www.facs.gov.au/triplebottomline/2004/index.htm). Using Global Reporting 

Initiative guidelines, the Department developed indicators and produced reports for 2002–03 and 

2003-04 (http://www.facs.gov.au/triplebottomline/2004/_lib/doc/facs_tbl.pdf). No further updates 

have subsequently been provided.

In 2006 the City of Calgary has also adopted a triple bottom line policy framework, this time using the “Melbourne Principles for Sustainable Cities”, and arrived at a different set of indicators of 

relevance. Based on Calgary’s website, however, it is not clear what the city has done with its policy 

framework over the past two years. 

(http://content.calgary.ca/CCA/City+Hall/Business+Units/Environmental+Management/Strategic+E

nvironmental+Initiatives/Triple+Bottom+Line/Triple+Bottom+Line.htm)

(http://www.calgary.ca/docgallery/bu/environmental management/tbl policy framework.pdf)

The International Council on Local Environmental Initiatives (ICLEI) is in the process of developing 

another indicator scheme for launch in 2008. The STAR Community Index will rate the sustainability of communities in a similar manner to the U.S. Green Building Council’s LEED program. 

http://www.iclei-usa.org/programs/sustainability/star-community-index

The appendix provides more examples of indicator schemes, but the bottom line is that there is no 

mutually agreed upon set of indicators for measuring the triple bottom line or sustainable development. It varies from project to project. 

2
18

Draft March 27, 2008 Draft

In contrast to sustainable development, the triple bottom line is a metaphor without a theory or 

explicit methodology. Although the terms are frequently used synonymously, they have different 

origins and bodies of knowledge to illuminate their meaning. Sustainable development refers to an 

interactive social-ecological process and a way to avoid crossing critical thresholds, namely by promoting social and environmental justice, practicing stewardship of built and natural environments, 

and greening the economy. The triple bottom is a metaphor that is meant to serve as an accountability tool. A semantic transition from sustainable development to triple bottom line is not advised, 

especially since the preponderance of theory and research is on the side of sustainable development.

Sustainable Tourism

“Sustainable tourism has become the dominant organisational paradigm of the global tourism industry” (Bianchi, 2004: 498). Barke and Towner write that:

“A broad consensus within the literature would suggest that a process of sustainable tourism development 

would be based on the substantial re-use of existing manmade and natural resources and would be associated with a low input of energy. It would also be viewed as a process founded in local cultures, producing 

an equitable distribution of services, managed and administered according to democratic principles, and 

maintaining and regenerating traditional social values and practices” (2003: 166).

However, research from Spain, the Canary Islands, Barbados, Indonesia, Costa Rica, and the Netherlands reveals that progress towards more sustainable forms of tourism activity is superficial at best 

(Barke and Towner, 2003; Bianchi, 2004; Butcher, 2006; Cole, 2006; Cottrell et al., 2004; Farrell 

and Twining-Ward, 2004; Liu, 2003; Mycoo, 2006; Schianetz et al., 2007; Stem et al., 2003; van der 

Duim, 2004; Vera Rebollo and Ivars Baidal, 2003). 

While benefits such as an increase in the average per-capita income, the expansion of employment 

opportunities, and the socio-economic advancement of women have been seen in some locations 

(Bianchi, 2004: 499), a partial list of negative indicators includes: environmental degradation (e.g., 

the death of coral reefs), air, water and land pollution, increased energy and water consumption, increased sprawl, economic exploitation of local populations, dislocation, policy failure or non-existent 

policy, low wages, job insecurity, high housing costs, and poor levels of public infrastructure provision. 

Sustainable development is being undermined in tourism locations because of a continuing commitment to growth oriented strategies and the treadmill of production. Bianchi argues that “the legacy 

of uneven development, and the entrenched power of regional economic and political elites, is likely 

to undermine the prospects for a just model of sustainable tourism, and to consolidate the continuing privatisation of space and socio-spatial inequalities across the region” (2004: 495).

The measurement of sustainable tourism is plagued with the same issues that surround the triple 

bottom line. In a detailed methodological review, Schianetz et al., (2007) argue that “no single tool 

addresses all of the environmental, social and economic issues at all levels and therefore a combination of different assessment tools may be required to answer specific questions pertinent to a project” 
19

Draft March 27, 2008 Draft

(380). Beratan et al. add that “it is not possible to develop an “ideal” set of indicators....One important objective of an indicator program therefore should be to create and strengthen communicative 

links among agencies, organizations and individuals with overlapping interests and responsibilities” 

(2004: 185).

Schianetz et al. identify seven tools for measuring sustainable tourism activities: Sustainability Indicators, Environmental Impact Assessment, Life Cycle Assessment, Environmental Audits, Ecological 

Footprints, Multi-Criteria Analysis and Adaptive Environmental Assessment.

► Sustainability Indicators: SI is the most broadly used and advocated tool to assess the sustainability of tourism destination, mainly because it is comparatively clear and simple to use at the 

many levels of a tourism destination (2007: 375).

► Environmental Impact Assessment: EIA’s are mainly used as a pre-project approval decisionmaking tool (e.g., for getting a permit. Schianetz et al. cite a study that found that between 1979 

and 1993 a total of 175 tourism developments in Australia were subject to EIA. However, the study 

found that the scientific quality of the conducted assessments was generally low and that the impact 

predictions were vague and unquantified. (2007: 377).

► Life cycle assessment (LCA): LCA’s are relatively complex, especially compared to SI, 

because it strives to include all possible input and output data over the whole life cycle of a product 

system. Schianetz et al. say that LCA’s have rarely been conducted for sustainable tourism projects 

(2007: 377).

► Environmental auditing (EA): EA examines the operations on a site (e.g., energy use) and, 

if necessary, identifies areas for improvement to management. The effectiveness of EA depends on 

the professional competence of the auditing team, the availability of data, and the follow through of 

managment staff. EA generally takes place at facilities such as hotels or tourism companies (2007: 

378).

► Ecological footprint (EF): As defined by Wackernagel and Rees (1996: 6), the EF is an 

‘estimate of resource consumption andwaste assimilation requirements for a defined human population or economy in terms of a corresponding productive land area’. The EF allows for relatively easy 

comparisons of environmental performance, but does not measure social issues. 

► Multi-criteria analysis (MCA): MCA has mostly been applied to environmental planning 

and project appraisal and to address conflicting objectives between stakeholders over the use of scarce 

natural resources MCA techniques allow comparison of alternatives, such as different design options 

or policy interventions, using a set of criteria and a method for ranking the alternatives. In principle, 

MCA goes beyond SI as it provides a method of evaluating data and indicators by using different 

procedures of data standardisation, ranking and weighting, but MCA depends highly on the expertise of the assessment team (2007: 379).

► Adaptive environmental assessment (AEA) / mediated modeling: AEA uses small collaborative workshops of scientists, decisionmakers and computer modelling experts to construct a 
20

Draft March 27, 2008 Draft

simulation model of the economic, social and/or environmental system likely to be affected by a 

development. Periodic workshops and the refinement of the model with newly available data initiates a learning cycle that promotes systems understanding and facilitates the exploration of management scenarios. AEA has only been applied to the assessment of large-scale developments a few 

times (2007: 380). An AEA/ mediated modeling process took place in Vermont for energy planning 

(http://publicservice.vermont.gov/planning/mediatedmodeling.html) and the results were fairly 

mixed since it is a very technical assessment.

The Green Globe 21 program (http://www.ec3global.com) also exists as a certification body for sustainable tourism operators, companies, and communities. The Green Globe 21 website indicates that 

the program specifically addresses: 

•	 Reduction in green house gas emissions;

•	 Energy efficiency, conservation and management;

•	 Reduction in the consumption of fresh water and resources;

•	 Ecosystem conservation and management;

•	 Support for local community development;

•	 Improved management of social and cultural issues;

•	 Improved land use planning and management;

•	 Improved air quality and noise reduction;

•	 Improved waste water management;

•	 Waste minimisation, reuse and recycling;

As a practical matter, Regional Technology Strategies will likely rely on sustainability indicators to 

evaluate the Appalachian Regional Commission’s programs. Barke and Towner suggest that the following indicators might be appropriate for evaluating sustainable tourism projects:

Sustainability Indicators Checklist

Indicator Sustainability Characteristics

Energy Maximize energy efficiency

Generate energy from renewable resources

Waste

Reduce waste

Encourage re-use and / or repair

Encourage recycling

Transport

Discourage use of cars

Encourage walking or cycling

Encourage use of public transport

Pollution Reduce or minimize local pollution - noise, air, 

water and land

Buildings and Land Use

Conserve and / or re-use older buildings

Provide local amenities

Improve access for disabled
21

Draft March 27, 2008 Draft

Indicator Sustainability Characteristics

Wildlife and Open Spaces

Encourage natural plant and animal life

Encourage use of open space for community 

benefit

Economy and Work

Increase local employment

Link local production with local consumption

Improve environmental awareness of local businesses and other key actors

Local Community

Involve local community in developments

Encourage local action and decision making

Recognize under-represented groups

(Barke and Towner: 2003: 168)

Vera Rebollo and Ivars Baidal use the well-know Pressure-State-Response framework to arrive at a 

different set of indicators:

Land Use-Tourism Model

Indicator Parameters

Tourist / Resource Attractions

Basic tourist resources

Average temperature

Length of coastline

National parks

Golf courses

Berths in marinas

Events of interest

Potential tourist resources

Better use of National parks

Remodeling of port

Health-oriented tourism (mud-baths)

Land Use

Land for residential use Square miles

Suburban sprawl versus concentrated areas for 

residential areas

Comparison of surface areas

Physical modifications of the coast Classification of coastal land uses

Economic Activity

Economic specialization Breakdown of sectors

Employment by sector Employment by sector

Official unemployment level Number of unemployed people

Demographic Structure

Increase in population Population levels
22

Draft March 27, 2008 Draft

Indicator Parameters

Origins of the resident population Population breakdown

The ageing of the population Age breakdown

Tourist-oriented Structure

Regulated accommodation offer Number of beds per hotel, apartment, etc.

Potential tourist accommodation available in 

private homes

Estimation of beds in second homes and apartments

Profile of demand Type of visitor

(Vera Rebollo and Ivars Baidal, 2003: 190-191).

Pressure Indicators

Indicator Parameters

Human pressure Population of municipality during peak period

Seasonal human pressure Urban garbage collection and water consumption during summer

Increase in land use for residential purposes Area of open space 

Increase in number of dwellings Average annual increase in dwellings

Increase in official supply of tourist accommodationAverage annual increase in tourist accommodations

Increase in urban garbage collection Average annual increase in urban garbage collection

Increase in water consumption Average annual increase in water consumption

Increase in consumption of electricity Average annual increase in electricity consumption

(Vera Rebollo and Ivars Baidal, 2003: 194).

State-quality Indicators

Indicator Parameters

Basic environmental measures Air pollution, noise pollution, water quality

Perceived quality of life Survey of residents

Tourist satisfaction Survey of tourists

(Vera Rebollo and Ivars Baidal, 2003: 195). 

Indicator Parameters

Actions on tourism resources Urban remodelling

Urban planning Regulation of Urban Plan

Protected non-urbanized land Acreage under protection

Tourism planning Publication of tourism planning document

Municipal budget Municipal budget
23

Draft March 27, 2008 Draft

Indicator Parameters

Green budget Percentage of budget devoted to garbage collection, gardens, street clean-up, beaches, pools

Waste water treatment Volume of water treated

Selective garbage collection Recycling 

Environmental surveillance and control Presence of environmental ordinances

(Vera Rebollo and Ivars Baidal, 2003: 197).

Issues with Indicators

Cobb writes that “The history of social measurement suggests that indicators have seldom been put 

to use when they are developed. Only occasionally has this form of knowledge led to action. The relevant issue, then, is the precise conditions under which indicators are most likely to be influential in 

shaping collective behavior” (2000: 15). Before “sustainability” can be implemented, those involved 

in the effort need to come to agreement on what is to be sustained, over what time period, and for 

whom. Answering these difficult questions requires broad public participation in crafting a widely 

acceptable vision of the future (Beratan et al., 2004: 180). The “Bellagio Principles” (www.iisd.ca/

measure/bellagio1.htm, see also Meadows, 1998) provide a widely used set of ten guidelines for assessing progress toward sustainable development. 

The first principle states that a clear vision of sustainable development and goals that define that vision need to be articulated in order to assess progress. This could be done, Meadows suggests (drawing on work done by Redefining Progress), by selecting a small working group responsible for the 

success of the project that would then “clarify the purpose of the indicator set” (e.g., ) and “identify 

the community’s shared values and vision” (1998: 26). 

The second principle encourages the promotion of a “holistic perspective” that stresses the fact that 

nature and community need to be sustained, while the economy and society need to be developed 

(primarily to meet the needs of the world’s poor). Both desired states can theoretically be achieved 

by promoting such “essential elements” (Bellagio principle three) as equity/opportunity, limits/

resilience, stewardship, qualitative development as opposed to growth, and social systems based on 

radical resource productivity, biomimicry, a service and flow economy, and increased investment in 

natural capital. Although the treadmill of production stands in the way of sustainable development, 

the Ford Foundation proposal can act as an enabling condition fostering progress towards sustainability in impoverished regions, and indicators of sustainability development can act as “leverage 

points” that influence the desired outcome (Meadows, 1998: 5). 

Principles four and five—maintaining an adequate temporal and spatial scope and a practical 

focus—are meant to provide clear signals of progress toward sustainability. The selected working 

group could review existing schemes, indicators and sources of data and draft a set of proposed indicators (Meadows, 1998: 26). Unfortunately, the Board on Sustainable Development of the National 

Academy of Sciences finds that “there is no consensus on the appropriateness of the current set of 

indicators or the scientific basis for choosing among them...[Past research] has yet to produce a set 

of goals for social and natural conditions that can plausibly lead to prosperity for all while conserving 
24

Draft March 27, 2008 Draft

the life support systems on which human economies rest” (1999: 243). Given the difficulty of selecting indicators Meadows recommends the “ten indicators or be shot at dawn” exercise to force people 

to come to an agreeement. 

Principles six through ten—Openness, effective communication, broad participation, ongoing 

assessment, and creating an institutional capacity for data collection, maintenance, documentation, evaluation, etc. —revolve around establishing a continuing capacity for assessment. Crucial 

to this process is presenting the draft indicators to a broad range of community participants for 

their input (Meadows, 1998: 27). The International Standards Organization (ISO) 14000 family of 

international environmental standards provide guidance on a number of issues- including helping to 

choose indicators to evaluate an organization’s environmental performance (see www.iso.org). And 

the Global Reporting Initiative (GRI) has come out with a detailed set of reporting principles and 

indicators meant to help organizations document and present their economic, social, and environmental performance (see www.globalreporting.org). The 11 reporting principles identified in GRI’s 

guidelines—transparency, inclusiveness, completeness, accuracy, clarity, relevance, neutrality, timeliness, comparability, and the sustainability context—allow for comparisons to be made over time and 

across organizations (GRI, 2002: 22). 

In another piece, Cobb and Rixford (1998) identify twelve lessons that they believe can help avoid 

mistakes that previous efforts have made: 

1) Having a number does not necessarily mean that you have a good indicator; 

2) Effective indicators require a clear conceptual basis (i.e., time needs to be spent clarifying exactly 

what is to be measured); 

3) There’s no such thing as a value-free indicator (cf., Meadows, 1998); 

4) Comprehensiveness may be the enemy of effectiveness (e.g., Oregon Benchmark’s “superabundance of indicators”); 

5) The symbolic value of an indicator may outweigh its value as a literal measure (e.g., GDP); 

6) Don’t conflate indicators with reality; 

7) A democratic indicators program requires more than good public participation processes (i.e., “an 

insistence on achieving a consensus of stakeholders or citizens usually produces a set of indicators 

that do little to challenge prevailing practices”); 

8) Measurement does not necessarily induce appropriate action; 

9) Better information may lead to better decisions and improved outcomes, but not as easily as it 

might seem; 

10) Challenging prevailing wisdom about what causes a problem is often the first step to fixing it 

(cf., Meadows, 1999, regarding leverage points); 

11) To take action, look for indicators that reveal causes, not symptoms; and 

12) You are more likely to move from indicators to outcomes is you have control over resources 

(1998: 14-29). 

Corporate Evaluation

Kinlaw suggest eight milestones for sustainable performance that program evaluators could presumably monitor to ensure that corporations were taking positive steps: 
25

Draft March 27, 2008 Draft

1) sustainable performance policy statement is published; 

2) sustainable performance baselines are established; 

3) initial sustainable performance training is accomplished; 

4) initial improvement projects are underway; 

5) development of environmental technologies are being supported; 

6) auditing and reporting system is functioning; 

7) coalitions with NGOs, government, etc., exist; 

8) management and human resource systems are revised to support sustainable performance (1993: 

23-24). 

Community Evaluation

Hart (1999) provides another method for determining the suitability of sustainability indicators. 

She created a 14-point checklist through which interested parties can assess how well an indicator 

addresses all facets of sustainable development. A “Yes” answer to any of the questions listed below 

counts as one point. Hart suggests that “In general, an indicator with a score of less than 4 will not 

be very useful for measuring sustainability...An indicator with a score of 7 or higher is, in general, a 

good indicator of sustainability” (1999: 140). 

Sustainable Community Indicator Checklist

1. Does the indicator address the carrying capacity of the community’s natural resources, both renewable and nonrenewable, whether local or from distant sources?

2. Does the indicator address the carry capacity of the ecosystem services upon which the community relies, whether local, global, or from distant sources?

3. Does the indicator address the carrying capacity of the aesthetic qualities—the beauty of nature—

that are important to the community?

4. Does the indicator address the carrying capacity of the community’s human capital—the skills, 

education, health and natural abilities of the people in the community?

5. Does the indicator address the carrying capacity of the community’s social capital—the connections between people in a community: the relationships of families, friends, neighborhoods, social 

groups, businesses, governments and their ability to cooperate and work together?

6. Does the indicator address the carrying capacity of the community’s built capital—the ability to 

maintain and enhance the community’s infrastructure, buildings, parks, playgrounds, and support 

systems—with existing resources?

7. Can the community at large understand and use the indicator?

8. Does the indicator provide a long-term view of the community?
26

Draft March 27, 2008 Draft

9. Does the indicator measure diversity—economic, social or biological?

10. Does the indicator measure equity—intergenerational or intragenerational?

11. Does the indicator measure a link between economy and environment?

12. Does the indicator measure a link between environment and society?

13. Does the indicator measure a link between society and economy?

14. Does the indicator focus on local sustainability at the expense of global sustainability? If the 

answer to this last question is “yes,” then the indicator is automatically disqualified (Hart, 1999: 35).

Finally, the Melbourne Principles for Sustainable Cities are the only internationally ratified set of 

sustainability principles for cities. They were developed in 2002 through an international charette 

process involving over 40 municipal and civic representatives from around the world. The process 

was carried out under the auspices of the United Nations Environment Program. Calgary used the 

Melbourne Principles to develop their triple bottom line policy framework (http://content.calgary.ca/

CCA/City+Hall/Business+Units/Environmental+Management/Strategic+Environmental+Initiatives/

Triple+Bottom+Line/Melbourne+Principles.htm)

Melbourne Principles for Sustainable Cities

Yes No If no, why?

If yes, how is it 

further related 

to operational 

plans?

Principle 1: Provide a long-term vision for cities based on sustainability; intergenerational, social, economic 

and political equity; and their individuality.

Does it have / align with a long-term 

vision? 

Does it define long term?

Does it have a definition of sustainability?

Are there means to review vision?

Are there means to monitor how the 

plan progresses to the vision?

Principle 2: Achieve long-term economic and social security.

Does it consider social, economic and 

environmental impacts in a transparent 

way?

Are the benefits of the plan shared with 

the community in a just and equitable 

manner?
27

Draft March 27, 2008 Draft

Yes No If no, why?

If yes, how is it 

further related 

to operational 

plans?

Are the negative effects of the plan 

shared with the community shared in a 

just and equitable manner?

Are the economic, social and environmental strategies consistent with your 

long-term vision?

Principle 3: Recognise the intrinsic value of biodiversity and natural ecosystems, and protect and restore 

them.

Does it protect the intrinsic value of 

ecosystems?

Are there means to monitor / measure 

the effect of the plan on biodiversity?

Principle 4: Enable communities to minimise their ecological footprint.

Does this transparently account for your 

impacts outside your boundaries?

Are there means to communicate the impact of your actions to the community?

Are there means to monitor / measure 

(e.g., with indicators) the impact of your 

actions with the community?

Will the plan proceed in the absence of 

scientific evidence about the extent of 

your impacts?

Principle 5: Build on the characteristics of ecosystems in the development and nurturing of healthy and 

sustainable cities.

Are there elements of the project that 

have been modelled on ecological principles?

Will the benefits of building upon the 

characteristics of ecosystem be communicated to the community?

Principle 6: Recognise and build on the distinctive characteristics of cities, including their human and cultural values, history and natural systems.

Does the plan consider / build upon cultural values, history and natural systems? 

Is the cultural / historical profile of your 

community embodied in the plan?

Is the plan compatible with the community’s cultural / historical profile?

Principle 7: Empower people and foster participation.
28

Draft March 27, 2008 Draft

Yes No If no, why?

If yes, how is it 

further related 

to operational 

plans?

Has the plan undergone community 

consultation?

Has community engagement been broad, 

and included typically marginalised 

voices?

Has the community’s participation influenced the strategy?

Are there means to continue community 

input throughout monitoring and evaluation of the plan?

Principle 8: Expand and enable cooperative networks to work towards a common sustainable future.

Have cooperative networks been established, facilitated, supported and / or 

consulted throughout the plan’s development? Will these networks be consulted 

throughout the monitoring and evaluation?

Are networks celebrated and publicised 

within and outside your boundaries? 

Have these networks enabled / facilitated 

collective action in the community?

Is the knowledge / learning of the networks widely shared?

Principle 9: Promote sustainable production and consumption, through appropriate use of environmentally 

sound technologies and effective demand management.

Does the plan / strategy promote sustainable consumption? 

Have environmentally sound technologies been employed in the design and 

implementation of the strategy / plan? 

Does it consider production / consumption costs along the supply chain?

Are there means to support other business engaging in sustainable production / 

consumption? 

Principle 10: Enable continual improvement, based on accountability, transparency and good governance.

Has decision making in the plan / strategy been undertaken in a transparent way?
29

Draft March 27, 2008 Draft

Yes No If no, why?

If yes, how is it 

further related 

to operational 

plans?

Does the plan contain indicators and 

targets to monitor continuous improvement?

Is good governance celebrated through 

the plan?

Does the plan contain reporting mechanisms that are based on accountable 

information?

Is continuous improvement evident by 

incremental change or innovative strategies, programs or technologies?
30

Draft March 27, 2008 Draft

Appendix 

A simple tally of sustainable development indicators was assembled on an Excel spreadsheet to 

demonstrate the wide variety of indicators that a selected group of eleven organizations utilize or 

suggest using. The organizations selected were the United Nations (Identified on the spreadsheet as 

UN; with international level indicators), the Global Reporting Initiative (GRI; any level- although 

their indicators seem tailored to corporations), the United States Interagency Working Group on 

Sustainable Development Indicators (SDI; national level), the Balaton Group (Meadows; any level), 

Jacksonville Community Council, Inc. (JCCI; city level), Sustainable Seattle (Seattle; city level), 

Santa Monica, California’s Sustainable City Program (S. Monica; city level), the Oregon Progress 

Board (OPB; state level), the Portland-Multnomah Progress Board (PMPB; city and regional level), 

the Fraser Basin Council, British Columbia (FBC; region level) and the ecological footprint model 

(Footprint; any level). 

A total of 413 indicators were identified (note that this number is not set in stone to the extent that 

it is possible for some indicators to be compressed). Although indicators can be categorized in any 

number of different ways, since I began with the UN’s indicators and because I assume that their 

categories reflect some sort of international consensus, I follow the categories used by the UN (i.e., 

social, environmental, economic, and institutional indicators). Within each of these four categories 

several themes became apparent. Among social indicators, themes based on “Equity,” “Health,” 

“Education,” “Government,” what might be called “News or Media,” “Employment,” “Crime,” 

“Transportation,” and what might be called “Quality of Life” emerged. Among environmental 

indicators, themes based on “Climate Change,” “Ozone Layer Depletion,” “Air Quality,” what might 

be called “Resources and Capacity,” “Land Use Trends,” “Agriculture,” “Forests,” “Coastal Waters 

and Fisheries,” “Water,” “Materials,” “Energy,” “Biodiversity,” “Emissions, Effluents, and Wastes,” 

what might be called “Organizational Impacts,” “Toxic Waste and Mines,” and “Transportation, 

Planning, and Aesthetics” could be identified. Among economic indicators, themes based on “Economic Performance,” “Taxes and Government Spending,” “Material Consumption,” “Energy Use,” 

“Total Amount of Waste by Type and Destination,” what might be called “Organizational Performance,” “Families and Income,” “Housing,” “Transportation,” and “Built Capital” emerged. Finally, 

institutional indicator themes such as “Strategic Implementation of Sustainable Development,” 

“International Cooperation,” “Agreements with Indigenous Groups,” “Information Access,” “Communication infrastructure,” “Science and Technology,” “Disaster Preparedness and Response,” and 

“Improving Decision Making” were identified. Within themes, indicators based on different units of 

analyses were identified.

The most frequently mentioned indicators (highlighted in yellow on the spreadsheet under the ‘total’ 

column) were: percent living in poverty or under the poverty line (5 mentions); unemployment rate 

(5); adult secondary education achievement level (7); the percent of the population that voted in 

general/presidential elections (6); the number of crimes per X population (5); percent of the population volunteering time (5); the number and size of nice places like parks (7); greenhouse gas emissions (8); total water use (6); abundance of key species such as salmon (6); annual energy consumption per capita (5); generation of industrial and municipal solid waste (7); and distance traveled per 

capita by mode of transport (5). 
31

Draft March 27, 2008 Draft

Meadows notes that indicators of sustainable development boil down to values (i.e., we measure 

what we care about) and mental models or assumptions that we hold about the world (cf., Cobb and 

Rixford, 1998; Cobb, 2000). Since our values and mental models differ, it follows that our indicators of sustainable development will likewise vary (1998: 6). Based on the current categorization of 

indicators on this spreadsheet, an amazing 75% (or 310) of the indicators were mentioned only once 

by any organization. Consequently, although hundreds of indicators are available, few organizations 

use the same units of analysis, and comparisons between the organizations selected would be difficult 

at best. 

Hart (1999) provides another method for determining the suitability of sustainability indicators. 

She created a 14-point checklist through which interested parties can assess how well an indicator 

addresses all facets of sustainable development. A “Yes” answer to any of the questions listed below 

counts as one point. Hart suggests that “In general, an indicator with a score of less than 4 will not 

be very useful for measuring sustainability...An indicator with a score of 7 or higher is, in general, a 

good indicator of sustainability” (1999: 140). Column R on the Excel spreadsheet provides Hart’s 

rankings—when possible—for the indicators used by these eleven organizations.

As can be seen on the spreadsheet, only three indicators—emissions of greenhouse gases, generation 

of solid waste, and total water use—are used by more than five organizations and score more than 

seven points on Hart’s checklist. From a policy point of view it becomes apparent that indicator 

selection and use is an imprecise endeavor: on one hand, the conceptual foundation of sustainable 

development is not well know, on the other hand, each region has different takes on what is considered to be important. 

The point of this exercise was to point out the diversity of indicators available (and, consequently, 

the difficulty of comparing indicators across organizations). Although many organizations focus on 

similar areas, the units of analyses and the quality of indicators used differ dramatically.

Indicator Sources

UN: Indicators of Sustainable Development: Guidelines and Methodologies. http://www.un.org/

esa/sustdev/indisd/indisd-mg2001.pdf

GRI: Global Reporting Initiative. http://www.globalreporting.org

SDI: The U.S. Interagency Working Group on Sustainable Development Indicators. http://www.

sdi.gov.

Meadows report to the Balaton Group: The Sustainability Institute, Vermont. http://sustainer.org.

JCCI: Jacksonville Community Council, Inc. www.jcci.org.

 

Seattle: Sustainable Seattle. www.sustainableseattle.org.
32

Draft March 27, 2008 Draft

S. Monica: Santa Monica’s Sustainable City Program. 

http://www.ci.santa-monica.ca.us/environment/policy/indicators.htm.

OPB: Oregon Progress Board. http://www.econ.state.or.us/opb/index.htm.

PMPB: Portland-Multnomah Progress Board. 

http://www.p-m-benchmarks.org/tblcnts.html.

FBC: the Fraser Basin Council, British Columbia. http://www.fraserbasin.bc.ca.

Footprint: the ecological footprint. http://www.redefiningprogress.org.
33

Draft March 27, 2008 Draft

Literature Cited

Anderson, Ray C. 1998. Mid-Course Correction. Toward a Sustainable Enterprise: The Interface Model. 

Atlanta: The Peregrinzilla Press.

Astleithner, Florentina et al. 2004. “Institutions and Indicators - The Discourse about Indicators in 

the Context of Sustainability.” Journal of Housing and the Built Environment. 19: 7-24. 

Ayres, Robert U., ed. 1998. Eco-restructuring: Implications for Sustainable Development. New York: 

United Nations University Press.

Barke, Michael and John Towner. 2003. “Learning from Experience? Progress Towards a Sustainable 

Future for Tourism in the Central and Eastern Andalucian Littoral.” Journal of Sustainable Tourism. 

11(2-3): 162-180.

Benyus, Janine M. 1997. Biomimicry: Innovation Inspired by Nature. New York: Perennial.

Beratan, Kathi K. et al. 2004. “Sustainability Indicators as a Communicative Tool: Building Bridges 

in Pennsylvania.” Environmental Monitoring and Assessment. 94: 179-191.

Berke, Philip R. and Maria Manta Conroy. 2000. “Are We Planning for Sustainable Development? 

An Evaluation of 30 Comprehensive Plans.” Journal of the American Planning Association. 66 (1): 

21-33.

Bianchi, Raoul V. 2004. “Tourism Restructuring and the Politics of Sustainability: A Critical View 

from the European Periphery (The Canary Islands). Journal of Sustainable Tourism. 12(6): 495-529.

Bramwell, Bill and Bernard Lane. 2003. “Editorial: Tourism and Sustainable Development in an 

Economic Downturn.” Journal of Sustainable Tourism. 1-2.

Buckley, Ralf. 2003. “Environmental Inputs and Outputs in Ecotourism: Geotourism with a Positive 

Triple Bottom Line?” Journal of Ecotourism. 2(1): 76-82.

Butcher, Jim. 2006. “Natural Capital and the Advocacy of Ecotourism as Sustainable Development.” 

Journal of Sustainable Tourism. 14(6): 529-544.

Campbell, Colin J. 2002. “Petroleum and People.” Population and Environment. 24 (2): 193-207

Catton, Jr., William R. 1982. Overshoot: The Ecological Basis of Revolutionary Change. Chicago: 

University of Illinois Press.

Chew, Sing C. 2001. World Ecological Degradation: Accumulation, Urbanization, and Deforestation 

3000 B.C. – A.D. 2000. Walnut Creek, CA: AltaMira Press.

Cobb, Clifford W. and Craig Rixford. 1998. Lessons Learned from the History of Social Indicators. 
34

Draft March 27, 2008 Draft

Redefining Progress. www.rprogress.org.

Cohen, Maurie J. 1997. “Risk Society and Ecological Modernization: Alternative Visions for PostIndustrial Nations.” Futures. 29 (2): 105-119.

Cole, Stroma. 2006. “Information and Empowerment: The Keys to Achieving Sustainable Tourism.” 

Journal of Sustainable Tourism. 14(6): 629-644.

Common, Michael. 1995. Sustainability and Policy: Limits to Economics. Cambridge: Cambridge 

University Press.

Costanza, Robert et al. 1997. “The Value Of the World’s Ecosystem Services and Natural Capital.”

Nature. 387: 253-260.

Cottrell, Stuart et al. 2004. “Measuring the Sustainability of Tourism in Manuel Antonio and Texel: 

A Tourist Perspective.” Journal of Sustainable Tourism. 12(5): 409-431.

Daly, Herman E. and John B. Cobb, Jr. 1994. For the Common Good: Redirecting the Economy Toward Community, the Environment, and a Sustainable Future. Boston: Beacon Press.

Deffeyes, Kenneth S. 2005. Beyond Oil: The View From Hubbert’s Peak. New York: Hill and Wang.

De Menocal, Peter B. 2001. “Cultural Responses to Climate Change During the Late Holocene.”

Science. 292(5517): 667: 774.

Diamond, Jared. 2005. Collapse: How Societies Choose to Fail or Succeed. New York: Viking.

---------------. 1997. Guns, Germs, and Steel: The Fates of Human Societies. New York: W.W. Norton & 

Company.

Dryzek, John S. 1997. The Politics of the Earth: Environmental Discourses. Oxford: Oxford University Press.

Dyson, Tim. 2005. “On Development, Demography and Climate Change: The End of the World as

We Know It?” Population and Environment. 27 (2): 117-149.

Eisenberg, Evan. 1998. The Ecology of Eden. New York: Alfred A. Knopf.

Elkington, John. 1998. Cannibals with Forks: The Triple Bottom Line of 21st Century Business. Gabriola Island, BC Canada: New Society Publishers.

Emanuel, Kerry. 2005. “Increasing Destructiveness of Tropical Cyclones Over the Past 30 Years.”

Nature. 436: 686-688.

Fagan, Brian. 2004. The Long Summer: How Climate Changed Civilization. New York: Basic Books.
35

Draft March 27, 2008 Draft

Farrell, Bryan and Louise Twining-Ward. 2005. “Seven Steps Towards Sustainability: Tourism in the 

Context of New Knowledge.” Journal of Sustainable Tourism. 13(2): 109-122.

Fernandez-Armesto, Felipe. 2001. Civilizations: Culture, Ambition, and the Transformation of Nature. 

New York: Simon and Schuster.

Flannery, Tim. 2005. The Weather Makers: How Man Is Changing the Climate and What It Means

for Life on Earth. New York: Atlantic Monthly Press.

Fowler, Stephen J. and C. Hope. 2007. “A Critical Review of Sustainable Business Indices and their 

Impact.” Journal of Business Ethics. 76: 243-252. 

Freese, Lee. 1988a. “Evolution and Sociogenesis. Part 1: Ecological Origins.” Advances in Group 

Processes. 5: 53-89.

---------------. 1988b. “Evolution and Sociogenesis. Part 2: Social Continuities.” Advances in Group 

Processes. 5: 91-118.

---------------. 1994. “Evolutionary Tangles for Sociocultural Systems: Some Clues from Biology.” 

Advances in Human Ecology. 3: 139-171.

---------------. 1995. “From Commons to Traps: Natural Balance and Human Ecological Series.” 

Advances in Human Ecology. 4: 117-140.

---------------. 1997a. Evolutionary Connections. Greenwich, CT: JAI Press.

---------------. 1997b. Environmental Connections. Greenwich, CT: JAI Press.

Gale, Richard P. and Sheila M. Cordray. 1994. “Making Sense of Sustainability: Nine Answers to 

‘What Should be Sustained?’” Rural Sociology. 59 (2): 311-332.

Global Reporting Initiative. 2002. Sustainability Reporting Guidelines. Downloadable at www.

globalreporting.org. 

Good, David H. and Rafael Reuveny. 2006. “The Fate of Easter Island: The Limits of Resource 

Management Institutions.” Ecological Economics. 58(3): 473-490.

Graafland, Johan J., S.C.W. Eijffinger, and H. Smid. 2004. “Benchmarking of Corporate Social Responsibility: Methodological Problems and Robustness.” Journal of Business Ethics. 53: 137-152. 

Gunderson, Lance H. and C.S. Holling (eds.). 2002. Panarchy: Understanding Transformations in 

Human and Natural Systems. Washington, D.C.: Island Press.

Hart, Maureen. 1999. Guide to Sustainable Community Indicators (Second Edition). North Andover, MA: Hart Environmental Data.
36

Draft March 27, 2008 Draft

Hawken, Paul. 1993. The Ecology of Commerce: A Declaration of Sustainability. New York: HarperBusiness.

Hawken, Paul, Amory Lovins, and L. Hunter Lovins. 1999. Natural Capitalism: Creating the Next 

Industrial Revolution. Boston: Little, Brown & Company.

Hempel, Lamont C. 1999. “Conceptual and Analytical Challenges in Building Sustainable Communities.” In Toward Sustainable Communities: Transitions and Transformations in Environmental 

Policy. David A. Mazmanian and Michael E. Kraft (eds.). Cambridge, MA: The MIT Press.

Hoffman, Andrew J. and Max H. Bazerman. 2007. “Changing Practice on Sustainability: Understanding and Overcoming the Organizational and Psychological Barriers to Action,” Pp. 84-105 

in Organizations and the Sustainability Mosaic: Crafting Long-Term Ecological and Societal Solutions. 

Sanjay Sharma et al. (eds). Northampton, MA: Edward Elgar.

Holdren, John P., Gretchen C. Daily and Paul R. Ehrlich. 1995. “The Meaning of Sustainability: 

Biogeophysical Aspects.” http://dieoff.org/page113.htm

Hubbert, M. King. 1976. “Exponential Growth as a Transient Phenomenon in Human History”, pp. 

75-84 in Margaret A. Storm, ed., Societal Issues: Scientific Viewpoints. New York: American Institute 

of Physics.

Hughes, J. Donald. 1975. Ecology in Ancient Civilizations. Albuquerque: University of New 

Mexico Press.

Janssen, Marco A. and Marten Scheffer. 2004. “Overexploitation of Renewable Resources by 

Ancient Societies and the Role of Sunk-Cost Effects.” Ecology and Society. 9(1): 6 [online] URL: 

http://www.ecologyandsociety.org/vol9/iss1/art6

Johnston, Paul et al. 2007. “Reclaiming the Definition of Sustainability.” Environmental Science and 

Pollution Research. 14(1): 60-67.

King, Andrew A. and Michael J. Lenox. 2001. “Does it Really Pay to be Green? An Empirical 

Study of Firm Environmental and Financial Performance.” Journal of Industrial Ecology. 5 (1): 105-

116. 

Kinlaw, Dennis C. 1993. Competitive and Green: Sustainable Performance in the Environmental Age. 

San Diego: Pfeiffer & Company.

Kunstler, Jame Howard. 2005. The Long Emergency: Surviving the Converging Catastrophes of the

Twenty-First Century. New York: Atlantic Monthly Press.

Lele, Sharachchandra M. 1991. “Sustainable Development: A Critical Review.” World Development. 19 (6): 607-621.
37

Draft March 27, 2008 Draft

Liu, Zhenhua. 2003. “Sustainable Tourism Development: A Critique.” Journal of Sustainable Tourism. 

11(6): 459-475.

McCool, Stephen F. and George H. Stankey. 2004. “Indicators of Sustainability: Challenges and Opportunities at the Interface of Science and Policy.” Environmental Management. 33(3): 294-305.

McDonough, William and Michael Braungart. 2002. Cradle to Cradle: Remaking the Way We Make 

Things. New York: North Point Press.

Meadows, Donella. 1998. Indicators and Information Systems for Sustainable Development. 

Four Corners, VT: The Sustainability Institute. Download at http://sustainer.org/pubs/

Indicators&Information.pdf

---------------. 1999. Leverage Points: Places to Intervene in a System. Four Corners, VT: The Sustainability Institute. Download at http://sustainer.org/pubs/Leverage_Points.pdf

Moffatt, Ian, Nick Hanley, and Mike D. Wilson. 2001. Measuring and Modelling Sustainable Development. New York: The Parthenon Publishing Group.

Mohai, Paul. 1995. “The Demographics of Dumping Revisited: Examining the Impact of Alternate Methodologies in Environmental Justice Research.” Virginia Environmental Law Journal. 14: 

615-653.

Mol, Arthur P. J. and Gert Spaargaren. 1993. “Environment, Modernity, and the Risk Society: The 

Apocalyptic Horizon of Environmental Reform.” International Sociology. 8 (4): 431-459.

---------------. 2000. “Ecological Modernization Theory in Debate: A Review.” Environmental Politics. 

9 (1): 17-49.

Mumford, Lewis. 1989 (1961). The City in History: Its Origins, its Transformations, and its Prospects. 

U.S.A.: A Harvest Book, Harcourt, Inc.

Mycoo, Michelle. 2006. “Sustainable Tourism Using Regulations, Market Mechanisms and Green 

Certification: A Case Study of Barbados.” Journal of Sustainable Tourism. 14(5): 489-511.

National Research Council of the National Academy of Sciences. 1999. Our Common Journey: A 

Transition Toward Sustainability. Washington, D.C: National Academy Press.

Nitkin, David and Leonard J. Brooks. 1998. “Sustainability Auditing and Reporting: The Canadian 

Experience.” Journal of Business Ethics. 17: 1499-1507.

Norman, Wayne and Chris MacDonald. 2004. “Getting to the Bottom of “Triple Bottom Line””. 

Business Ethics Quarterly. 14(2): 243-262.

Pava, Moses L. 2007. “A Response to “Getting to the Bottom of ‘Triple Bottom Line’”. Business Ethics 
38

Draft March 27, 2008 Draft

Quarterly. 17(1): 105-110. 

Ponting, Clive. 1991. A Green History of the World: The Environment and the Collapse of Great Civilizations. New York: Penguin Books.

Redclift, Michael. 1991. “The Multiple Dimensions of Sustainable Development.” Geography. 36-42.

---------------. 1992. “The Meaning of Sustainable Development.” Geoforum. 23 (3): 395-403.

Schianetz, Karin, Lydia Kavanagh, and David Lockington. 2007. “Concepts and Tools for Comprehensive Sustainability Assessments for Tourism Destinations: A Comparative Review.” Journal of 

Sustainable Tourism. 15(4): 369-389.

Schmidheiny, Stephan, with the Business Council for Sustainable Development. 1992. Changing 

Course: A Global Business Perspective on Development and the Environment. Cambridge, MA: The 

MIT Press.

Schnaiberg, Allan and Kenneth Alan Gould. 1994. Environment and Society: The Enduring Conflict. 

New York: St. Martin’s Press.

Sherwood, Peter, Leo Jago and Margaret Deery. “Triple Bottom Line Evaluation of Special Events: 

Does the Rhetoric Reflect Reporting?” 

Smil, Vaclav. 1994. Energy in World History. Boulder, CO: Westview Press.

Stem, Caroline J. et al. 2003. “How ‘Eco’ is Ecotourism? A Comparative Case Study of Ecotourism 

in Costa Rica.” Journal of Sustainable Tourism. 11(4): 322-347.

Tainter, Joseph A. 2000. “Problem Solving: Complexity, History, Sustainability.” Population and 

Environment. 22: 3-41.

---------------. 1996. “Complexity, Problem Solving, and Sustainable Societies” in Getting Down to 

Earth: Practical Applications of Human Ecology. Island Press. 

---------------. 1988. The Collapse of Complex Societies. Cambridge: Cambridge University Press.

Thompson, William R. 2004. “Complexity, Diminishing Marginal Returns, and Serial Mesopotamian Fragmentation.” Journal Of World-systems Research. 10(3): –.

United Nations. 2007. Indicators of Sustainable Development: Guidelines and Methodologies. New 

York: United Nations.

United Nations Millennium Ecosystem Assessment. 2006. Ecosystems and Human Well-Being: Our

Human Planet. Summary for Decision-Makers. Millennium Ecosystem Assessment Series. Washington: 

Island Press. Url: http://www.millenniumassessment.org//en/products.global.summary.aspx.
39

Draft March 27, 2008 Draft

van der Duim, Rene. 2004. “The Margins of Texel.” Journal of Sustainable Tourism. 12(5): 367-387.

Vera Rebollo, J. Fernando and Josep A. Ivars Baidal. 2003. “Measuring Sustainability in a Mass Tourist Destination: Pressures, Perceptions and Policy Responses in Torrevieja, Spain.” Journal of Sustainable Tourism. 11 (2-3): 181-203.

Wackernagel, Mathis and William Rees. 1996. Our Ecological Footprint: Reducing Human Impact on 

Earth. Gabriola Island, B.C. Canada: New Society Publishers.

Walker, Brian et al. 2002. “Resilience Management in Social-ecological Systems: A Working Hypothesis for a Participatory Approach.” Conservation Ecology. 6(1): http://www.consecol.org/vol6/

iss1/art14.

Weiss, Harvey and Raymond S. Bradley. 2001. “What Drives Societal Collapse?” Science. 291(26): 

609-610.

Willard, Bob. 2002. The Sustainability Advantage: Seven Business Case Benefits of a Triple Bottom 

Line. Gabriola Island, BC: New Society Publishers.

World Commission on Environment and Development. 1987. Our Common Future. Oxford: Oxford University Press.

World Conservation Union, The United Nations Environment Programme, World Wide Fund for 

Nature. 1991. Caring for the Earth. Gland, Switzerland: Author.