Program Evaluation of the 

Appalachian Regional 

Commission’s Job 

Creation and Retention 

Projects

DEVELOPED FOR THE

Appalachian Regional Commission

BY

HDR, Inc.

with 

Jack Faucett Associates, Inc.

September 2015
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

September 2015 | i

Contents

1 Executive Summary ............................................................................................................................ 1

1.1 Evaluation Overview.................................................................................................................. 3

1.2 Program Evaluation Review...................................................................................................... 3

1.3 Performance Measure Database .............................................................................................. 5

1.3.1 Data Analysis ............................................................................................................... 5

1.4 Case Studies of Best Practices................................................................................................. 6

1.5 Rural Policy Interviews .............................................................................................................. 9

1.6 Study Findings and Recommendations .................................................................................. 10

2 Introduction........................................................................................................................................ 13

3 Definition of Evaluation Criteria......................................................................................................... 14

3.1 Introduction.............................................................................................................................. 14

3.2 Program Evaluation Methods.................................................................................................. 15

3.2.1 Surveys ...................................................................................................................... 15

3.2.2 Observation or Site Visits........................................................................................... 16

3.2.3 Analysis of Experimental Data ................................................................................... 18

3.2.4 Before and After Comparisons................................................................................... 19

3.2.5 Theory-Based Evaluation and Modeling .................................................................... 19

3.2.6 Case Studies .............................................................................................................. 20

3.2.7 Application of Other Studies....................................................................................... 22

3.2.8 Other Potentially Applicable Evaluation Approaches................................................. 22

3.3 Summary and Conclusions ..................................................................................................... 25

4 Review of Past Reports and Program Evaluations ........................................................................... 29

4.1 “Program evaluation of ARC’s Tourism, Cultural Heritage, and Natural Asset-Related 

Projects” .................................................................................................................................. 29

4.1.1 Purpose and Goals..................................................................................................... 29

4.1.2 Key Issues Identified .................................................................................................. 29

4.1.3 Key Performance Measures....................................................................................... 30

4.1.4 Analytical Approach.................................................................................................... 30

4.2 “Creating an Entrepreneurial Appalachian Region: Findings and Lessons from an 

Evaluation of the Appalachian Regional Commission’s Entrepreneurship Initiative 

1997-2005”.............................................................................................................................. 31

4.2.1 Purpose and Goals..................................................................................................... 32

4.2.2 Key Issues Identified .................................................................................................. 32

4.2.3 Key Performance Measures....................................................................................... 32

4.2.4 Analytical Approach.................................................................................................... 33

4.3 “Evaluation of the Appalachian Regional Commission’s Community Capacity-Building 

Projects” .................................................................................................................................. 34

4.3.1 Purpose and Goals..................................................................................................... 34

4.3.2 Key Issues Identified .................................................................................................. 34

4.3.3 Key Performance Measures....................................................................................... 34

4.3.4 Analytical Approach.................................................................................................... 35

4.4 “Evaluation of the BITS Incubator Program & the Intelligent Island Incubator” ...................... 35

4.4.1 Purpose and Goals..................................................................................................... 35

4.4.2 Key Issues Identified .................................................................................................. 36

4.4.3 Key Performance Measures....................................................................................... 36

4.4.4 Analytical Approach.................................................................................................... 37
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

ii | September 2015

4.5 “Implementation Evaluation of the Community-Based Job Training Grant (CBJTG) 

Program”.................................................................................................................................. 38

4.5.1 Purpose and Goals..................................................................................................... 38

4.5.2 Key Issues Identified .................................................................................................. 38

4.5.3 Key Performance Measures....................................................................................... 39

4.5.4 Analytical Approach.................................................................................................... 39

5 Performance Measurement............................................................................................................... 41

5.1 Data Collection ........................................................................................................................ 41

5.1.1 ARC.net Database ..................................................................................................... 41

5.1.2 Survey ........................................................................................................................ 42

5.2 Performance Measure Database ............................................................................................ 43

5.2.1 Construction of Survey Database .............................................................................. 43

5.2.2 Performance Measure Database Development......................................................... 44

5.3 Analysis and Summary of Project Performance Measure Database...................................... 44

5.3.1 Characteristics by State, All Funded Projects............................................................ 45

5.3.2 Characteristics by Project Type ................................................................................. 47

5.3.3 Anticipated vs. Actual Impacts ................................................................................... 64

5.3.4 Perceived Qualitative Project Outcomes.................................................................... 64

5.4 Summary of Data Analysis Findings ....................................................................................... 69

6 Case Studies of Best Practices for Job Creation and Retention....................................................... 70

6.1 Capacity Building for Asset-Based Economies in Western North Carolina: 

Resourceful Communities Program ........................................................................................ 75

6.1.1 Community Profile...................................................................................................... 75

6.1.2 Project Description ..................................................................................................... 76

6.1.3 Project Planning and Implementation ........................................................................ 77

6.1.4 Economic and Community Impacts............................................................................ 78

6.1.5 Lessons Learned........................................................................................................ 79

6.1.6 Conclusion.................................................................................................................. 80

6.2 University of Alabama Export Trade Financing Program........................................................ 81

6.2.1 Community Profile...................................................................................................... 81

6.2.2 Project Description ..................................................................................................... 82

6.2.3 Project Planning and Implementation ........................................................................ 82

6.2.4 Economic and Community Impacts............................................................................ 85

6.2.5 Lessons Learned........................................................................................................ 85

6.2.6 Conclusion.................................................................................................................. 87

6.3 Toccoa Downtown Revitalization Project................................................................................ 87

6.3.1 Community Profile...................................................................................................... 87

6.3.2 Project Description ..................................................................................................... 88

6.3.3 Project Planning and Implementation ........................................................................ 88

6.3.4 Economic and Community Impacts............................................................................ 89

6.3.5 Lessons Learned........................................................................................................ 91

6.3.6 Conclusion.................................................................................................................. 92

6.4 Mountain Empire Alternative Energy Initiative ........................................................................ 92

6.4.1 Community Profile...................................................................................................... 92

6.4.2 Project Description ..................................................................................................... 93

6.4.3 Project Planning and Implementation ........................................................................ 93

6.4.4 Economic and Community Impacts............................................................................ 94

6.4.5 Lessons Learned........................................................................................................ 97

6.4.6 Conclusion.................................................................................................................. 97

6.5 Great Allegany Passage Economic Development Program ................................................... 97

6.5.1 Community Profile...................................................................................................... 98
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

September 2015 | iii

6.5.2 Project Description ..................................................................................................... 99

6.5.3 Project Planning and Implementation ...................................................................... 100

6.5.4 Economic and Community Impacts.......................................................................... 101

6.5.5 Lessons Learned...................................................................................................... 101

6.5.6 Conclusion................................................................................................................ 102

6.6 Heartwood Southwest Virginia Artisan Gateway .................................................................. 102

6.6.1 Community Profile.................................................................................................... 103

6.6.2 Project Description ................................................................................................... 103

6.6.3 Project Planning and Implementation ...................................................................... 104

6.6.4 Economic and Community Impacts.......................................................................... 107

6.6.5 Lessons Learned...................................................................................................... 108

6.6.6 Conclusion................................................................................................................ 109

6.7 MARC Custom Medical Products, Western North Carolina.................................................. 110

6.7.1 Community Profile.................................................................................................... 111

6.7.2 Project Description ................................................................................................... 112

6.7.3 Project Planning and Implementation ...................................................................... 113

6.7.4 Economic and Community Impacts.......................................................................... 115

6.7.5 Lessons Learned...................................................................................................... 116

6.7.6 Conclusion................................................................................................................ 116

6.8 Strengthening Appalachia’s Farmers’ Markets ..................................................................... 117

6.8.1 Community Profile.................................................................................................... 117

6.8.2 Project Description ................................................................................................... 118

6.8.3 Project Planning and Implementation ...................................................................... 118

6.8.4 Economic and Community Impacts.......................................................................... 119

6.8.5 Lessons Learned...................................................................................................... 120

6.8.6 Conclusion................................................................................................................ 121

6.9 Finger Lakes ReUse Center Start-Up ................................................................................... 121

6.9.1 Community Profile.................................................................................................... 121

6.9.2 Project Description ................................................................................................... 122

6.9.3 Project Planning and Implementation ...................................................................... 123

6.9.4 Economic and Community Impacts.......................................................................... 124

6.9.5 Lessons Learned...................................................................................................... 126

6.9.6 Conclusion................................................................................................................ 127

6.10 Expanding Entrepreneurship in Appalachia – REAL Entrepreneurship Curriculum 

Improvement ......................................................................................................................... 128

6.10.1 Community Profile.................................................................................................... 128

6.10.2 Project Description ................................................................................................... 129

6.10.3 Project Planning and Implementation ...................................................................... 130

6.10.4 Economic and Community Impacts.......................................................................... 131

6.10.5 Lessons Learned...................................................................................................... 132

6.10.6 Conclusion................................................................................................................ 133

6.11 Northeastern Pennsylvania Alliance Enterprise Development ............................................. 133

6.11.1 Community Profile.................................................................................................... 133

6.11.2 Project Description ................................................................................................... 134

6.11.3 Project Planning and Implementation ...................................................................... 135

6.11.4 Economic and Community Impacts.......................................................................... 136

6.11.5 Lessons Learned...................................................................................................... 137

6.11.6 Conclusion................................................................................................................ 137

6.12 Building the Clean Energy Economy in Western North Carolina.......................................... 138

6.12.1 Community Profile.................................................................................................... 138

6.12.2 Project Description ................................................................................................... 139

6.12.3 Project Planning and Implementation ...................................................................... 139

6.12.4 Economic and Community Impacts.......................................................................... 141

6.12.5 Lessons Learned...................................................................................................... 142
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

iv | September 2015

6.12.6 Conclusion................................................................................................................ 142

6.13 West Virginia High Technology Consortium Foundation INNOVA Commercialization 

Technical Assistance............................................................................................................. 143

6.13.1 Community Profile.................................................................................................... 143

6.13.2 Project Description ................................................................................................... 144

6.13.3 Project Planning and Implementation ...................................................................... 145

6.13.4 Economic and Community Impacts.......................................................................... 145

6.13.5 Lessons Learned...................................................................................................... 147

6.13.6 Conclusion................................................................................................................ 148

6.14 Kentucky Highlands Energy Business Boot Camp ............................................................... 148

6.14.1 Community Profile.................................................................................................... 149

6.14.2 Project Description ................................................................................................... 149

6.14.3 Project Planning and Implementation ...................................................................... 150

6.14.4 Economic and Community Impacts.......................................................................... 152

6.14.5 Lessons Learned...................................................................................................... 152

6.14.6 Conclusion................................................................................................................ 153

6.15 North Carolina Department of Commerce FY 2010-2011 Wood Products/Home 

Furnishings Initiative.............................................................................................................. 153

6.15.1 Community Profile.................................................................................................... 154

6.15.2 Project Description ................................................................................................... 154

6.15.3 Project Planning and Implementation ...................................................................... 155

6.15.4 Economic and Community Impacts.......................................................................... 156

6.15.5 Lessons Learned...................................................................................................... 157

6.15.6 Conclusion................................................................................................................ 158

6.16 Key Lessons Learned............................................................................................................ 158

6.16.1 Engage, Educate and Involve the Community......................................................... 158

6.16.2 Change is Good – Be Flexible ................................................................................. 159

6.16.3 Mind Your Measures ................................................................................................ 159

6.16.4 Fine-tune the Grant Process .................................................................................... 160

7 Rural Policy Interviews .................................................................................................................... 161

7.1 Background ........................................................................................................................... 161

7.2 Summary of Discussions....................................................................................................... 162

7.2.1 Challenges in US Rural Communities...................................................................... 162

7.2.2 Target Industries and Sectors for the Appalachian Region ..................................... 164

7.2.3 Measuring Success of Investments ......................................................................... 167

7.2.4 Role of Funders in Appalachian Economic Development........................................ 169

7.3 Findings and Recommendations........................................................................................... 170

8 Study Findings and Recommendations .......................................................................................... 173

8.1 Key Findings.......................................................................................................................... 173

8.2 Recommendations ................................................................................................................ 175

Appendix 1 - Survey.................................................................................................................................. 177

Appendix 2 – Policy Discussion ................................................................................................................ 225

Appendix 3 – Policy Interviewees ............................................................................................................. 227
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

September 2015 | v

Tables

Table 1: Summary of Approaches and Methods......................................................................................... 25

Table 2: Rating of Approaches and Methods against Criteria .................................................................... 28

Table 3: Number and Share of Grants & Survey Responses Received by State....................................... 46

Table 4: ARC Investment by State, in Millions of Dollars ........................................................................... 47

Table 5: Job Creation and Retention Project Distribution by Database Type ............................................ 48

Table 6: Job Creation and Retention Project Distribution by Self-Reported Type...................................... 49

Table 7: Job Creation and Retention Project Distribution by Dollars Invested, in Millions ......................... 50

Table 8: Key Project Outcomes, All Project Types ..................................................................................... 51

Table 9: Predicted vs. Actual Outputs & Outcomes, Excluding Enterprise Development .......................... 64

Table 10: Case Study Overview ................................................................................................................. 72

Table 11: MARC Members and North Carolina Counties Served ............................................................ 111

Table 12: MARC Custom Medical Products Funding Table, July 2006 – February 2011 ........................ 114

Figures

Figure 1: Total ARC Job Creation and Retention Projects by State, Full Database................................... 45

Figure 2: Business/Training Projects, Jobs Created and Retained (n=40)................................................. 52

Figure 3: Business/Training Projects, Businesses Created and Retained (n=40)...................................... 52

Figure 4: Business/Training Projects, Leveraged Private Investment, $ millions (n=40) ........................... 53

Figure 5: Child Care/Education Projects, Jobs Created and Retained (n=9) ............................................. 54

Figure 6: Child Care/Education Projects, Businesses Created and Retained (n=9) .................................. 55

Figure 7: Child Care/Education Projects, Leveraged Private Investment, $ millions (n=9)........................ 55

Figure 8: Community Building/Tourism Projects, Jobs Created and Retained (n=17)............................... 56

Figure 9: Community Building/Tourism Projects, Businesses Created and Retained (n=17) .................... 57

Figure 10: Community Building/Tourism Projects, Leveraged Private Investment, $ millions 

(n=17)............................................................................................................................................. 57

Figure 11: Energy/Environmental Projects, Jobs Created and Retained (n=10)........................................ 58

Figure 12: Energy/Environmental Projects, Businesses Created and Retained (n=10)............................. 59

Figure 13: Energy/Environmental Projects, Leveraged Private Investment, $ millions (n=10)................... 59

Figure 14: Local Food Projects, Jobs Created and Retained (n=7) ........................................................... 60

Figure 15: Local Food Projects, Businesses Created and Retained (n=7)................................................. 61

Figure 16: Local Food Projects, Leveraged Private Investment, $ millions (n=7) ...................................... 61

Figure 17: Planning/Research Projects, Jobs Created and Retained (n=10)............................................. 62

Figure 18: Planning/Research Projects, Businesses Created and Retained (n=10).................................. 63

Figure 19: Planning/Research Projects, Leveraged Private Investment, $ millions (n=10)........................ 63

Figure 20: Qualitative Survey Responses: Economic Measures (n=97) .................................................... 66

Figure 21: Qualitative Survey Responses: Competitiveness Measures (n=97) ......................................... 67

Figure 22: Qualitative Survey Responses: Environmental Measures (n=97)............................................. 68

Figure 23: Qualitative Survey Responses: Quality of Life Measures (n=97).............................................. 69

Figure 24: Rennovation of Toccoa, Georgia............................................................................................... 89

Figure 25: Toccoa, Georgia – Before and After Renovation....................................................................... 90

Figure 26: Currahee Military Museum in Toccoa, Georgia ......................................................................... 90

Figure 27: Great Allegheny Passage Map.................................................................................................. 98
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

vi | September 2015

Figure 28: Maryland Section of the Great Allegheny Passage................................................................... 99

Figure 29: MARC Surgical Drape Manufacturing...................................................................................... 110

Figure 30: Automated Fabric Cutter in Use at MARC Custom Medical Products .................................... 112

Figure 31: Map of Locations of Appalachian Ohio Farmers’ Markets....................................................... 119
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

September 2015 | vii

This page is intentionally left blank.

Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

September 2015 | 1

1 Executive Summary

Report Highlights

This report presents the approach and findings from an evaluation of ARC grants 

awarded between FY 2004 and FY2010 in support of job creation or retention efforts.

Methodology

The 286 Job Creation and Retention Grant recipients from FY 2004 to FY 2010 provided 

information on their initial grant application regarding expectations for various outcome 

measures, including jobs and businesses created and retained, and leveraged private 

investment. The evaluation of program success was conducted using data obtained 

through a combination of surveys and case studies. Survey responses were received 

from all Appalachian Regional Commission (ARC) states and each of the defined project 

types, though not all were equally represented. Grantees from Pennsylvania provided the 

most responses of any state, in part due to the Enterprise Development Programs. 

Business/Training projects were the most frequently funded and generated the most 

responses of any project type. Fifteen case studies were selected based on notable 

approaches and results. Case studies represent each project category and 10 of 13 

Appalachian states. Additional insight into future program focus was provided by 

discussions with rural policy experts.

Findings

Overall, the 123 survey response demographics were consistent with the full ARC.net 

demographics. The largest share of the $42.5 million funding went to Pennsylvania. 

Business/Training was the most heavily represented project type. ARC funding 

supported projects that created and retained jobs and businesses in excess of their initial 

predictions. Excluding Enterprise Development Program impacts, survey respondents 

indicated:

• 5,964 jobs created or retained as compared to predictions of 3,766.

• 2,698 businesses created or served, exceeding the initial projections of 166.

• Leveraged private investment of $216.8 million, $179.5 million more than 

initial expectations.

Qualitatively, respondents indicated that ARC funding had a successful impact on 

economic measures, meeting the target of the program funding.

Lessons Learned

• Engage, educate and involve the community.

• Change is good – be flexible.

• Mind your measures.

• Fine-tune the grant process.
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

2 | September 2015

Recommendations for Future Programs

• Improve the data collection and tracking process.

• Clarify and specify measurement metrics and timing.

• Collect additional information related to funding sources.

• Consider targeted investment and focus on key growth industries.

• Increase focus on workforce training.

• Support local leadership.

• Consider additional assistance or resources for program administrators.
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

September 2015 | 3

1.1 Evaluation Overview

ARC investments fund locally developed projects or programs to assist the localities in 

establishing, supporting, and maintaining businesses. The purpose of these grants is to 

enhance economic development through a variety of avenues with the hope of meeting 

the Commission’s goals.

The Appalachian Regional Commission has four main goals, as laid out in the 2011-2016 

Strategic Plan:

1. Increase job opportunities and per capita income in Appalachia to reach parity with 

the nation.

2. Strengthen the capacity of the people of Appalachia to compete in the global 

economy.

3. Develop and improve Appalachia’s infrastructure to make the Region economically 

competitive.

4. Build the Appalachian Development Highway System to reduce Appalachia’s 

isolation.

The performance of ARC’s job creation and retention investments is closely related to 

these goals, particularly the first two, and this program evaluation intends to assess the 

performance and economic impacts of these investments between FY 2004 and FY 

2010. In addition, the evaluation offers insight into ways that ARC can enhance its ability 

to document and report program impacts, as well as assess potential industries or 

improvements to help improve rural areas in the Region.

The primary purpose of this study is to determine the extent to which ARC’s funding of 

job creation and retention projects has contributed to the achievement of ARC’s 

economic development objectives, as outlined in its strategic plan. The evaluation seeks 

to assess the allocation, impact and cost-effectiveness of the program through 

correlations between types of projects, levels of funding and real project outcomes. The 

evaluation also aims to provide recommendations to aid the Commission in optimizing 

future investments.

Overall, the findings of the study suggest that the grant program is successful in 

achieving many of ARC’s economic development goals, but there is room for 

improvement, particularly in the areas of data collection and monitoring. The remainder 

of the Executive Summary provides an overview of the study’s findings and provides 

some recommendations for enhancing the evaluation process.

1.2 Program Evaluation Review

The first stage of the project was to review program evaluation methodologies 

implemented both in previous ARC Program Evaluations and measures used by other 

agencies to determine the approach best suited for this study. The review discussed 13

different evaluation methods and also detailed five past reports and program evaluations 

from ARC and other government agencies.

The strengths and weaknesses of each evaluation approach, and their applicability to 

ARC’s job creation and retention program evaluation, were summarized. To determine 
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

4 | September 2015

the best approach for this evaluation, the qualities of each method were rated against 

criteria to determine the approaches best suited for this particular evaluation. The 

comparison matrix is presented in Table ES-1 below. Using information gathered from 

the review of evaluation methods and past studies, the team ultimately recommended 

using a combination of surveys and case studies to complete this program evaluation.

Table ES-1: Rating of Approaches and Methods against Criteria

Criteria

Approaches & Methods

Ease of Use for ARCEase of InterpretationFlexibilityData AvailabilityEstimation of CounterfactualTransparencyAcceptance & Credibility

Surveys     No  

Observations or Site Visits     No  

Analysis of Experimental 

Data

    Yes  

Before & After 

Comparisons

    No  

Theory Based Evaluation 

& Modeling

    Sometime  

Case Studies     Generally 

Not

 

Application of Findings 

from Other Studies

    No  

Use of Expert Opinion     Not 

Explicitly

 

Statistical Analysis and 

Use of Causal Models

    Sometime  

Input-Output Modeling     No  

Benefit-Cost Analysis     Yes, 

Needed

 

Sensitivity Analysis or 

Risk Analysis

    N/A  
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

September 2015 | 5

1.3 Performance Measure Database

Once the evaluation approach was determined, the project team examined the ARC.net 

database of the 286 job creation and retention projects funded by ARC between FY 2004 

and FY 2010. The projects considered in this evaluation cover a wide range of categories 

and project types. Some project types were underrepresented in the ARC.net database. 

The project team worked together with ARC to aggregate some of the more similar 

projects together to provide larger sample sizes in each of the six defined categories. 

After the project categories were agreed upon, a survey was designed to collect 

supplemental information regarding the project outputs and outcomes. 

Several steps were taken in an effort to generate the highest possible response survey 

rate. These include testing a draft survey on select grantees, distribution to the grantees 

both electronically and by mail, and extensive follow-up with non-responsive grantees. In 

the end, of the 286 program grantees, the team received 123 responses with sufficient 

information to analyze. There were a total of 138 survey responses, but 15 were either 

blank or duplicates, and were thus excluded.

The information gathered from the survey responses was appended to the ARC.net 

database to analyze the reported outputs and outcomes associated with the funded 

projects. Responses were received from all 13 Appalachian states. Pennsylvania 

accounted for the largest number of responses, 41 of the 123, or 33.3 percent. The large 

number of projects funded and responses in Pennsylvania is at least partially due to the 

50 Enterprise Development Program grants funded during this time period.

In terms of project types, Business/Training projects were the most heavily represented 

both in the full database and in the survey responses. Seventy of the 144 

Business/Training projects responded to the survey, accounting for 56.9 percent of all 

responses received. In general, the demographic composition of the survey responses 

was consistent with that of the full database.

From FY 2004 to FY 2010, ARC contributed $42.5 million to these 286 projects. The 

survey responses accounted for approximately half, or $20.5 million, of this funding. The 

long-running Enterprise Development Programs account for $17.8 million of the $42.5 

million in total funding during this period; these projects accounted for approximately half 

of the funding value of survey responses.

1.3.1 Data Analysis

The performance measure database included information on 123 projects from all 13 

Appalachian states and each of the six project types. One of the primary goals of the 

evaluation was to assess the economic development impacts, such as jobs created or 

retained and businesses created or retained. Overall, based on the survey results for 

which sufficient data were available, ARC job creation and retention investments created 

or retained more than 237,000 jobs, primarily driven by the Enterprise Development 

Program projects. The Pennsylvania Enterprise Development Programs contributed 

approximately 51,000 jobs created and 176,000 retained.

The 123 survey responses showed that projects that received ARC funding between FY 

2004 and FY 2010 contributed to the creation of 776 business establishments, and the 

retention of 3,427 businesses, as shown in Table ES-2 below. As with jobs, much of this 
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

6 | September 2015

is attributable to the Enterprise Development Programs – 430 businesses created and 

2,884 businesses retained. Finally, the $20.5 million that ARC contributed to the 

programs that responded to the survey leveraged an additional $422.7 million in private 

investments.

Table ES- 2: Key Project Outcomes, All Project Types

Project Type Jobs 

Created

Jobs 

Retained

Businesses 

Created

Businesses 

Retained

Leveraged 

Private 

Investment 

($ millions)

Business/Training 58,158 177,688 668 3,093 $362.3

Child Care/Education 48 43 25 48 $2.4

Community Building/Tourism 193 561 14 6 $30.8

Energy/Environmental 348 39 23 5 $9.1

Local Food 169 548 32 271 $0.7

Planning/Research 111 19 14 4 $17.5

Total 59,026 178,897 776 3,427 $422.7

Source: Survey response data

1.4 Case Studies of Best Practices

In addition to the survey, the project team conducted case studies of 15 projects that had 

notable results and approaches. The case studies are representative of each of the 

project types and all but three of the Appalachian states. These projects were identified 

through review of the survey data and discussions with ARC program staff. These 

programs created jobs, generated income and other quantitative outcomes, 

accompanying qualitative impacts, and policy implications for Appalachian residents and 

businesses. They enhanced economic development by attracting new industry, 

encouraging business expansion, diversifying local economies, and generating 

permanent, private-sector jobs.

The case studies focused on project planning and implementation, economic and 

community impacts, and lessons learned for communities undertaking similar projects in 

the future. The table below presents basic information on each of the 15 case studies, 

with details presented in Chapter 6.
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

September 2015 | 7

Table ES- 3: Case Study Overview1

Grantee Project Title

(Grant Number)

Project 

Description

Project Type Year ARC 

Funds

Total Funds Primary 

Outcomes

6.1 The 

Conservation 

Fund

Capacity building 

for Asset-Based 

Economies in 

Western North 

Carolina

(NC-16686-I)

To develop 

local AssetBased projects 

through a 

series of 

training 

workshop and 

hands-on 

technical 

assistance.

Planning / 

Research

2010 $100,000 $229,700 17 jobs 

created

6.2 University of 

Alabama

University of AL 

Export Trade 

Financing 

Program (EXTRA)

(AL-15573-C3)

Continuing 

support for a 

program that 

addresses 

access to 

technical 

assistance and 

finances in 

export 

marketing.

Business / 

Training

2010 $151,125 $302,250 92 jobs 

created; 

$6.7 

million LPI

6.3 City of 

Toccoa

Toccoa 

Downtown 

Revitalization 

Project (GA15259-I)

Revitalize the 

historic 

downtown 

business 

district.

Community 

Building / 

Tourism

2005 $300,000 $1,400,000 1,100 jobs 

retained

6.4 Mountain 

Empire 

Community 

College

Mountain Empire 

Alternative Energy 

Initiative

(CO-16396-I)

Develop a 

career Studies 

Certificate to 

articulate with 

an Energy 

Management 

Associate 

Degree 

program.

Child Care / 

Education

2009 $40,000 $82,386

6.5 The Progress 

Fund

Great Allegheny 

Passage 

Economic 

Development 

Program

(MD-16055-I)

To coordinate 

the Trail Town 

Program in the 

Maryland 

communities

Business / 

Training

2008 $100,000 $200,000 76 jobs 

created; 17 

new 

businesses 

created

6.6 Round the 

Mountain 

Southwest 

VA Artisan 

Network

Heartwood: 

Southwest VA 

Artisan Gateway

(VA-16005-I)

For state-ofthe-art 

iconography 

interior 

exhibits, media 

presentations, 

furnishing and 

equipment at 

The 

Heartwood: 

Southwest 

Virginia’s 

Artisan 

Gateway

Business / 

Training

2008 $1,000,000 $2,058,140

 

1 Note that the Mountain Empire Alternative Energy Initiative (6.4) did not directly have any reportable job or business 

creation impacts, though there have been 24 students enrolled in the program. Additionally, the Heartwood 

Southwest VA Artisan Gateway (6.6) does not have easily isolated impacts, and thus none have been noted in the 

table.
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

8 | September 2015

Grantee Project Title

(Grant Number)

Project 

Description

Project Type Year ARC 

Funds

Total Funds Primary 

Outcomes

6.7 Marketing 

Association 

for 

Rehabilitation 

Centers 

(MARC)

MARC Custom 

Medical Products 

(NC-15696-I)

A program to 

develop 

manufacturing 

capacity to 

produce 

custom 

medical 

drapes and 

other products 

and generate 

funds for 

development 

programs for 

people with 

disabilities

Business / 

Training

2007 $200,000 $883,400 162 jobs 

created; 

150 jobs 

retained; 

$2.7 

million LPI

6.8 Ohio State 

University 

Research 

Foundation

Strengthening 

Appalachia’s 

Farmers’ Markets 

(OH-16104-I)

To provide 

low- and nocost marketing 

and 

promotional 

strategies and 

templates for 

Appalachian 

Ohio farmers’ 

markets.

Local Food 2008 $75,000 $150,157 75 jobs 

retained

6.9 Cornell 

Cooperative 

Extension of 

Tompkins

Finger Lakes 

Reuse Center 

Start-up

(NY-16085-I)

To launch the 

Finger Lakes 

Reuse Center.

Business / 

Training

2008 $148,463 $300,444 9 jobs 

created

6.10 North 

Carolina 

REAL 

Enterprises, 

INC

Expanding 

Entrepreneurship 

in Appalachia –

REAL 

Entrepreneurship 

Curriculum 

Improvement

(NC-15807-I)

To revise the 

REAL 

Enterprise 

curriculum to 

include new 

technologies, 

more financial 

literacy, and 

information 

about 

emerging 

industry 

clusters and 

train new 

faculty.

Business / 

Training

2007 $82,600 $205,350 90 jobs 

created; 90 

new 

businesses 

created

6.11 Northeastern 

Pennsylvania 

Alliance

Northeastern PA 

Alliance 

Enterprise 

Development

(PA-8291-C29)

Project will 

continue small 

business 

financing, 

international 

business 

development 

assistance, 

procurement 

technical 

assistance, 

support of PA 

Source Net 

and other ecommerce 

activities.

Business / 

Training

2010 $400,000 $800,000 169 jobs 

created; 

640 jobs 

retained
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

September 2015 | 9

Grantee Project Title

(Grant Number)

Project 

Description

Project Type Year ARC 

Funds

Total Funds Primary 

Outcomes

6.12 Land-of-Sky 

Regional 

Council

Building the Clean 

Energy Economy 

in Western North 

Carolina

(NC-16543-I)

Activities to 

support the 

growth of a 

clean energy 

economy in 

Western North 

Carolina

Energy / 

Environmental

2010 $100,000 $200,000 9 jobs 

created; 

173 new 

businesses 

created

6.13 West Virginia 

High 

Technology 

Consortium 

Foundation

WV High 

Technology 

Consortium 

Foundation 

INNOVA 

Commercialization 

Technical

Assistance (WV15262-I)

Continuing 

support for an 

early-stage 

investment 

fund linked to 

a range of 

technical 

assistance 

efforts.

Business / 

Training

2005 $125,000 $250,000 22 jobs 

created

6.14 Kentucky 

Highlands 

Investment 

Corporation

Kentucky 

Highlands Energy 

Business Boot 

Camp

(CO-15789-I)

To develop 

and Energy 

Business Boot 

Camp, 

providing 

intensive 

performancebased 

entrepreneurial 

training and 

mentoring to 

growing firms.

Energy / 

Environmental

2007 $75,000 $225,000 15 jobs 

created

6.15 North 

Carolina 

Department 

of Commerce

FY 2010-2011 

Wood 

Products/Home 

Furnishings 

Initiative (CO15268-I)

To assist 

companies in 

North Carolina 

and other 

states in the 

Appalachian 

Region export 

wood 

products, 

manufacturing 

supplies and 

home 

furnishing.

Business / 

Training

2010 $25,000 $75,000 Floor sales 

= 

$200,000; 

Expected 

sales = 

$8.5 

million

1.5 Rural Policy Interviews

Beyond the survey and case studies, the project team also conducted interviews with 18 

rural policy and development experts to provide context to the findings and gain insight 

into and understanding of key development challenges, desired outcomes, and potential 

solutions to best meet the goals of ARC and the Region.

Discussions focused on three main themes: key challenges for rural communities in the

United States, particularly in Appalachia; target industries for the Region; and how to 

measure success of investments in rural communities. A few of the experts were also 

asked about the role of funders in Appalachian economic development.

The discussions indicated three main challenges for the Region: lack of leadership and 

vision for future development; lack of stable, well-paying jobs; and a reduction in 

investment in rural regions and their development. These challenges have been 

relatively consistent over time, though funding has been decreasing in the recent past. 
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

10 | September 2015

Emerging challenges include a lack of broadband infrastructure and worker skills to 

handle rapidly changing technology.

Experts identified several industries and job sectors to target for future growth, though 

they noted that the specific industries will depend on the specific characteristics and 

assets of each community. Some potential industries include: shale gas fracking and 

other clean energy production; farming and food supply; tourism and hospitality; 

manufacturing; healthcare administration; retail and services; and ecological services.

The experts have acknowledged that ARC has been successful at determining the needs 

of rural communities and making investments to try to meet those needs. They identified 

two main areas of improvement for measuring the success of these investments: 

improvements to the process of evaluating success and improvement to the metrics used 

to measure these successes. The consensus was that a standard set of metrics for all 

projects is not ideal and a unique set of indicators that best fits the project should be 

agreed upon at the project onset. They also agreed that more qualitative measures 

should also be tracked. The timing of project tracking and success measurement is also 

variable depending on the investment type. Additional detail about the discussions, 

findings and recommendations is provided in Chapter 7.

1.6 Study Findings and Recommendations

The job creation and retention program evaluation examined the impact of investments 

on income, employment opportunities, job and business creation and retention, and other 

qualitative impacts. The evaluation considered all quantitative and qualitative outcomes 

of these investments to gauge the overall performance of the projects.

Business/Training projects were the most frequently and heavily funded project types. 

These projects also generated the most benefit in terms of job and business creation and 

retention. A large share of the success was driven by the Enterprise Development 

Programs, though Business/Training projects still accounted for most benefits even when 

the Enterprise Development Programs are excluded.

The survey responses indicated that grantees had a tendency to underestimate the job 

and business impacts of the investment on their area. Respondents also indicated that 

the funding they received contributed to success on qualitative economic and 

competitiveness measures, including jobs and sources of income, business sales, and 

the stability of the economy. Highlighting the importance of ARC funding, approximately 

25 percent of the survey respondents indicated that without ARC’s contributions, their 

project would not have been undertaken.

Though the results indicated general success and positive outputs and outcomes, the 

results of the survey, case studies, and policy discussions have led the project team to 

develop a list of lessons learned and several recommendations to increase the likelihood 

of future success. These recommendations are not meant to be a complete assessment 

of the program or its priorities, but are designed to support future success and help the 

Commission achieve its goals.

The “lessons learned” were identified through the survey and case studies and may help 

ARC and future grant recipients best leverage funding to maximize economic 

development outcomes:
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

September 2015 | 11

• Engage, educate and involve the community.

• Change is good – be flexible.

• Mind your measures.

• Fine-tune the grant process.

The recommendations below are intended to improve the data collection process, the 

quality of the data used for program evaluation and success of localities within the 

Region.

Improve the data collection and tracking process. Despite thorough efforts at 

gathering data for these projects, the team was able to collect data for fewer projects 

than desired. In addition to difficulty tracking some grantees, many of those individuals 

contacted did not have data on the measured outputs and outcomes. Over the course of 

nearly 10 years, some of the grant recipients had previously collected this information but 

no longer had the records. Others simply did not track the outputs and outcomes.

Given the high priority of tangible, measured improvements to job opportunities and other 

measures of competitiveness, it is in the best interest of ARC to require tracking for 

reporting purposes. Tracking of the performance metrics that are deemed most important 

to ARC should be required. It is important not to penalize grant recipients should factors 

outside their control result in a lack of expected performance, but tracking and measuring 

these outputs and outcomes will help improve ARC’s project prioritization and best utilize 

limited funds into the future. Informing the Local Development Districts of the importance 

of collecting and maintain performance metrics may be a first-step to improving overall 

data collection.

Clarify and specify measurement metrics and timing. Grant recipients and rural 

policy experts agreed that tracking and measurement of project outcomes was difficult 

and needed improvement. They suggested that ARC implement a standardized process 

for collecting, measuring, and regularly reporting the impact data on each investment. 

This can either be done by an outside resource or by grant applicants educated on the 

tracking and measurement process desired by ARC. At present, there is some confusion 

about the exact nature of each metric and how it should be measured and tracked.

Several additional metrics were suggested to supplement the standard job creation 

measures, including growth in household income and level of entrepreneurship in the 

culture. More qualitative measures of success are also necessary, as there are many 

“soft” outcomes, such as civic capacity, that are very important to communities but are 

not easily quantified. Rural policy experts suggested customizing the indicators for each 

project type, due to the wide variation in types and categories of investment.

The time period for outcomes also varies by project type, with some investments such as 

education programs, taking a longer period to reach their full impact. The tracking and 

measurement process should account for this.

Collect additional information related to funding sources. In many cases, ARC funds 

are not the only source of capital for these projects. Gaining a better understanding of all 

funding required to implement a project, as well as the sources of these funds, could 

provide useful insight to ARC. Understanding differences in funding sources and options 

for various job creation and retention programs may help target and prioritize future 
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

12 | September 2015

investments. Additionally, tracking detail on leveraged private investment may be useful 

for other potential project investments where a variety of funding sources are more 

available. This may free some of ARC’s limited available resources for those high-impact 

projects with more limited access to other funding sources may help ARC best utilize the 

limited available resources. A better understanding of the complete funding picture may 

be informative to ARC as future investments are considered.

Consider targeted investment and focus on key growth industries. Focusing some 

grant resources on potentially high-growth, high-paying industries within the region could 

lead to long-term gains in overall quality of life. The decline of coal mining has led to a 

reduction in high paying jobs in many parts of the Region. Focus on clean energy 

industries and other high-growth potential industries may help provide well-paying jobs 

for area residents while potentially reducing some of the “brain drain” that the region has 

experienced.

Increase focus on workforce training. Increase focus on both general workforce 

training and the means by which workers are trained. A focus on investment in workforce 

training for some higher-skilled jobs within existing and targeted industries, including 

healthcare and ecological services, could help increase per capita income in the Region. 

The way that workers are trained should also be examined. The rapid nature of 

technological improvements requires constant learning. Workforce training programs 

should provide guidance on continuous learning to best prepare area workers for longterm employment.

Support local leadership. One key to continued success in the region is an increased 

focus on leadership within the rural communities. Supporting local leadership can help 

promote long-term stability and growth in the Region. This support can help foster civic 

capacity investment and environmental protections in the longer-term. Strong local 

leadership may also help address two key challenges – access to well-paid, stable jobs 

and a lack of public funding in rural areas. Development plans and local leadership could 

provide rural areas with champions who can organize in an effort to secure additional 

funding.

Consider additional assistance or resources for program administrators. Feedback 

has indicated that the grant writing and tracking process can be burdensome to 

recipients. Several agencies noted that they had forgone potential funding for new 

projects due to the time and effort required in the grant writing process. Though it has the 

potential to be costly and divert resources from actually funding projects and programs, 

additional support or resources both during the grant writing process and during the 

monitoring of grants would be beneficial to grantees. Many of these rural communities 

are strapped for resources, and the time required to plan and monitor a grant is often be 

cumbersome. Providing additional resources may contribute to greater long-term 

success, as well as better data collection and performance monitoring.
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

September 2015 | 13

2 Introduction

Appalachian Regional Commission (ARC) investments fund locally developed projects to 

assist localities in establishing and supporting private-sector businesses. ARC’s job 

creation and retention grants are specifically designed to financially support these locally 

grown initiatives. The purpose of ARC’s grants is to enhance economic development by 

attracting new industry, encouraging business expansion, diversifying local economies, 

and generating permanent, private-sector jobs. Non-infrastructure job creation and 

retention projects supported by ARC include business development and 

entrepreneurship, education and training, health care programs, and leadership 

development and civic capacity.

An evaluation of these non-infrastructure job creation and retention investments 

facilitates a better understanding of how to best distribute limited resources. Information 

gathered during this program evaluation supports accountable decision making going 

forward. To effectively evaluate these programs and the policy direction, an 

understanding of data-driven, performance-based business practices that link 

organizational goals and objectives to resources is required. Performance management 

has been used for more than a decade in various public works and policy fields, and over 

that time has evolved from a focus on identifying measures and tracking performance to 

using performance measures in decision-making. 

This study aims to evaluate ARC funded projects that were focused on job creation and 

retention to assess whether or not the projects have achieved their stated goals. The 

evaluation also seeks to determine whether these projects have contributed to the 

strategic objectives of the Commission as planned. The study covers projects funded 

from FY 2004 to FY 2010 throughout the Appalachian Region and across a wide variety 

of categories with varying objectives. The focus of this analysis is to examine economic 

impacts, including job creation and retention, business service, economic diversification 

and income growth. In addition to the data analysis component, discussions with rural 

policy experts were also held to gather additional insight into particular challenges and 

potential solutions that can be implemented to help achieve the goals of the Appalachian 

Regional Commission.

Chapter 3 of this report provides an overview of various methods of program evaluation 

employed by ARC and other organizations. Chapter 4 provides a review of past reports 

and program evaluations and an assessment of their relevance to this particular 

evaluation. Chapters 5 and 6 highlight performance measurement and outcomes 

applicable to the evaluation and present the findings of 15 case studies of funded 

projects. Chapter 7 highlights interviews with rural policy experts and Chapter 8 provides 

a summary of study findings and recommendations. 
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

14 | September 2015

3 Definition of Evaluation Criteria 

3.1 Introduction

Appalachian Regional Commission (ARC) investments fund a wide variety of locally 

developed projects, including those targeted at job creation and retention. ARC’s job 

creation and retention projects generally fall into four categories: business development, 

community development, infrastructure and public works, and other. The Job Creation 

and Retention Program Evaluation focuses on non-infrastructure and public works 

projects.

Business development programs include non-physical infrastructure projects to enhance 

local economic development efforts related to business outreach, small-business 

support, and industry-specific initiatives. Specific project types within the ARC.net 

database under this category include: technical assistance, entrepreneurship, business 

incubators, energy, agriculture development, revolving loan funds and access to capital, 

market expansion, and international trade assistance. Community Development projects 

include ARC investments that support a range of community development initiatives such 

as downtown revitalization, development of community facilities, planning, and parks and 

recreation initiatives. Other/Miscellaneous projects include projects in areas such as 

tourism development, health planning, flood-related investment, and mental health. 

A goal of ARC is to better understand the success of its programs in achieving its goals. 

Performance management is a policy-directed, data-driven, performance-based business 

practice that links organizational goals and objectives to resources and results. The 

outcomes of performance-based management include a more efficient distribution of 

limited resources and a focus on accountability of decision-making. Performance 

management has been in use for over a decade in various public works fields, and over 

that time it has evolved from a focus on identifying measures and tracking performance 

to using performance measures in decision-making. This increased focus on managing 

with performance measures has enhanced the role of performance evaluation to address 

two important questions: 1) How are we doing? And 2) How can we do better?

This report provides detail on the various methods employed by ARC and other 

organizations to evaluate program performance. This information is based on a review of 

three ARC studies and two studies from other agencies that focus on program evaluation 

reports. Traditional evaluation methods and other approaches are described in Section 

3.2, and studies that have relied on these methods to measure program performance are 

highlighted throughout this report. The major evaluation approaches discussed in the 

report include:

• Surveys

• Observation or site visits

• Analysis of experimental data

• Before and after comparisons

• Theory-based evaluation and modeling

• Case studies
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

September 2015 | 15

• Application of other studies

• Use of expert opinion 

• Statistical analysis and causal models 

• Input-output modeling

• Benefit-cost analysis 

• Sensitivity analysis and risk analysis

A complete annotated bibliography that provides the name of the study, primary purpose, 

overall conclusions and other information is also provided in this report. Five studies 

were reviewed for this project: 

• Program Evaluation of ARC’s Tourism, Cultural Heritage, and Natural AssetRelated Projects (September 2010)

• Creating an Entrepreneurial Appalachian Region: Findings and Lessons from an 

Evaluation of the Appalachian Regional Commission’s Entrepreneurship Initiative 

1997-2005 (April 2008)

• Evaluation of The Appalachian Regional Commission’s Community CapacityBuilding Projects (July 2004)

• Evaluation of the BITS Incubator Program & the Intelligent Island Incubator 

(November 2003)

• Implementation Evaluation of the Community-Based Job Training Grant (CBJTG) 

Program (December 2012)

The remainder of this section describes different methods to conduct program 

evaluation, keeping in mind that it is common to apply two or more techniques to 

strengthen evaluation findings and examine causal relationships between a program and 

its desired outcomes. 

3.2 Program Evaluation Methods

There are a number of different approaches to measuring project performance. A 

description of each method, its pros and cons, and its applicability to ARC job creation 

and retention project evaluation is provided below.

3.2.1 Surveys

A very common approach to assessing project performance used by private and public 

entities is the implementation of a survey. There are three major survey types discussed 

in the literature: cross sectional, panel surveys, and criteria referenced. 

1. Cross sectional: Measurements are made for multiple units at a single point in time. 

2. Panel: Similar to a cross-sectional survey but has the added feature that the 

information is acquired from a given sample unit at two or more points in time. 

3. Criteria Referenced: Enables evaluators to answer normative questions, which 

compare actual performance to an external standard of performance. 
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

16 | September 2015

Surveys are versatile and, when properly done, produce reliable and valid information. 

They require expertise in their design, implementation and interpretation, however, and 

the results can be easily misused. The guidance offered in several commonly referenced 

studies2 recommends several basic steps before implementing a survey.

1. Other evaluation options: Evaluator should check to see if required data are available 

elsewhere and determine whether the survey is the most cost-effective and efficient 

way to collect the data.

2. Good front end planning: 

a. Identify what specific information will address a given evaluation issue;

b. Determine sources of information (e.g., types of respondents);

c. Choose methods to be used for sampling sources (e.g., random sampling) and 

collecting information (e.g., structured interviews and self-administered 

questionnaires);

d. Determine timing and frequency of information collection;

e. Decide on basis for comparing outcomes with and without a program; and

f. Develop the analysis plan.

3. Survey design: Avoid “nice to know” data collection and keep survey succinct and 

clear

4. Survey pretest: Pretest on a representative sample of the survey population. 

Provides information on clarity of questions, response rate, time and length to 

complete, and the appropriateness of survey method. It also allows adjustments to 

the survey approach as necessary.

Applicability to ARC Program Evaluation

ARC utilizes cross-sectional surveys to collect data for program evaluation in a number 

of different areas and then uses this information to estimate the full impacts of a 

program. Previous ARC program evaluations, including both the Infrastructure and the 

Education & Workforce Development Program Evaluations, have successfully employed 

this approach. The ARC.net database of projects includes a significant amount of data 

collected through the use of surveys, which have most recently been conducted online.

3.2.2 Observation or Site Visits

This program evaluation technique involves on-site visits by an evaluator to locations 

where a program is operating to observe activities and to take notes. Program 

participants and leaders may or may not be aware that the program is being evaluated. 

When considering this evaluation approach, it is important to be aware that program staff 

may alter their behavior if they know they are being observed. The following 

recommendations were made in the reviewed guidance:

 2 These studies include, but are not limited to, the Treasury Board of Canada’s “Evaluation Methods: Measurement 

and Attribution of Program Results” and HM Treasury’s “The Magenta Book: Guidance for Evaluation”
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

September 2015 | 17

1. U.S. General Accounting Office3 encourages an evaluator to consider social/cultural 

mores of the area where the evaluation is to be conducted. If the evaluation is in a 

rural area, for example, the evaluator must be sensitive to the difference in culture 

between a rural and urban environment. 

2. Consider whether the population being studied is stable. Are residents transient or do 

people live in the study area for decades? A neighborhood where residence is 

transient may require a different implementation strategy than one in which people 

do not move.

3. Determine whether an on-site visit to evaluate a program is justifiable on a cost 

basis. On-site visits can be very useful but also very costly.

4. Be aware that an evaluator may input some subjectivity in his or her assessment of a 

program. 

5. Keep in perspective that observation offers only anecdotal evidence unless it is 

combined with a planned program of data collection. By itself, it is not grounds for 

generalization about a program.

While this type of evaluation approach may be costly and may be subject to the 

perception of the evaluator, it does offer the opportunity to collect important qualitative 

information that may not be collected through other evaluation methods. The guidance 

offered by the Treasury Board of Canada4 points out that this approach may allow an 

evaluator to see things that may escape staff members or identify issues they are 

reluctant to raise in an interview. This is an important benefit of this evaluation approach.

Applicability to ARC Program Evaluation

Site visits have been a part of previous program evaluations conducted by ARC. In the 

past, these visits have been one approach of several used in these evaluations. As 

another example, the Economic Development Administration (EDA) conducted “Public 

Works Program Performance Evaluation,” in May 1997, which involved site visits. 

Specifically, EDA invited grantees to seminars at thirteen locations across the country. At 

these seminars, they were instructed on technicalities of the team’s information requests 

and the specific information that would be required as part of the program evaluation. 

One-day seminars were held in Atlanta, Austin, Chicago, Cincinnati, Denver, Hartford, 

Little Rock, Los Angeles, Myrtle Beach, Orlando, Philadelphia, San Francisco, and St. 

Louis. Research team members also physically visited 25 percent of the grantee 

locations for site inspections. Information from these visits, combined with survey data, 

was used in the evaluation performance metrics. Like ARC, EDA’s performance 

measures include permanent jobs created and private and public funds leveraged. 

Information related to the diversification of the local economy and tax base added to the 

community was also assembled by EDA program evaluators.

 3 United States General Accounting Office, Program Evaluation and Methodology Division, Designing Evaluations, 

March 1991.

4 Treasury Board of Canada. Program Evaluation Methods: Measurement and Attribution of Program Results, Third 

Edition.
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

18 | September 2015

3.2.3 Analysis of Experimental Data

Based on the Treasury Board guidance, program evaluation through experimental 

designs is the most rigorous approach available for establishing a causal relationship 

between programs and their outcomes. Unfortunately, this approach is impossible to 

implement for many programs, particularly when the program has been running for some 

time.

Experimental designs are characterized by a random assignment of participants to a 

“treatment” or separate “control” group. This process ensures that the groups being 

evaluated are equivalent (other than treatment), and the randomized assignment of 

participants to different groups prior to a program being implemented helps create an 

even playing field from which to draw comparisons, once the program is operational. 

Other related designs include:

1. Quasi-experimental design

a. Do not use randomization to create treatment and control groups. Treatment 

group is usually already given and one or more comparison groups are selected 

to match the treatment group as closely as possible. 

b. Implementation takes creativity and skill to design.

c. Can provide highly accurate findings. 

2. Implicit design

a. Typical evaluation design, but least rigorous. 

b. Post-program evaluation with no control group – make a measurement after 

exposure to the program and then make assumptions about conditions prior to 

the program. 

c. Flexible, versatile and practical to implement.

d. Offers little objective evidence of the results caused by the program.

Applicability to ARC Program Evaluation

A quasi-experimental study funded by a National Science Foundation grant and 

conducted by Andrew Isserman and Terance Rephann performed an empirical analysis 

examining the extent to which ARC projects have stimulated the Appalachian economy. 

The study measures the effects of ARC programs on 391 counties within the region 

using this evaluation approach.5 Additionally, the Appalachian Regional Commission 

conducted a quasi-experimental study examining changes to the region since 1965.6

This study noted that this approach is highly desirable to determine whether or not ARC 

intervention or other factors were the cause of the changes. This approach could be 

useful to ARC in its job creation and retention program evaluations as well, though it 

would be challenging because of the number and range of projects being evaluated, the 

 

5 “The Economic Effects of the Appalachian Regional Commission: An Empirical Assessment of 26 Years of Regional 

Development Planning,” prepared by Andrew Isserman and Terance Rephann, Journal of the American Planning 

Association, Volume 61, Issue 3, 1995.

6 http://www.arc.gov/research/researchreportdetails.asp?REPORT_ID=113
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

September 2015 | 19

cost to conduct this type of evaluation, and the need to determine a statistically valid 

“control” group for comparison purposes.

3.2.4 Before and After Comparisons

Before and after assessments7 compare outcomes for the units of study before the units 

were exposed to a program to outcomes measured after they began to participate in it. 

Unlike experiments, there is no separate comparison group, which is a weakness of this 

approach. This can be mitigated somewhat by evaluating the program at many different 

points in time. With a sufficient number of evaluation points, an interrupted time series 

analysis can be applied to this design to help draw causal inferences. A rule-of-thumb is 

a minimum of 50 before/after observations, based on the evaluation guidance reviewed. 

Data consistency is also an important element to before and after comparisons. The time 

series developed needs to be free of definitional and measurement changes, because 

these can be mistaken for program effects.

Applicability to ARC Program Evaluation

This comparison approach has been utilized by ARC in its prior program evaluations, 

including past studies to estimate the impacts of ADHS highway investments8, though it 

has not been done specifically on job creation and retention projects. The methodology 

to analyze the highway investments involved developing profiles of counties prior to 

project implementation and comparing this baseline to the counties after the program 

was put in place. The ADHS study was strengthened by comparing the results to 

counties with no investment.

3.2.5 Theory-Based Evaluation and Modeling

According to The HM Treasury’s “The Magenta Book, Guidance for Evaluation9,” theorybased evaluation approaches can offer an overarching conceptual framework within 

which specific evaluation studies can be designed. This approach is complementary, and 

not an alternative, to other evaluation approaches. The use of a theory-based evaluation 

can provide new data and evidence that can be incorporated into the evaluation 

framework as appropriate. One practical way to do this is through a simulation model.

Simulation models can be used to combine existing and new evidence to answer 

evaluation questions. It can be subject to some uncertainty, however, due to the need to 

make assumptions about the relationships between different pieces of evidence. 

Simulation models do allow an evaluator to estimate the incremental effects of a program 

in complex and uncertain situations, which is not the case with some other evaluation 

approaches. For very complex programs, evaluation may only be possible through 

theory-based evaluation or simulation modeling.

 7 Implicit Designs and Before/After Comparisons are similar evaluation approaches, but the former makes 

assumptions about conditions before the program was put in place and the latter uses observations both before and 

after the program was implemented.

8 These studies include “Appalachian Development Highways Economic Impact Studies” prepared by Wilbur Smith 

Associates for the ARC in 1998 and “Economic Impact Study of Completing the Appalachian Development Highway 

System” prepared for ARC by Cambridge Systematics in 2008.

9 United Kingdom Cross-Government Evaluation Group (HM Treasury), The Magenta Book: Guidance for Evaluation, 

April 2011.
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

20 | September 2015

Generally, a simulation has three main components based on evaluation guidance10:

1. Input data

2. Mathematical model

a. Stochastic – incorporates a random data generator

b. Deterministic – does not incorporate a random data generator

3. Output data

Simulation resembles other statistical techniques, such as regression analysis, and these 

techniques may actually be used to build a simulation model. Once the model is 

constructed, however, the inputs provided are actually used by the model to generate 

outputs. The output generated by the simulation model can be checked against actual 

real world outcomes.

The guidance suggests that some form of simulation modeling is likely to play a role in a 

large proportion of program evaluations. Where outcomes are expected to occur over a 

number of years, some simulation of these effects might be necessary to ensure that 

evaluation evidence is obtained in a timely fashion. The primary benefit of simulation 

modeling is that an evaluator can estimate incremental effects of a program in a complex 

and uncertain situation. The primary drawback to this approach is that a sophisticated 

understanding of the dynamics of the program is required, as is some skill in building 

quantitative models (or the cost of obtaining models, such as the Regional Economic 

Models, Inc. [REMI] a dynamic, time-series based economic forecasting and simulation 

model).

Applicability to ARC Program Evaluation

ARC has utilized simulation models for its economic impact assessments of programs, 

including IMPLAN, ARC-LEAP, REMI, and TREDIS. These models are primarily used to 

estimate indirect and induced effects based on direct effects elicited through surveys. 

EDA also included the use of simulation models in some of its program evaluations. For 

example, EDA conducted input-output analysis to estimate indirect employment and 

investment impacts (multiplier effects) of EDA investments. They also conducted 

regression analysis to estimate the impact of the EDA investments on countywide 

employment and wages.

3.2.6 Case Studies

Case studies usually involve a variety of data collection methods, including surveys and 

interviews. While it is tempting to do more case studies than fewer, guidance suggests 

that the study of a critical case may be more defensible than the case study of a 

representative sample. Unfortunately, using one case study in an evaluation does not 

enable the evaluator to make generalizations about the overall program. Case studies:

1. Enable an in-depth analysis that would not be possible with more general 

approaches;

 10 Guidance from the Treasury Board of Canada Program Evaluation Methods: Measurement and Attribution of 

Program Results, Third Edition.
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

September 2015 | 21

2. Are typically expensive and time consuming;

3. Usually lack a statistical basis from which to generalize conclusions – it is difficult to 

do enough case studies to provide an adequate sample from which to draw 

conclusions; and

4. Provide broad insights on the program and the real world context of impacts.

The Magenta Book recommends that case studies be carried out before, or in tandem 

with, other data collection efforts. According to the US GAO guidance11 on program 

evaluation, there are generally three types of case studies:

1. Single case study:

a. Information is acquired about a single individual project, process, or entity. 

b. Qualitative information that describes events and conditions from various points 

of view can be collected. 

c. Selection of a single case and the data collection may be challenging. 

d. Analyzing and reporting qualitative data can be difficult, and the evaluation 

design must have explicit plans for these tasks. There are several different types 

of single case studies based on the literature:

• Illustrative – Describes event or condition.

• Exploratory – May serve as a precursor to a larger evaluation or provide 

preliminary information.

• Critical instance – Investigates one problem or event.

2. Multiple case studies: 

a. Likely to produce stronger conclusions than a single case study.

b. May be appropriate for evaluating either program operations or results. 

c. Selecting variety among cases is important to avoid bias, as is uniform 

information collection.

3. Criteria-referenced: 

a. Can be adapted to answer normative questions about how well program 

operations or outcomes meet certain performance criteria. 

b. Carefully choosing the sample of cases to evaluate is critical.

Applicability to ARC Program Evaluation

A case study can help develop a clear, well-reasoned and comprehensive understanding 

of the situation, project or people affected by a program. Case studies are often used to 

supplement or complement a more quantitative evaluation approach, helping evaluators 

better understand the causal logic. 

 

11 United States General Accounting Office, Program Evaluation and Methodology Division, Designing Evaluations, 

March 1991.
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

22 | September 2015

The “Program Evaluation of ARC’s Tourism, Cultural Heritage, and Natural AssetRelated Projects” utilized case studies, as well as a survey and information obtained 

from other reports, in its evaluation. For the job creation and retention program 

evaluation, case studies of 12-15 projects are expected to help highlight best practices of 

existing programs, as well as provide context for the data analysis components of the 

evaluation.

3.2.7 Application of Other Studies

A literature review of other studies, or a file review of data and information, enables an 

evaluator to make the best use of previous work in the field under investigation. 

Application of other studies can help: a) to determine the best analytical methods and 

performance measures to use; and b) use assumptions and data findings to help 

compare and estimate program impacts. In addition, the evaluator may learn from the 

experiences, findings, and mistakes of other entities that have conducted program 

evaluations. Past research can suggest hypotheses to be tested or evaluation issues that 

should be examined in the current study, and specific methodological difficulties can be 

identified and mitigated. In some cases, evaluation questions may be directly answered 

on the basis of past work or redundant data collection. 

Another advantage to reviewing other studies is that sources of secondary data may be 

uncovered, which may lessen the need to collect primary data. While this evaluation 

approach is economical, it is not always appropriate for a particular program evaluation. 

Additionally, when using other studies to evaluate a particular program, it is sometimes 

difficult to determine the accuracy of the data gathered through a previous study.

Many project evaluations utilize a review of other studies as a first step in a larger 

evaluation of a program. ARC has employed this approach in its work, and other 

organizations have incorporated the same sort of preliminary review of studies prior to 

embarking upon a larger program evaluation.

Applicability to ARC Program Evaluation

Reviews of other studies are primarily applicable in terms of determining methodology 

and performance measures for the job creation and retention evaluation. We anticipate 

cross-checking results from the new program evaluation with the results from other 

program evaluations that have attempted to measure similar projects.

3.2.8 Other Potentially Applicable Evaluation Approaches

Other applicable approaches to program evaluation were discussed in the studies

reviewed by the team. These methods are generally sub-sets of or related to the overarching methods described above.

Use of Expert Opinion

Collecting data from experts is a method best suited to supplementing other measures of 

program outcomes. An advantage to this approach is that data can be collected and 

summarized systematically. In addition, the opinions of experts may vary and experts 

may not be equally knowledgeable about a program or issue. While this approach should 
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

September 2015 | 23

not be used as a sole source of data for an evaluation, it can help fill in the gaps or offer 

detail on a particular program. For ARC program evaluations, this approach could be 

incorporated as part of a case study or survey effort.

Statistical Analysis and Causal Models

Statistical analysis and development of a causal model requires expertise, and not all 

programs can be analyzed using these approaches. A causal model is based on an 

equation that describes the marginal impact of a set of selected independent variables 

on a dependent variable. This approach focuses on variables to be included in the model 

(endogenous and exogenous) and their postulated relationships. Regression analysis is 

a technique employed to measure the impact of one variable or variables on another, the 

causal relationship. With respect to program evaluation, the program is only one of 

several independent variables that are expected to affect the dependent variable. 

Determining when it is appropriate and how to apply statistical analysis to measure 

causality is the focus of Impact Analysis for Program Evaluation.12 The author also 

discusses the use of experiments and quasi-experiments as part of an overall evaluation 

that includes statistical analysis.

Input-Output (I-O) Modeling

A variation of theory-based evaluation and modeling, this type of approach consists of 

using a model designed to depict the mutual interdependence among different parts of 

an economy. It describes how one industry uses the outputs of other industries as inputs, 

and how its own outputs are used by other companies as inputs. ARC uses I-O models 

to measure the total economic impacts (including multiplier effects on indirect and 

induced demand) of direct project impacts, which are often obtained through interviews 

with grantees. While useful in better understanding the impact of a program, these types 

of models and their results can be misused in evaluations. For example, offsetting 

negative effects generated by taxes and borrowing necessary to support a program need 

to be factored into an I-O model. Otherwise, the impacts of the program on the economy 

may be overstated. Other challenges include measuring the direct impacts of a program 

in terms of additional spending, employment, etc., or controlling for potential 

displacement of activity from other sectors of the economy (e.g., new retail activity that 

displaces or impacts existing retail establishments).

EDA and ARC have utilized I-O models in their program evaluations to estimate total job 

impacts, business sales, and other economic variables. 

Benefit-Cost Analysis

Benefit-cost analysis looks at a program’s net worth and assesses and compares 

program alternatives. The issue of attribution and incrementality of effects over a “base 

case” must be addressed prior to doing this type of analysis. Benefit-cost analysis is 

more a comprehensive approach to assessing the effects of a program or project, than 

an approach to measuring its performance. It uses the results of interviews, experiments, 

case studies, and so forth, as inputs and produces summary measures of monetary 

value. Most typically, benefit-cost analysis is used to assess the future impacts of 

 12 Impact Analysis for Program Evaluation, Second Edition, by Lawrence B. Mohr, 1995.
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

24 | September 2015

proposed investments, although it can be used for program evaluation if benefits can be 

measured and isolated. Regardless of the timing of this analysis, the guidance 

recommends that a sensitivity analysis of key assumptions be conducted if this analytical 

approach is employed.

Sensitivity Analysis and Risk Analysis

Analytical work is often directed toward a single “expected outcome,” supplemented with 

alternative scenarios. The limitation of this approach is clear – while it may provide the 

single best statistical estimate, it offers no information about the range of other possible 

outcomes and their associated probabilities. 

There are two ways to mitigate this issue. One common approach to providing added 

perspective on reality is “sensitivity analysis.” Key assumptions are varied one at a time 

in order to assess their relative impact on the expected outcome. The problem here is 

that the assumptions are often varied by arbitrary amounts. A more serious concern with 

this approach is that, in the real world, assumptions do not veer from actual outcomes 

one at a time. It is the impact of simultaneous differences between assumptions and 

actual outcomes that is needed to provide a realistic perspective on the riskiness of a 

forecast. 

Risk Analysis avoids the problems outlined above by measuring the probability or “odds” 

that an outcome will actually materialize. This is accomplished by attaching ranges 

(probability distributions) to the forecasts of each input variable. The approach allows all 

inputs to be varied simultaneously within their distributions, thus avoiding the problems 

inherent in conventional sensitivity analysis. The approach also recognizes 

interrelationships between variables and their associated probability distributions. 

Sensitivity and risk analysis can be useful as part of an overall program evaluation.13

Comparison to Current Program Evaluation Approaches Used by ARC

Generally, ARC has evaluated its investments using a combination of approaches 

applied to a sample of ARC projects. The sample is selected to be representative of the 

mix of project types and area characteristics within the region. The first step in the 

evaluation has been a review of ARC records and classification of the programs into a 

database. A phone or online survey of several local or regional development staff, local 

government and civic leadership, and/or private sector representatives was then 

conducted. Information collected through the survey and other secondary data sources 

were used to construct project profiles. Site visits were then made to validate project 

results and to develop more detailed case studies. Site selection has historically been 

based on project type, geography, and area demographics.

In past studies, the projects were evaluated by comparing the anticipated and actual 

project outcomes in terms of key performance measures. In the initial project applications 

for ARC funding, local applicants are required to estimate the number of jobs to be 

created or retained, the number of businesses to be served or retained, and the number 

of workers or trainees to be impacted directly by the project. This information is 

considered the “anticipated” outcomes.

 13 As an example, see the Federal Highway Administration (FHWA) Economic Analysis Primer, August 2003.
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

September 2015 | 25

ARC studies that use several evaluation approaches are consistent with guidance on 

program evaluation and similar to how some other agencies assess their program’s 

impacts. The evaluation guidance that was reviewed suggests that utilizing multiple 

evaluation approaches is useful in that each approach may help fill in the gaps that may 

occur when using only a single approach. ARC’s decision to utilize surveys, on-site visits 

and case studies, as well as before and after comparisons aided by an input-output 

model, for many of their evaluations is consistent with the program evaluation guidance 

reviewed by the team.

3.3 Summary and Conclusions

The approaches and methods described in this section are summarized in Table 1 

below. The table includes a brief description of the methods, identifies the specific 

questions being addressed, and assesses the extent to which the methods have been 

used in the evaluation of ARC programs.

Table 1: Summary of Approaches and Methods

Approaches & 

Methods

Brief Description Principal Strengths & 

Weaknesses

Use in ARC 

Evaluations

Surveys Collecting data or information 

directly from those affected by a 

program or action, typically 

through a mail or online 

questionnaire; or through phone 

interviews. May be crosssectional, panel, or criteria 

referenced.

Allow gathering large amounts of 

performance data at a relatively 

low cost. Accuracy of data 

depends on survey and 

questionnaire design, as well as 

quality of information available to 

interviewees. Can only be used 

to evaluate the direct effects of a 

program or action.

Yes, 

extensive

Observations or 

Site Visits

Visits by an evaluator to 

locations where a program is 

operating, to observe activities, 

assess changes, and take notes.

Allow gathering detailed, 

qualitative information on the 

effects of a program at a specific 

location, or set of locations. But 

is often costly to implement; and 

findings may be biased by 

evaluators’ perception.

Yes

Analysis of 

Experimental Data

Comparison of performance 

between a treatment group and 

a control group. Design may be 

experimental (with random 

assignment of participants), 

quasi-experimental (without 

randomization), or implicit 

(without control group).

Considered most rigorous 

approach to establishing causeand-effect relationships between 

programs and observed 

changes. But is generally difficult 

and costly to implement, and 

may not be suited to all 

programs and actions.

Limited

Before & After 

Comparisons

Comparison of performance 

before and after implementation 

of a program, for a single group 

of units or participants.

Effective in measuring changes 

over time, but causal 

relationships between programs 

and outcomes may not be 

formally established.

Yes
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

26 | September 2015

Approaches & 

Methods

Brief Description Principal Strengths & 

Weaknesses

Use in ARC 

Evaluations

Theory Based 

Evaluation & 

Modeling

Use of a conceptual framework 

and/or model to develop 

assumptions about, or simulate 

the impacts of a program.

Must be used in conjunction with 

other approaches and methods. 

Particularly effective in 

estimating the impacts of 

complex programs. If not 

properly documented, simulation 

models may be viewed as “black 

boxes,” limiting the applicability 

of their output.

Yes, in the 

estimation of 

indirect and 

induced 

economic 

impacts

Case Studies Detailed analysis of a critical 

case or a limited set of cases, 

involving a variety of information 

gathering techniques (including 

interviews, site visits, etc.).

Provide in-depth information and 

analysis of effects, but can only 

be applied to a limited number of 

units or locations. Results can 

rarely be generalized. 

Particularly useful in validating or

augmenting the findings of a 

broader data collection effort 

(e.g., survey). 

Yes

Application of 

Findings from 

Other Studies

Use of methods, data sources, 

assumptions and/ or estimates 

from existing studies.

Useful in methodology 

development stages. “Transfer” 

of impact estimates and other 

quantitative or qualitative 

findings are generally low-cost, 

but not considered best-practice.

Only to 

inform 

selection of 

methods 

and 

performance 

measures

Use of Expert 

Opinion

Eliciting information directly from 

Subject Matter Experts, to 

supplement other data sources 

and collection efforts.

Particularly useful when 

observation and traditional data 

collection techniques fail to 

produce adequate results. May 

also be used for review and 

validation. Requires use of 

formal elicitation techniques to 

limit potential for bias.

No

Statistical Analysis 

and Use of Causal 

Models

Use of statistical methods, 

including regression analysis, to 

estimate “causal” relationships 

between program attributes and 

performance. Often used for the 

analysis of experimental or 

quasi-experimental data.

Allows formal hypothesis testing, 

but robustness of results 

depends on quality of data and 

survey design (see Analysis of 

Experimental Data). Is not 

applicable to all types of 

programs.

Limited

Input-Output 

Modeling

Use of accounting tables tracing 

linkages of inter-industry 

purchases and sales to estimate 

the indirect and induced 

economic impacts of a program 

or action.

A sub-set of Theory Based 

Evaluation & Modeling. 

Economic impacts estimated

with off-the-shelf I/O models are 

typically gross impacts, ignoring 

many further possible 

interactions and responses.

Yes, in the 

estimation of 

total 

economic 

impacts
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

September 2015 | 27

Approaches & 

Methods

Brief Description Principal Strengths & 

Weaknesses

Use in ARC 

Evaluations

Benefit-Cost 

Analysis

Estimation of the costs and 

benefits of a program, defined in 

terms of well-being (welfare) with 

a view to providing an overall 

assessment of value-for-money. 

Is not a performancemeasurement approach per se.

Primarily used for program 

assessment, ex-ante; but may 

be used ex-post to estimate the 

return on investment of existing 

programs (based on 

performance measures 

developed through other 

means).

No

Sensitivity 

Analysis or Risk 

Analysis

General approaches to 

incorporating uncertainty into 

performance measurement and 

program evaluations. Can be 

applied in conjunction with any 

of the above quantitative 

methods.

Assists decision-makers in 

assessing the degree of 

confidence they should place in 

the outcomes of an evaluation.

No

Table 2 evaluates each approach or method against a set of criteria, defined as follows:

• Ease of use in ARC evaluations: whether, once developed, the method and 

associated tools can be used by local, regional and ARC staff with a range of 

technical expertise;

• Ease of interpretation: whether the resulting performance measures can be 

easily understood and interpreted;

• Flexibility: whether the method can be applied to a wide range of conditions and 

projects;

• Data availability: whether the data required to apply the method are readily 

available and/or accessible at a reasonable cost;

• Ability to “attribute” impacts: whether the contribution of the program to 

observed outcomes can be formally tested;

• Estimation of counterfactual: whether the method requires use of a control 

group and/or estimation of a counterfactual;

• Transparency: whether all steps of the method can be described clearly, 

including where applicable the “logic” used to derive outcomes from output 

measures; and

• Acceptance and credibility: whether the method is widely accepted and/or 

conforms to industry standards.

The ratings shown in the table were developed, somewhat subjectively, by the Project 

Team. The criteria are rated on a scale of 1 to 4 (stars), 4 representing the easiest or 

most flexible method, and 1 the least.
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

28 | September 2015

Table 2: Rating of Approaches and Methods against Criteria

Criteria

Approaches & Methods

Ease of Use for ARCEase of InterpretationFlexibilityData AvailabilityEstimation of CounterfactualTransparencyAcceptance & Credibility

Surveys     No  

Observations or Site Visits     No  

Analysis of Experimental 

Data

    Yes  

Before & After 

Comparisons

    No  

Theory Based Evaluation 

& Modeling

    Sometime  

Case Studies     Generally 

Not

 

Application of Findings 

from Other Studies

    No  

Use of Expert Opinion     Not 

Explicitly

 

Statistical Analysis and 

Use of Causal Models

    Sometime  

Input-Output Modeling     No  

Benefit-Cost Analysis     Yes, 

Needed

 

Sensitivity Analysis or 

Risk Analysis

    N/A  

Chapter 4 of this report provides the annotated bibliographies for the five studies 

reviewed for this program evaluation.
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

September 2015 | 29

4 Review of Past Reports and Program 

Evaluations

4.1 “Program evaluation of ARC’s Tourism, Cultural 

Heritage, and Natural Asset-Related Projects”

This report was prepared for the Appalachian Regional Commission (ARC) by Regional 

Technology Strategies, Inc. with Mt. Auburn Associates and Appalachian State 

University in September 2010.

4.1.1 Purpose and Goals

The purpose of the evaluation was to examine and critique 132 of ARC’s investments in 

Tourism, Cultural Heritage and Natural Asset-Related projects. These projects were 

analyzed within the context of ARC’s 2005-2010 Strategic Plan.

4.1.2 Key Issues Identified

1. Grantees used inconsistent and unsound methods to document employment 

impacts. Some grantees did not collect any data that can be used to analyze the 

impact of these projects. Methods exist to measure the economic impacts associated 

with tourism projects, but grantees did not typically use them. For example, economic 

impacts can be quantified by: tracking the number of users of marketing tools or the 

number of visitors to tourist attractions; surveying visitors to find out what role the 

marketing or tourist attraction had in bringing them to the area and the amount of 

visitor spending that occurred; and using economic models to estimate the impact of 

visitors. Some grantees reported increased visitation to tourism attractions or the 

number of visits to tourism websites, as proxies for employment impacts. Getting all 

of the grantees to use a single proven method to collect or estimate economic 

impacts would improve the quality of data.

2. Some of the employment impacts will not happen until after project reports are 

written and the current metrics may not accurately capture the impacts. Most of 

the data received by ARC come from final reports that are due 30 days after the 

project is completed. Employment impacts typically take place after that 30 day 

period. Post-project reporting could help solve this problem, although this would likely 

add more issues related to the reporting cost and implementation. The current 

categorization of “job creation” and “job retention” impacts does not always lend itself 

to a clear definition that can be used to generate a useful estimate of the actual 

overall impacts for these types of projects.

3. Leveraged private investment (LPI) is difficult to measure for most projects.

However, LPI is generally considered by ARC to be a proxy for job creation and 

retention impacts. Due to the difficulty in measuring LPI, a more efficient approach 

could be to collect local sales data and use an economic impact model to generate 

job impacts. This can only be done if reasonably accurate sales data generated from 

the projects could be collected.
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

30 | September 2015

4. Visitation and visitor spending are affected by factors other than ARC-Funded 

projects. There are many factors that influence an individual’s decision to visit a 

particular location, including available tourist attractions, exposure to marketing, the 

local economy, travel time required, cost to visit, and local crime levels. While 

projects designed to improve or repair tourist locations may have an impact on the 

decision-making process, it is impossible to determine whether the investment 

impacts visitation and visitor spending in any significant, measurable way.

5. Differences in project success definitions may lead to different measurements 

of results. The results of the study indicated that in two of the three categories 

where actual results were able to be compared to predicted results, the actual results 

reflected much greater impacts than the predicted. The ratio of the actual to 

predicted results suggests that this is most likely due to differences in definitions of 

the criteria, such as “participants served” and “programs and plans developed.” 

Changes in staff and the passing of time can lead to inconsistencies in these 

definitions which may generate falsely positive results.

4.1.3 Key Performance Measures

The study used a variety of performance measures to gauge the extent to which projects 

achieved their projected outcomes and aligned with the strategic goals of ARC. These 

measures were both qualitative and quantitative, including:

1. Initial projections and actual outcomes on key parameters, including numbers of jobs 

created and retained, leveraged private investment, and new businesses created.

2. Initial projections and actual outputs, such as people served, businesses served or 

assisted, and programs or plans created.

3. Impacts related to the Triple Bottom Line including competitiveness measures such 

as job stability, productivity, and access to markets, and other social measures such 

as civic life, health, education, public safety, etc.

4.1.4 Analytical Approach

The RTS team received 212 projects from ARC to use in their evaluation. However, 

several of these projects were removed from the evaluation because they were either 

recently used in an entrepreneur evaluation, projects that were solely intended to create 

a strategic plan, or were considered to be minor projects, such as funding attendance to 

a conference. Other projects were removed from the evaluation if they were from before 

2000 or if they were funded after 2006 because it was determined that data for these 

projects would either be too difficult to collect or not yet available. After removing these 

projects, RTS had 132 projects left for their evaluation, including 72 closed and 60 open 

projects between 2000 and 2006. The sample includes projects from every state except 

South Carolina. The overall analytical approach is detailed as follows:

1. Survey: Two electronic surveys, one qualitative and one quantitative, were sent to 

project managers or other knowledgeable individuals within the grantee organization. 

The first survey was completed by 51 out of 98 recipients, or 52 percent; the second 

survey had a 60 percent completion rate, with responses from 69 of the 114 
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

September 2015 | 31

recipients. When an individual qualified to take the survey could not be identified, no 

survey was sent, thus further reducing the number of projects surveyed.

a. Survey 1: The first survey focused on qualitative data such as the project 

manager’s perception of project success and discussion of evaluation measures. 

This survey also asked respondents to qualitatively assess the Triple Bottom 

Line (TBL) for economic, social, and environmental impacts. For this part of the 

survey, respondents were asked about the impact their project had on categories 

including tourism revenues, visual landscape, and quality of jobs. Respondents 

answered with either “positive impact,” “slight impact,” “no impact,” or “don’t 

know.”

b. Survey 2: The second survey focused on more quantitative data and was used to 

measure the outputs and outcomes of the project and visitation patterns. Where 

possible, the actual outputs and outcomes from the survey results were 

compared to the predicted values in the ARC.net database. This survey also 

measured impacts across the TBL, including economic, efficiency, social, and 

environmental measures. The TBL measures were rated on a qualitative scale in 

this survey as well.

2. Interviews: Thirty-two project managers and 61 stakeholders from 32 projects were 

interviewed. Generally speaking, participants’ attitudes were positive and optimistic 

about their projects’ outcomes and the importance of the role served by ARC, both in 

terms of funding and supporting grantees.

3. Case Studies: Seven case studies were conducted at sites throughout the 

Appalachian Region. These case studies were selected because they covered 

several different project factors, including focus, project size, and projected 

outcomes. The case studies highlighted several common outcomes that may be 

useful in preparing for future studies. Some highlights include:

a. Flexibility in project implementation is helpful in achieving success as projects 

tend to evolve from conception to completion.

b. Impacts often extend beyond initially anticipated areas in terms of both spin-off 

and impacting unrelated businesses.

Impacts are often difficult to track and estimate, posing some data collection and analysis 

issues. The spillover impacts are difficult to measure and the impacts being measured 

are not always the most important.

4.2 “Creating an Entrepreneurial Appalachian Region: 

Findings and Lessons from an Evaluation of the 

Appalachian Regional Commission’s 

Entrepreneurship Initiative 1997-2005”

This report was prepared for the Appalachian Regional Commission (ARC) by Deborah 

Markley, Brian Dabson, Thomas Johnson, and Karen Dabson of the Rural Policy 

Research Institute Center for Rural Entrepreneurship, Sara Lawrence and Sara Yanosy 

of RTI, and Erik Pages of EntreWorks Consulting in April 2008.
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

32 | September 2015

4.2.1 Purpose and Goals

The goal of this study was to review and evaluate ARC’s investments in 

entrepreneurship development. Specifically, this study evaluated the Entrepreneurship 

Initiative (EI) by looking at the outcomes from the sample projects as well as the broader 

policy impacts within the region. The evaluation sought to assess the three primary goals 

of the initiative – increasing the number of entrepreneurs establishing businesses in the 

region, increasing the survival rate of these businesses, and increasing the share of 

these businesses that grow to create jobs and wealth within the region. These objectives 

all support the strategic goals of ARC.

4.2.2 Key Issues Identified

Evaluating entrepreneurship investments can be a difficult task as there are no standard 

metrics for measuring success and no requirements for tracking outcomes of these 

investments. Several issues related to the evaluation were highlighted in this review:

1. Entrepreneurship is a difficult concept to measure. Many aspects of 

entrepreneurship are intangible, thus causing difficulty in qualitatively or 

quantitatively measuring outputs or outcomes of these investments. When program 

managers use traditional economic development metrics to gauge success, they are 

unable to accurately capture the full range of impacts generated by these projects. 

There is a layer of complexity associated with meeting the traditional ARC goals and 

metrics with these types of projects.

2. This study relied heavily on self-reporting. Self-reporting can lead to inaccurate 

assumptions about the outcomes of the projects. The projects were contractually 

obligated to submit final reports which were substantiated through audits and 

verification field visits to help ensure accurate reporting of results.

3. Statistical models and methods were difficult to perform considering the small 

and diverse sample. However, the descriptive statistical methods used, including 

the detailed interviews, helped to answer the research questions and provide 

appropriate recommendations for further funding.

4.2.3 Key Performance Measures

This evaluation interviewed project staff and stakeholders to determine the effectiveness 

of the entrepreneurship investments. Given the nature of the evaluation, the study team 

chose a before and after approach to assess the impacts of the project. Due to the 

diverse nature of the set of projects, much like the Job Creation & Retention projects 

being evaluated in this study, a broad set of metrics was required to show the impact of 

these diverse projects on the region. The team created a series of indicators for both 

outputs and outcomes that varied slightly to capture the best information for each type of 

project.

The following list indicates some of the performance measures used to gauge the extent 

to which projects achieved their anticipated outcomes and aligned with the strategic 

goals of ARC:

1. Businesses served
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

September 2015 | 33

2. Jobs created

3. Jobs retained

4. Project participants

a. Training in entrepreneurship activities

b. Sector focused activity participation

c. General training and technical assistance

5. New businesses created

6. Private investment leveraged

7. Collaboration changes, and partnerships created

8. Change in sales

9. Performance improvements in terms of participation increases and performance of 

participants

4.2.4 Analytical Approach

The evaluation reviewed 229 closed projects out of the 448 total projects in five main 

categories that were funded from 1997 through 2005: 18 capital access projects, 42 

entrepreneurship education projects, 30 incubators projects, 40 sectors14 projects, 71 

technical assistance and training projects, and 28 other/not coded projects. The sample 

size was narrowed to 229 by removing any projects that were not closed (94) or that 

were funding for follow-on investments related to the initial project (125) to only 

investigate unique, closed projects. All grant information related to each project, even if it 

was for multiple grants, was reviewed for the analysis. Projects came from every state in 

the region.

Recognizing that entrepreneurship projects are diverse, the study team conducted a 

detailed review of a sample of projects and implemented a thorough interview process 

with key personnel and stakeholders for these projects. Data for this analysis were 

collected from various sources in a four-step process. The first step was to review ARC’s

project file to gather information about the project and implementation. The second step 

was to conduct phone interviews with the project directors to gather additional 

quantitative and qualitative data related to the project. The third step was to conduct 

phone interviews with project and non-project stakeholders to gain data on impacts 

throughout the region. The final step was to conduct site visits to gather data on placebased and policy impacts. The information gathered from these sources was analyzed 

using a metrics framework developed by the team to conduct a meta-analysis of the 

outcomes and impacts of these projects at both the local and regional levels.

 14 Sectors projects combine aspects of the other types of projects but are targeted at a single business sector.
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

34 | September 2015

4.3 “Evaluation of the Appalachian Regional 

Commission’s Community Capacity-Building 

Projects”

This report was prepared for the Appalachian Regional Commission (ARC) by Westat in July 

2004.

4.3.1 Purpose and Goals

The study sought to evaluate the success of community capacity building projects at 

meeting their original objectives, their success at achieving the Appalachian Regional 

Commission’s strategic goals, and the direct community impacts. Though there is not 

one specific definition of community-capacity projects, they all aim to achieve the 

common goal of developing resources to improve the conditions and quality of life of 

people within communities. The evaluation also evaluated best practices and policy 

changes that could improve future ARC program evaluations.

4.3.2 Key Issues Identified

Two key data-collection issues were identified during the process of this evaluation: 

reliance on self-reported data, and lack of consistency in data-collection requirements 

and performance measurement.

The first issue is that performance measurement for a given project was based off of selfreported data. Several issues are related to the reliance on self-reported data, including 

the lack of evidence to corroborate the reports. Two primary contributors to the lack of 

data are that some projects did not take the time to collect the data, and others lacked 

clear, measurable goals.

Compounding the data collection issue is the fact that projects funded before 1998 were 

not subjected to the same application guidelines as those funded after. The study period 

was from 1995 to 2003, so not all projects had the same data collection requirements. 

Until 1998, ARC had few guidelines about the inclusion of outcomes in applications and 

final reports. In 1998, ARC developed application guidelines that were oriented toward 

the Government Performance and Results Act (GPRA), which helped to improve the 

quality and consistency of reports. These issues result in a general lack of consistency in 

evaluating the purpose, outcomes and impacts of these types of projects.

4.3.3 Key Performance Measures

The study proposed 179 outcomes related to individual, organizational, and community 

based capacity improvements as well as economic, environmental and social 

development outcomes. The results suggested that most of the outcomes were related to 

short-term rather than long-term goals. These performance measures were used to 

assess whether the outcomes had been achieved according to ARCs performance 

measurement system.

The projects were evaluated based on their impacts on individuals, the enhancement of 

skills, enhancement of empowerment, increased awareness of community issues, their 

impact on organizations, increases in collaboration, stability and growth, impacts on 
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

September 2015 | 35

communities, civic participation, and community pride among others. Due to the difficulty 

in defining outcomes and collecting quantitative data, most of the measures provided 

qualitative outcomes. The wide range of measures used in this study will be considered 

when determining measures for the Job Creation & Retention evaluation.

4.3.4 Analytical Approach

ARC funded 168 community capacity-building projects between 1995 and 2003. This 

study reviewed 100 of these projects, which were divided into 11 activity types that were 

grouped into four overall strategies: vision and direction, involvement, skills and 

knowledge, and support.

The evaluation approach involved four steps. The first step was to conduct a literature 

review to help the authors better understand the concepts and parameters associated 

with community capacity building projects. The second step was to review the 

applications and final reports of the selected 100 projects. The third step was to conduct 

25 telephone interviews to obtain information about project activities and 

accomplishments for a representative sample. The final step was to conduct 12 site visits 

to supplement the data found in the other steps with more specific information related to 

the performance measures. The site visits were selected from projects that implemented 

innovative practices, achieved their desired outcomes, and had measureable 

performance indicators. These projects were selected to form a basis for best practices 

and the discussions focused on project context, approach, implementation, problems and 

obstacles, attainment of the goals, lessons learned and recommendations for 

performance measures.

The study also looked at the problems that arose during project implementation and how 

they were handled, beneficial practices that could be adopted elsewhere, and possible 

policy changes that could improve future ARC program evaluations.

4.4 “Evaluation of the BITS Incubator Program & the 

Intelligent Island Incubator”

This report was prepared for the Australian Government’s Department of 

Communications, Information Technology and the Arts, by the Allen Consulting Group in 

November 2003.

4.4.1 Purpose and Goals

The objective of this evaluation was to assess the success and progress of the Building 

on Information Technology Strengths (BITS) Incubator Program. The BITS Program was 

funded by the Australian Government, and sought to build upon IT strengths to promote 

employment growth, local wealth creation, and entrepreneurship in the areas of 

information and communications technology (ICT). The BITS Incubator Program was 

established in June 1999 with a $158 million commitment over five years to support three 

distinct areas: incubators to assist ICT small to medium enterprises ($78 million); 

advanced networks and test-beds ($40 million); and developing Tasmania as an 

“Intelligent Island” ($40 million).
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

36 | September 2015

This evaluation focused on the Incubators and the Intelligent Island. The BITS Incubator 

Program supported business incubators that help entrepreneurs turn their technology 

and ideas into successful and competitive businesses. The Intelligent Island Program 

promoted employment growth and wealth by accelerating the growth of the ICT 

industries. This evaluation identified the key design factors impacting the success of the 

incubators, and assessed the contribution of the BITS Incubator program as part of the 

broader range of government programs supporting start-up and early-stage companies in 

Australia’s ICT innovation system.

4.4.2 Key Issues Identified

Several issues were identified during the course of the evaluation. The first issue is that 

the Program was created during the height of the technology boom of the early 2000’s, 

with Incubators structured under the presumption that the growth would continue. The 

bursting of the technology bubble forced the Incubators to reconsider their approaches 

and structures to accommodate the changing economic climate.

Another set of issues involves the ability to compare metrics and performance across 

Incubators. The incubators within the BITS programs are located in diverse and different 

geographic areas and have adopted different investment strategies, creating difficulty in 

making comparisons across the various sites. These Incubators were also relatively 

“immature” at the time of evaluation, with the average time of operation being three 

years. It typically takes approximately eight years for business incubators to fully 

establish themselves.

Finally, a traditional measure of success is return-on-investment, frequently measured in 

terms of revenues or profits. For the BITS program, this measure proves difficult due to 

the prevalence of non-profits within the Incubators.

4.4.3 Key Performance Measures

Due to the differences in the nature of the various companies and the business models 

of the various Incubators studied, each Incubator had relatively tailored performance 

indicators in order to best capture success. That said, several broad performance 

measures were identified to allow for comparison across the investments. These 

measures included those at the program level, those that measured the extent to which 

Incubators supported Incubatees, and quantitative assessment measures.

At the program level, these performance measures included: ability to leverage program 

funds, both cash and in-kind; the total in-kind contributions leveraged, which reflects the 

ability to convince others to contribute; and the extent to which the Incubators have 

achieved self-sustaining operations.

Measures that identified the level to which Incubators have supported Incubatees 

include:

• Support for Incubatees – a measure of the extent to which government and 

other resources reach the Incubatees, indicating the efficiency and 

effectiveness of the Incubator;

• Incubatee achievements – a measure of increased revenue and 

employment;
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

September 2015 | 37

• Incubatee capital leveraged – a measure of the amount of additional 

funding that the Incubatee gathered;

• Incubatee access to business angels – a measure of access to those who 

bridge the gap between the Incubator and venture capital providers;

• Incubatee/graduate sustainability – a measure of the extent that the 

business remains viable after the intensive assistance ends; and

• Satisfaction of Incubatees – a measure of the overall satisfaction of the 

participants with the Incubator.

Development of quantitative measures to fully evaluate the success of the Incubators is 

difficult due to several factors, including the short operating duration, different operating 

locations, and different business models. Though there are difficulties, and it will take 

time to fully evaluate the success of the program, trends among several quantifiable 

measures can be seen. These include:

• Efficiency in leveraging government funding – measured as the dollar 

value of Incubator support divided by the total program funds spent;

• Extent to which government funds went to Incubatees – measured as the 

dollar value of support to Incubatees divided by the percent of Program funds 

spent;

• Effectiveness of capital raised to date – measured by the total Incubatee 

funds divided by the Program funds spent to date;

• Effectiveness of increase in revenue to date – measured by the increase 

in Incubatee and graduate revenue divided by the total program funds spent;

• Effectiveness of Incubatee assessment of services provided – measured 

by the percent of Incubatee and graduates who rated the Incubator in-kind 

services as “good”; and

• Overall utility of the Program – measured by the percent of Incubatees and 

graduates rating the Incubator as “good” or better.

4.4.4 Analytical Approach

The evaluation used a variety of qualitative and quantitative factors to measure the 

success of the Incubator Program. The first step in the approach was to identify the basic 

data to be gathered from the Incubators, Incubatees, and graduates to best evaluate the 

overall program. This input was supplemented with information from State government 

officials and venture capitalists to complete the assessment. The data collected for this 

evaluation was gathered by two primary methods: structured questionnaires and 

stakeholder interviews.

The surveys were aimed at gathering information on the Incubatee experience by polling 

all Incubatees, Graduates, and companies that have withdrawn from the Program before 

graduating. A total of 254 Incubatee surveys were distributed via the Incubators with 100 

responses received. The survey contained various questions regarding the incubation 

experience including timing, funding and services, and their opinion of the benefits of 

incubation.
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

38 | September 2015

The stakeholder interviews included Incubator management, staff, and some Board 

members; a sample of the Incubatees; some State government officials associated with 

the Incubators; and a number of venture capitalists and financiers also associated with 

the Incubators. The interviews were structured based on a list of questions that reflect 

the Incubators each took a slightly different approach to aiding the start-ups and 

supported different business models. The interviews were designed to elicit qualitative 

information that could be readily gathered while still providing real-world perspective on 

how the BITS Incubators operate. The questions focused on gathering inputs and 

outputs on five key best practice areas – efficiency, effectiveness, relevance, utility, and 

sustainability.

4.5 “Implementation Evaluation of the CommunityBased Job Training Grant (CBJTG) Program”

This report was prepared for the United States Department of Labor by the Urban 

Institute in December 2012.

4.5.1 Purpose and Goals

The purpose of this evaluation was to provide a comprehensive picture of the varying 

initiatives funded by the Community-Based Job Training Grant, including innovations 

developed, successes and challenges to date, and trends and patterns across projects.

The results of these CBJTG-funded initiatives are considered through the lenses of 

program sustainability, accomplishments, implementation issues, and performance.

The Community-Based Job Training Grant was established by the U.S. Department of 

Labor’s Employment and Training Administration in 2005. This grant program sought to 

strengthen regional economic competitiveness by addressing critical shortages at 

community and technical colleges in order to train workers in high-growth occupation 

fields. Grants were intended to help these institutions design and implement sustainable 

training programs to effectively provide workers with skills needed for specific industries, 

through providing a variety of resources. Between 2005 and 2009, CBJTG funding was 

provided for 279 individual initiatives in 49 states. There were four separate rounds of 

competitive funding, with between 68 and 72 grants awarded per year. Grants were 

typically awarded for an initial period of three years, with an option for no-cost 

extensions. Funding amounts ranged from $499,014 to $3,600,768. Due to a decrease in 

federal funding, all initiatives funded to start in 2007 or later experienced a one-percent 

rescission of funding.

4.5.2 Key Issues Identified

Several potential evaluation issues were identified during the course of the evaluation.

Key differences in funding eligibility parameters during the different rounds of funding 

created challenges in comparing initiatives across the CBJTG Program. The first round 

only allowed community and technical colleges as applicants, while each of the later 

rounds expanded the applicant pool to include workforce investment system 

organizations, such as workforce investment boards, local workshop agencies, and OneStop Career Centers. Because the first two rounds had different reporting requirements 

than the last two rounds, data are more representative of rounds three and four. As a 
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

September 2015 | 39

result, several survey respondents from the first two rounds did not submit a 

performance report or final report with their survey. Rounds one and two also ended 

several years prior to the evaluation, and a number of workers involved with these grants 

had likely moved on to different jobs.

Furthermore, some programs from the later rounds were still ongoing at the time of 

evaluation, so that real employment numbers were ultimately likely to be higher than 

those reported. The various CBJTG-funded initiatives also differ greatly in the number of 

participants trained - from as few as 37, to several thousand - leading to potential outlier 

issues.

Lastly, it is also important to frame the initiatives in a historical perspective. Many 

programs began during a time of relative economic strength, and continued through or 

ended during a bust period.

4.5.3 Key Performance Measures

The evaluation used a variety of measurements to gauge the extent to which initiatives 

achieved their targeted outcomes, and what challenges they had in doing so. The four 

categories of evaluative results were: performance, accomplishments, implementation 

issues, and sustainability of programming.

Performance: These measurements were primarily quantitative, and include the number 

of programs receiving funding in a given industry, the number of participants enrolled, the 

rate of participants who graduated/completed training, the number of participants who 

found employment, and the number of participants who found employment within their 

industry of focus. Data was also requested to determine the program’s ability to leverage 

resources, both cash and in-kind, and to establish the capacity-building potential of these 

programs. Respondents were also asked to rate how important these individual

measures were to their specific initiative.

Accomplishments: Through an open-ended question, respondents were asked to 

identify what they believed were the three greatest accomplishments of their program, 

and to provide their rationale.

Implementation Issues: Respondents were also asked via an open-ended question to 

indicate what presented the biggest challenges to the implementation of their program.

Sustainability of Programming: These measures analyzed the actual, logistical 

experience for programs that were no longer operational, as well as the plans to sustain 

activities in the future by those programs that were currently operational. These 

measures were both quantitative and qualitative, and included financial resources and 

expected levels of effort needed to sustain individual programs beyond their periods of 

performance.

4.5.4 Analytical Approach

Due to the wide range of foci and approaches across CBJTG programs, a variety of 

qualitative and quantitative factors were needed to properly measure the success of the 

Community-Based Job Grant Training program.
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

40 | September 2015

Surveys: Surveys were sent to all grant initiatives in the first four rounds of the CBJTG

Program. Of the 279 surveys sent out, 220 were answered, for a response rate of 78.9 

percent. Rounds three and four had a higher response rate (88 percent each) than 

rounds one and two (61 and 78 percent, respectively).

Grant documents:

1. Grant applications

2. Narrative quarterly reports

3. Financial reports

Case studies: Site visits to eleven grant initiatives at eight grant locations were 

conducted. These sites were selected to ensure variation along several criteria including 

industry, type of organization, round of grant, variation in target population, plans for 

partnerships, and geographic region.
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

September 2015 | 41

5 Performance Measurement

5.1 Data Collection

A critical element of the Job Creation and Retention Program evaluation is the collection 

of data to supplement the existing grant information provided in ARC.net for projects 

funded between fiscal years (FY) 2004 and 2010. The combined ARC.net and survey 

database (i.e., Performance Measures Database) underlie the data analysis conducted 

as part of this overall program evaluation.

In order to best measure the success of ARC’s job creation and retention grants in 

meeting the Commission’s goals and objectives, the project team developed a survey 

instrument to supplement the information provided by the grantees. While data is 

collected directly from the grant recipients by ARC at the time of project funding, 

comparing this information to project outcomes and outputs after this funding has been 

awarded and the project has closed is a critical step in the overall evaluation process.

5.1.1 ARC.net Database

The starting point for the job creation and retention program evaluation was the ARC.net 

database of all ARC funded projects from FY 2004 to FY 2010. This database contains 

information from the initial grant application, including project identification number, 

amount of ARC funding, location, project type, and anticipated outcomes and outputs for 

all projects, including the 286 job creation and retention projects. Much of the data 

included in ARC.net is provided by the grant recipient.

The projects considered in this job creation and retention evaluation cover a wide variety 

of categories and project types. As defined in ARC.net, some project types consisted of a 

relatively small number of projects. In order to best enable a meaningful analysis, the 

evaluation team collaborated with ARC to aggregate some of the more similar project 

types based on the project description provided in ARC.net. This re-categorization and 

aggregation allowed for larger sample sizes than provided by the initial categorization. 

The aggregated categories for project types are:

• Business/Training: Any project that creates a business or incubator for an 

industry not listed in other categories. These projects may also try to train 

employees or adults who are not considered students.

• Child Care/Education: Any project looking to build or expand a school, 

design a new class or curriculum, set up a day care, or benefit children or 

students in some other way.

• Community Building/Tourism: Any project that looks to build, renovate or 

market tourist attractions, or repair local infrastructure to support the 

community’s goals; for example, building recreational trails or street-scaping.

• Energy/Environmental: Any project that seeks to build a power plant, 

improve energy efficiency, or protect the environment.

• Local Food: Any project that tries to start or develop a farm, or a business, 

incubator, etc. in the fields of agriculture or culinary arts.
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

42 | September 2015

• Planning/Research: Any project that aims to develop a strategic plan, 

possibly associated with one of the other categories, or conduct a form of 

research.

After aggregating the projects, the project team updated the grantee contact information 

to optimize data collection. The resulting database incorporated the updated contact 

information to the existing information initially collected. The resulting database was as 

up-to-date as achievable in the timeframe, prior to the program evaluation data analysis.

5.1.2 Survey 

Once the ARC.net database was updated, a survey was designed to collect information 

that would further inform the program evaluation of ARC’s job creation and retention 

efforts. The project team agreed that the survey would include general questions seeking 

information on project outputs and outcomes, but also specific questions based on 

project type. The project-type questions sought more qualitative detail regarding the 

beneficiaries, goals, and impacts of the project. To ensure as complete a response as 

possible, delivery of the survey would be offered in a number of different ways. Details 

related to survey design and implementation are presented below.

Design and Testing

A draft survey instrument was constructed by the project team. The survey asked 

respondents to categorize their project type, how the project was carried out, and whom 

or what benefited the most from the projects. Quantitative questions regarding a variety 

of measures, particularly jobs created and retained, were included, along with qualitative 

questions asking respondents to rate the long-term impact of their project on a number of 

community well-being measures, from economic strength to environmental sustainability. 

The survey sought to elicit a range of responses for all projects, as well as some 

responses pertaining to specific project types. A copy of the survey instrument is 

provided in Appendix 1 of this report.

Prior to sending the survey to the full list of grantees, a “test run” was administered to 

seek feedback regarding the questions. Before the “test run,” the project team’s internal 

statistician reviewed the survey instrument and revisions were made based on her 

review. The final survey was then provided to some of the program evaluation case study 

candidates for review and feedback. The feedback received was incorporated into the 

survey, and edits were made as necessary before the survey was delivered to the full set 

of grantees.

Survey Implementation

Once the survey was tested and finalized, it was sent to all grantees that received job 

creation and retention project funding from ARC between FY 2004 and FY 2010. Based 

on the information in the database and in an attempt to be consistent with other program 

evaluations, such as those for education and infrastructure, the team distributed the 

survey electronically, via e-mail, to all grantees.

The electronic correspondence included: information about the survey purpose and 

background; how to access the survey on-line via Survey Monkey; information regarding 

the relevant project number, project title, year of funding, and project type; and a request 
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

September 2015 | 43

for participation. In this correspondence, respondents were also given the option of filling 

out the attached survey and returning by mail. For those e-mails that were undeliverable, 

or recipients that did not provide a response, strategic follow-up efforts were made. The 

follow-up process entailed:

1. Initial follow-up telephone call

2. Initial follow-up email

3. Second follow-up email from ARC’s program director

4. Final telephone call

Efforts to pursue responses were carried out for five consecutive months, and were 

ended after the final round of calls. Follow-up efforts had a substantial impact on 

response rates; 31 responses were received initially and 110 additional responses were 

received after the follow-up process was complete. The majority of responses were 

received via the Survey Monkey data collector. In some instances, respondents emailed 

their responses directly to the survey administrator, who then input the information into 

the Survey Monkey database.

Of the 286 job creation and retention program grantees, the team received 123

responses with sufficient information to analyze, based on a review for erroneous, blank,

or duplicated responses. There were 138 survey responses in total, but 15 were either 

blank or duplicate, and thus not counted.

5.2 Performance Measure Database

Prior to conducting the data analysis for the program evaluation, the project team 

appended the information collected from the survey to the original data provided in the 

ARC.net database at project onset. This combined performance measure database was 

used to evaluate the success of the projects in achieving ARC’s goals and objectives 

related to job creation and retention, as well as other qualitative factors. This section 

provides detail related to the combination of the ARC.net and survey databases to 

develop the full performance measure database.

5.2.1 Construction of Survey Database

Survey responses were collected through Survey Monkey and exported to Microsoft 

Excel as a first step in developing the survey database. A review of the data was then 

conducted. Several challenges were encountered during the data review.

Some respondents left answers blank or did not know the outcome or benefits of the 

project. Clarification was sought with individual respondents when deemed necessary by 

the survey administrator. This process enabled the administrator to aggregate duplicate 

responses and ascertain a firm response where vague or indirect responses were initially 

provided. For example, some respondents indicated a number of part-time jobs, but it 

was not clear whether a part-time job was half of a full-time job. Other explanations were 

also required to ensure that the information collected was as accurate as possible. To 

clarify, the administrator contacted the respondents and adjusted the initial survey 

response to reflect the guidance provided.
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

44 | September 2015

Project titles and project numbers as provided by respondents were checked against the 

ARC.net database, and occasionally amended to ensure accuracy and conformity. 

Without this step, it would not have been possible to append the survey data to the 

original ARC.net data, which was critical for the analysis. Because some survey 

questions were specific to a given project type, and because respondents did not always 

categorize their projects the same way that they were categorized by the evaluation 

team, counts and sums by project-type are based on the evaluation team’s project type

definitions, unless otherwise noted.

5.2.2 Performance Measure Database Development

After cleaning the response data, this information was then appended to the ARC.net 

database in order to show the survey responses as a subset of the total database of 

projects. PivotTables were used to ensure the accuracy of counts and sums for all 

relevant question responses, including project types, jobs created, jobs retained, and so 

forth. These tables were then checked against the Survey Monkey response data for 

conformity.

5.3 Analysis and Summary of Project Performance 

Measure Database

The project team examined a wide range of performance measures in order to assist 

ARC staff, managers, and grant applicants in the process of evaluating the benefits of 

the program as clearly and transparently as possible.

The analysis focuses on three key areas:

• How is the program funding spent? These measures describe how ARC 

has made investments across several dimensions, including by state, by 

project type, and by year.

• What are the real impacts of the program? These measures evaluate 

economic development impacts, such as jobs created/retained, businesses 

created/retained, households/businesses improved, communities served, and 

so forth.

• How do recipients perceive program benefits? These measures describe 

how grant recipients perceive the long-range impacts of the projects on the 

economy, the environment, and the quality of life in the communities.

The 286 job creation and retention projects represent roughly $42.5 million in ARC 

investment between FY 2004 and FY 2010. Based on the survey responses, a total of 

123 projects at least partially funded by ARC during this time period were evaluated in 

detail by the project team. Projects for which survey responses were received represent 

$20.7 million of ARC funding from FY 2004 to FY 2010. It is important to note that this 

information and the following detailed analysis is strictly based on the responses to the 

survey, and as highlighted in the previous section, the characteristics of the projects for 

which surveys were completed may vary from the characteristics of the overall 

investments funded during the relevant time period by ARC.
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

September 2015 | 45

5.3.1 Characteristics by State, All Funded Projects

For consistency with ARC reporting measures, project states are classified by the state 

indicated in their ARC Project number, as reported in the ARC.net database. Thus, “CO” 

denotes “Commission” projects rather than the traditional state abbreviation for Colorado. 

The Commission projects are labeled as such because they are funded directly with 

Commission funds rather than the specific state’s allocations. These projects sometimes, 

though not always, cover multiple states.

Number of ARC Funded Projects by State

The ARC.net database included 286 job creation and retention projects funded during 

the FY 2004 to FY 2010 period. As shown in Figure 1 and Table 3 below, Pennsylvania, 

with 78 projects (27.3 percent), received the largest number of grants overall, followed by 

New York, and Commission projects with 32 (11.2 percent) each. Pennsylvania's 

particularly high number of grants is largely due to the Enterprise Development 

Programs. These long-running programs were renewed on an annual basis throughout 

the duration of the FY 2004 to FY 2010 study period, and account for 50 (50.6 percent) 

of Pennsylvania's job creation and retention projects in the ARC.net database; greater 

detail on Enterprise Development Programs is provided later in the report.

Figure 1: Total ARC Job Creation and Retention Projects by State, Full Database

Source: ARC.net database

Survey responses were received from every state in the Appalachian Region, as well as 

Commission projects. North Carolina and Pennsylvania accounted for the highest 

response rates with 73.3 percent and 52.6 percent of all grantees in their state, 

respectively. The total number of grants awarded by state, based on ARC.net, and the 

total number of surveys received are shown below. The overall response rate was 43 

percent. The distribution of responses by state compared to grants provided by state is 

relatively consistent, though South Carolina and Georgia are slightly under-represented 

16

32

23

12

18

4

15

32

7

78

7 6

27

9

0

10

20

30

40

50

60

70

80

90

AL CO GA KY MD MS NC NY OH PA SC TN VA WV
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

46 | September 2015

in the responses and North Carolina and Pennsylvania are slightly over-represented. 

These differences are not expected to materially impact the results of the analysis.

Table 3: Number and Share of Grants & Survey Responses Received by State

State # Total Database 

Projects

% Total 

Projects

# Survey 

Responses

% Survey 

Responses

Response Rate 

Relative to State Share

AL 16 5.6% 6 4.9% 37.5%

CO 32 11.2% 13 10.6% 40.6%

GA 23 8.0% 6 4.9% 26.1%

KY 12 4.2% 6 4.9% 50.0%

MD 18 6.3% 8 6.5% 44.4%

MS 4 1.4% 2 1.6% 50.0%

NC 15 5.2% 11 8.9% 73.3%

NY 32 11.2% 10 8.1% 31.3%

OH 7 2.4% 2 1.6% 28.6%

PA 78 27.3% 41 33.3% 52.6%

SC 7 2.4% 1 0.8% 14.3%

TN 6 2.1% 3 2.4% 50.0%

VA 27 9.4% 11 8.9% 40.7%

WV 9 3.1% 3 2.4% 33.3%

Total 286 100.0% 123 100.0% --

Source: ARC.net database and survey response data

Dollars of Investment by State

As shown in Table 4, Pennsylvania received the largest share of ARC funding – $20.4

million of the $42.5 million for job creation and retention programs – consistent with the 

large number of programs funded in the state. Enterprise Development Programs 

accounted for $17.8 million, 87 percent, of Pennsylvania’s funding during this time 

period. North Carolina, Georgia, Kentucky, and Virginia all received greater than $2 

million in ARC investment for job creation and retention between FY 2004 and FY 2010, 

while the remaining states received less than $2 million in funding during the same time 

frame. The survey responses accounted for nearly half of all funding during the time 

period. Of all states, the relative shares of survey responses for South Carolina, Ohio 

and Tennessee were proportionally lower than the relative shares of projects in the 

ARC.net database.
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

September 2015 | 47

Table 4: ARC Investment by State, in Millions of Dollars15

State Total Database Projects % Total Projects Survey Responses % Survey Responses

AL $1.4 3.3% $0.5 2.4%

CO $1.7 3.9% $0.6 3.1%

GA $2.8 6.6% $0.9 4.3%

KY $2.2 5.1% $1.1 5.3%

MD $1.4 3.3% $0.8 3.7%

MS $0.5 1.1% $0.3 1.4%

NC $3.1 7.3% $1.7 8.1%

NY $2.0 4.7% $0.9 4.2%

OH $0.8 1.8% $0.1 0.4%

PA $20.4 47.9% $11.9 57.4%

SC $1.0 2.4% $0.1 0.5%

TN $0.9 2.0% $0.2 0.9%

VA $2.8 6.5% $0.8 3.8%

WV $1.7 4.1% $1.0 4.6%

Total $42.5 100.0% $20.7 100.0%

Source: ARC.net database and survey response data

5.3.2 Characteristics by Project Type

Number of Projects by Project Type

One of the most important elements of the program evaluation is gaining an 

understanding of the outcomes and outputs for particular types of job creation and 

retention projects. The performance measure database (“Total Projects” in the tables 

below) developed for the program evaluation includes projects distributed among all 13 

Appalachian states (plus the Commission projects), as well as a wide range of job 

creation and retention programs, including business or training programs, local food 

projects, and energy sustainability projects, among others.

Survey respondents were provided with the project type as categorized by the team, but 

they had the opportunity to identify what they believed to be their project type; some 

differences arose between respondents’ self-reported project types, and those project 

types categorized by the project team.

The largest number of projects funded fall within the business/training project category, 

with a total of 144 of the 286 total database projects. Of the 123 projects analyzed (those 

with survey responses), 70 projects, or 56.9 percent, are categorized as 

Business/Training, as shown in Table 5. Community Building/Tourism-focused projects 

received the second greatest number of grants, a total of 50, and also accounted for the 

second largest number of survey responses, 17. The table below shows the distribution 

of all 286 projects, as well as the 123 survey responses, by database project type.

 

15 Please note that totals may not add due to rounding.
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

48 | September 2015

Table 5: Job Creation and Retention Project Distribution by Database Type

Project Type Total Database 

Projects

% Total 

Projects

Survey 

Responses

% Survey 

Responses

Business/Training 144 50.3% 70 56.9%

Child Care/Education 16 5.6% 9 7.3%

Community Building/Tourism 50 17.5% 17 13.8%

Energy/Environmental 18 6.3% 10 8.1%

Local Food 26 9.1% 7 5.7%

Planning/Research 32 11.2% 10 8.1%

Total 286 100.0% 123 100.0%

Source: ARC.net database and survey response data

When classified based on how respondents categorized their projects, there were some 

distributional differences, but Business/Training and Community Building/Tourism 

projects still represented the largest share of survey responses for the defined 

categories. Much of the difference between the two project type categorizations is 

attributable to the “other” category, which was presented as an option for respondents in 

the survey. Most of the respondents who chose “other” when asked how to categorize 

their project were Enterprise Development projects. In fact, 16 of the 37 “other” projects 

fell into this category. Enterprise Development projects are unique and likely difficult to 

categorize into a rigid project type. This may have led respondents to categorize their 

projects as “other” rather than one of the more specific project type categories.

As illustrated in Table 6, of the 123 projects for which survey data was received, 29.3 

percent of surveys received were categorized as Business/Training projects by 

respondents. Community Building/Tourism comprised 17.1 percent of all survey 

responses. The largest category of responses fell into the “other” category, when 

respondents self-reported their project type.
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

September 2015 | 49

Table 6: Job Creation and Retention Project Distribution by Self-Reported Type

Project Type Total Database 

Projects

% Total 

Projects

By Survey 

Respondent Type

% Survey

Responses

Business/Training 144 50.3% 36 29.3%

Child Care/Education 16 5.6% 8 6.5%

Community Building/Tourism 50 17.5% 21 17.1%

Energy/Environmental 18 6.3% 11 8.9%

Local Food 26 9.1% 4 3.3%

Planning/Research 32 11.2% 6 4.9%

Other 0 0.0% 37 30.1%

Total 286 100.0% 123 100.0%

Source: ARC.net database and survey response data

For purposes of consistency, all data analysis presented in this program evaluation 

is based on the project team’s more aggregate project types, rather than those 

given by the survey respondents or provided in the ARC.net database, except 

where specifically noted. The analysis could have been conducted using either project 

type designation, but the large number of self-reported “other” projects, and project team 

concerns that the respondents for the Enterprise Development projects may have been 

uncertain about how best to categorize their projects, led the team to default to its project 

type definitions for the analysis. The project team designations were based on grant 

descriptions provided in ARC.net. As such, the team feels comfortable with its project 

type assignments, including Enterprise Development projects.16 In addition, using the 

project team project type designations increases the sample size for each of the more 

specific project type categories (e.g., Local Food, Business/Training, etc.).

Dollars of Investment by Project Type

As shown in Table 7, the survey responses accounted for $20.7 million of the just under 

$42.5 million that ARC invested in job creation and retention programs during the time 

period. A vast majority of this funding was invested in Business/Training programs. 

Receiving just short of $28 million in ARC funding, Business/Training projects received 

65.8 percent of total ARC job creation and retention funding. Community 

Building/Tourism projects received 14.4 percent of ARC funding, while Child 

Care/Education, Energy/Environmental, Local Food, and Planning/Research projects 

each received between three to seven percent of ARC funding.

The distribution of funding among survey responses generally mirrored that of the full set 

of projects funded during the time period. Business/Training projects are slightly overrepresented while Community Building/Tourism and Local Food projects are slightly 

under-represented. The slight difference in distribution does not meaningfully impact the 

overall analysis results.

 

16 Note that Enterprise Development projects are considered separately as a subset of Business/Training projects for 

purposes of discussion. Inclusion of their results in the category specific breakdowns greatly skews outputs, thus 

they are separated.
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

50 | September 2015

Table 7: Job Creation and Retention Project Distribution by Dollars Invested, in 

Millions

Project Type Total Database 

Projects

% Total 

Projects

Survey 

Responses

% Survey 

Responses

Business/Training $28.0 65.8% $15.6 75.3%

Child Care/Education $2.8 6.6% $1.3 6.2%

Community Building/Tourism $6.1 14.4% $1.9 9.4%

Energy/Environmental $1.5 3.5% $0.8 3.7%

Local Food $2.6 6.0% $0.6 3.0%

Planning/Research $1.6 3.8% $0.5 2.4%

Total $42.5 100.0% $20.7 100.0%

Source: ARC.net database and survey response data

Overall Economic Development Impacts

Another important aspect of the evaluation is examining real, tangible economic 

development impacts, such as jobs created or retained, and businesses created or 

retained by state and project type. The limited set of completed surveys leaves some 

investment categories and states without sufficient impact data to adequately analyze the 

effects of ARC projects. In addition, it is important to note that not all of the respondents 

to the survey tracked or provided answers to every measure.

Based on the survey responses for which sufficient data were available, ARC job 

creation and retention investments created and/or retained a total of more than 237,000 

jobs. This total is primarily driven by a few projects in a small number of states. 

Receiving almost six times the next greatest amount of ARC funding during the study 

period, Pennsylvania created and retained the most number of jobs by far, with over 

55,000 jobs created and over 178,000 retained. Of these, Enterprise Development 

Programs contributed to approximately 51,000 jobs created and 176,000 jobs retained.17

Examining only the 123 projects for which survey responses were received, projects that 

received funding from ARC between FY 2004 and FY 2010 contributed to the creation of 

776 business establishments, and the retention of 3,427 businesses, as shown in Table 

8 below. ARC investments contributed to the largest number in Pennsylvania, with a total 

of 555 new businesses established, and nearly 3,100 retained; of these, Enterprise 

Development Programs contributed to the creation of 430 businesses and the retention 

of 2,884 businesses.

 

17 It should be noted that these values were primarily provided by the Southwestern PA Enterprise Development 

Program. At this time, the team is attempting to seek detail regarding the nature of these responses and how they 

were generated.
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

September 2015 | 51

Table 8: Key Project Outcomes, All Project Types

Project Type Jobs 

Created

Jobs 

Retained

Businesses 

Created

Businesses 

Retained

Leveraged 

Private 

Investment 

($ millions)

Business/Training 58,158 177,688 668 3,093 $362.3

Child Care/Education 48 43 25 48 $2.4

Community Building/Tourism 193 561 14 6 $30.8

Energy/Environmental 348 39 23 5 $9.1

Local Food 169 548 32 271 $0.7

Planning/Research 111 19 14 4 $17.5

Total 59,026 178,897 776 3,427 $422.7

Source: Survey response data

The business establishment outcomes were primarily the result of Business/Training 

programs, accounting for 668 of 776 newly established businesses.

In addition to impacts on jobs and businesses, ARC investments also contributed to 

improvements to households. Based on survey responses, ARC job creation and 

retention investments improved 12,019 households. Nearly seventy-five of these 

households were the result of the Beaver Falls Downtown Revitalization Project, which 

sought to remake Main Street in rural Beaver Falls, Pennsylvania. Based on survey 

responses, ARC job creation and retention investments also resulted in 4,946 unique 

businesses served.

ARC contributed nearly $43 million in funding to job creation and retention projects from 

FY 2004 to FY 2010. According to survey respondents, these projects leveraged an 

additional $422.7 million in private investments. The survey did not collect any detailed 

information on the components, sources or uses of the leveraged private investment.

Findings by Project Type

Given the wide variety of projects considered in this evaluation, additional detail of the 

results by project type is provided below.

Business/Training

This category of projects represents any project that creates a business or incubator for 

an industry not listed in other categories. These projects may also train employees or 

adults who are not considered students. Though all Enterprise Development projects 

were categorized as Business/Training, these projects are excluded below and analyzed 

separately due to their large reported impacts, creating a distortion in the overall results.

Of the 286 total job creation and retention projects in the ARC.net database, 144 were 

categorized as “Business/Training” projects based on the project team’s classification of 

the projects, including 50 Enterprise Development projects. Of the 123 total survey 

responses, 40 responses (32.5 percent) were for non-Enterprise Development 

Business/Training projects.
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

52 | September 2015

Based on survey responses, non-Enterprise Development Business/Training projects 

received $10.2 million in ARC funding. These projects created 2,688 new jobs, retained 

1,198 jobs, created 238 new businesses, and retained 209 existing businesses. Figures 

2 and 3 illustrate the findings by state.

Figure 2: Business/Training Projects, Jobs Created and Retained (n=40)

Source: Survey results database

Figure 3: Business/Training Projects, Businesses Created and Retained (n=40)

Source: Survey results database

Additionally, non-Enterprise Development Business/Training projects served nearly 

1,100 businesses and leveraged roughly $156.4 million in private investment (see Figure 

4). Compared to other project type categories, Business/Training received the greatest 
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

September 2015 | 53

number of grants and the greatest amount of funding. They also had the most notable 

economic impacts among the different project types.

Figure 4: Business/Training Projects, Leveraged Private Investment, $ millions 

(n=40)

Source: Survey response data

In addition to the quantitative impacts associated with the business/training projects, 

survey respondents also provided feedback regarding additional activities undertaken by 

the project that may not be captured in the output and outcome measures. Unlike the 

findings presented above, which were based on the project team’s definition of a project 

type, these findings are based on the survey respondent’s classification of their project, 

as these questions were specific to the nature of the grant. The Business/Training 

projects exclude the Enterprise Development Programs, where self-identified, for the 

purposes of this qualitative results presentation.

In the area of education and training, 14 of the 22 survey respondents indicated that their 

project provided instruction in business management. The business and training projects 

were particularly beneficial in the area of community outreach, establishing community or 

business partnerships as a component of 18 projects, and providing community outreach 

activities as a component of nine of the projects. Nine projects increased community 

capacity by aiding in implementation of outcome-based community or economic 

development activity. These projects were also particularly beneficial to entrepreneurs, 

with 15 of the projects working with entrepreneurs to develop business plans, and 21 

projects providing entrepreneurship education at some level between school age and 

adults. Technical assistance also played a vital role, with 13 projects assisting 

businesses in developing marketing strategies and eight providing expertise in the areas 

of finance or accounting.

ENTERPRISE DEVELOPMENT PROGRAMS

These long-running programs were renewed on an annual basis throughout the duration 

of the FY 2004 to FY 2010 study period. All Enterprise Development Programs were 

$18.6

$110.2

$0.2

$0.1 $8.1

$12.8

$4.8 $0.3 $1.3

AL

CO

GA

KY

MD

NC

PA

SC

TN
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

54 | September 2015

categorized as Business/Training projects. There were seven unique Enterprise 

Development Programs, all based in Pennsylvania, that account for 50 (60.2 percent) of 

Pennsylvania's grant projects in the ARC.net database.

The number of Enterprise Development Program grants is greater than the number of 

grants for all project type categories other than Business/Training (of which they are a 

subset) and Community Building/Tourism. These programs had a substantial outlier 

impact on our survey results, representing a bulk of ARC funding, and tracked outcomes 

(jobs created, jobs retained and so forth). By themselves, Enterprise Development 

Programs created roughly 51,000 jobs and retained another 176,000 jobs. The vast 

majority of these jobs were associated with the Southwest Pennsylvania Enterprise 

Development Program.

Child Care/Education

This category of projects represent any project looking to build or expand a school, 

design a new class or curriculum, set up a day care, or benefit children or students in 

some other way. These projects were not included in a previous ARC evaluation of 

education grant programs, as they tend to reflect supplemental education rather than 

traditional schooling.

Of the 286 total job creation and retention projects in the ARC.net database, 16 were 

categorized as “Child Care/Education” projects. Of the 123 total survey responses, 9

responses (7.3 percent) were for Child Care/Education projects. Figures 5 and 6 

illustrate the findings.

Those that responded to the survey accounted for $1.3 million of the $2.8 million in 

funding for Child Care/Education projects. These projects created 48 new jobs, retained 

43 jobs, created 25 new businesses, and retained 48 existing businesses.

Figure 5: Child Care/Education Projects, Jobs Created and Retained (n=9)

Source: Survey response data
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

September 2015 | 55

Figure 6: Child Care/Education Projects, Businesses Created and Retained (n=9)

Source: Survey response data

In addition, Child Care/Education projects served 198 businesses, and leveraged over 

$2.4 million in private investment, as presented in Figure 7. Child Care/Education 

accounts for less than six percent of the total number of job creation and retention 

grants, and the results reported are consistent with the extent of the grants provided for 

this project category.

Figure 7: Child Care/Education Projects, Leveraged Private Investment, $ millions 

(n=9)

Source: Survey response data

$0.3

$0.1

$2.0

GA

NY

VA
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

56 | September 2015

As with all of the other project types, specific questions were asked of the grant 

recipients regarding additional beneficiaries or activities undertaken by their project. The 

nine respondents indicated that their projects were primarily focused on the purchase or 

development of educational materials and curriculum or manuals. They also developed 

educational programs and provided financial assistance.

Community Building/Tourism

This category of project represents any project that looks to build, renovate or market 

tourist attractions, or repair local infrastructure. Of the 286 total job creation and retention 

projects in the ARC.net database, 50 were categorized as “Community Building/Tourism” 

projects. Of the 123 total survey responses, 17 responses (13.8 percent) were for 

Community Building/Tourism projects.

The survey responses accounted for $1.9 million of the $6.1 million in funding for

Community Building/Tourism projects. These projects created 193 new jobs, retained 

561 jobs, created 14 new businesses, and retained 6 existing businesses (Figures 8 and 

9). Many of the jobs created and retained were associated with the Upper Potomac 

Industrial Park Flood Protection program in Maryland.

Figure 8: Community Building/Tourism Projects, Jobs Created and Retained 

(n=17)

Source: Survey response data
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

September 2015 | 57

Figure 9: Community Building/Tourism Projects, Businesses Created and 

Retained (n=17)

Source: Survey response data

Additionally, Community Building/Tourism projects served 163 businesses, and 

leveraged roughly $30.8 million in private investment, as shown in Figure 10. Community 

Building/Tourism projects received the second greatest amount of ARC funding for job 

creation and retention programs, and ranked second-or-third highest outcomes for all of 

the measures provided in the above paragraph, except for businesses retained.

Figure 10: Community Building/Tourism Projects, Leveraged Private Investment, 

$ millions (n=17)

Source: Survey response data

$27.0

$3.8

GA

MD
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

58 | September 2015

Of the 21 Community Building/Tourism projects, as classified by survey respondents, 

seven supported heritage, craft, or music trails and six provided planning for, design, or 

construction of new attractions. Four of them provided new community facilities through 

the construction of a community center, day care, or theater.

Energy/Environmental

This category of project represents any project that seeks to build a power plant, improve 

energy efficiency, or protect the environment. Of the 286 total job creation and retention 

projects in the ARC.net database, 18 were categorized as “Energy/Environmental” 

projects. Of the 123 total survey responses, 10 responses (8.1 percent) were for 

Energy/Environmental projects.

Survey respondents for Energy/Environmental projects received roughly $757,000 of the 

$1.5 million in ARC funding spent on this project type. As shown in Figures 11 and 12, 

these projects created 348 new jobs, retained 39 jobs, created 23 new businesses, and 

retained five existing businesses. Much of the job creation is attributed to the Building the 

Clean Energy Economy in Western North Carolina project.

Figure 11: Energy/Environmental Projects, Jobs Created and Retained (n=10)

Source: Survey response data
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

September 2015 | 59

Figure 12: Energy/Environmental Projects, Businesses Created and Retained 

(n=10)

Source: Survey response data

Additionally, Energy/Environmental projects served over 375 businesses, and leveraged 

roughly $9.1 million in private investment (Figure 13). Energy/Environmental projects 

received the least amount of total funding for job creation and retention investment.

Figure 13: Energy/Environmental Projects, Leveraged Private Investment, $ 

millions (n=10)

Source: Survey response data

In general, the 11 respondents that classified themselves as energy or environmental did 

not identify with any of the qualitative characteristics presented to them. That said, one 

did indicate a community revitalization benefit of the hiring of a marketing coordinator.

$6.7

$2.3

CO

NC
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

60 | September 2015

Local Food

This category of projects represents any project that seeks to start or develop a farm, or 

a business, incubator, etc. in the field of agriculture or culinary arts. Of the 286 total job 

creation and retention projects in the ARC.net database, 26 were categorized as “Local 

Food” projects. Of the 123 total survey responses, seven responses (5.7 percent) were 

for Local Food projects.

Survey responses for Local Food projects received roughly $627 thousand of the $2.6 

million in ARC funding. These projects created 169 new jobs, retained 548 jobs, created 

32 new businesses, and retained 271 existing businesses, as presented in Figures 14 

and 15 below.

Figure 14: Local Food Projects, Jobs Created and Retained (n=7)

Source: Survey response data
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

September 2015 | 61

Figure 15: Local Food Projects, Businesses Created and Retained (n=7)

Source: Survey response data

Additionally, Local Food projects served over 150 businesses, and leveraged roughly 

$654,000 in private investment (Figure 16).

Figure 16: Local Food Projects, Leveraged Private Investment, $ millions (n=7)

Source: Survey response data

The four self-identified local food projects indicated additional project benefits in the area 

of technical assistance. Two of the respondents indicated that their grant allowed them to 

assist businesses in developing marketing strategies, and these two also indicated that 

the grant allowed them to work with potential entrepreneurs to develop a business plan.

$0.10

$0.07

$0.46

$0.02

NC

OH

PA

VA
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

62 | September 2015

Planning/Research

This category of projects represents any project that aims to develop a strategic plan, 

possibly associated with one of the other categories, or conduct a form of research. Of 

the 286 total job creation and retention projects in the ARC.net database, 32 were 

categorized as “Planning/Research” projects. Of the 123 total survey responses, 10

responses (8.1 percent) were for Planning/Research projects.

The survey responses for Planning/Research projects accounted for nearly $500

thousand of the $1.6 million in ARC funding allocated to this project type. These projects 

created 111 new jobs, retained 19 jobs, created 14 new businesses, and retained four

existing businesses. Figures 17 and 18 present the results of the analysis for this project 

type.

Figure 17: Planning/Research Projects, Jobs Created and Retained (n=10)

Source: Survey response data
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

September 2015 | 63

Figure 18: Planning/Research Projects, Businesses Created and Retained (n=10)

Source: Survey response data

Additionally, Planning/Research projects served over 370 businesses, and leveraged 

roughly $17.5 million in private investment. The shares by state are provided in Figure 19 

below.

Figure 19: Planning/Research Projects, Leveraged Private Investment, $ millions 

(n=10)

Source: Survey response data

Planning/Research projects accounted for six of the self-identified responses. Two of 

these respondents indicated that their grant allowed for the funding of an economic 

impact study, one allowed for the improvement of education practices, and one allowed 

for an understanding of environmental impacts.

$0.1

$0.8

$4.8

$11.8

KY

NC

PA

VA
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

64 | September 2015

5.3.3 Anticipated vs. Actual Impacts

When applying for ARC funding, the prospective grantees are asked to estimate the 

impacts of their projects. An additional assessment of the success of projects is to

examine the actual outcomes compared to the perceived outcomes. For purposes of this 

analysis, Enterprise Development Program projects were excluded from the anticipated 

and actual outcome comparisons. These projects tended to skew the impacts, as the 

information presented was typically modeled from an economic impact model rather than 

true observations. Where possible, this comparison was made for the 93 non-Enterprise 

Development projects that had survey responses. Some respondents did not track all of 

the measures, or left their response for a particular measure blank. The total number of 

projects with sufficient information for analysis on each measure is indicated in the 

rightmost column of Table 9. On the aggregate, all measures that were both predicted 

and reported by survey respondents exceeded the initial projections.

As seen in Table 9, respondents anticipated serving only 19 businesses and actually 

served more than 2,300; respondents also more than doubled the predicted number of 

businesses created. Job creation exceeded expectations, with 3,556 jobs created 

compared to 2,715 projected. Job retention more than doubled the predictions, with over 

2,400 jobs actually retained compared to the just over 1,000 predicted. Lastly, the 

projects leveraged a substantial amount of private investment, with actual private 

investment leveraged at $216.8 million, as compared to $37.3 million predicted. More 

information would be required to determine the exact explanation for why actual 

outcomes were greater than predicted outcomes, but some possible explanations include 

a greater tendency for successful projects to report results, greater impacts due to a 

combination of ARC grants with additional outside investments or a misunderstanding of 

the question and possible double-counting.

Table 9: Predicted vs. Actual Outputs & Outcomes, Excluding Enterprise 

Development 

Predicted Actual Difference # Projects 

Considered

Businesses Served 19 2,352 2,333 44

Businesses Created 147 346 199 47

Jobs Created 2,715 3,556 841 60

Jobs Retained 1,051 2,408 1,357 50

Leveraged Private Investment ($ millions) $37.3 $216.8 $179.5 51

Source: Survey response data

5.3.4 Perceived Qualitative Project Outcomes

In addition to questions seeking quantitative data, the evaluation survey also contained 

questions to qualitatively assess the long-range project outcomes. These measures 

describe how grant recipients perceive the long-range impacts of the projects on the 

economy, the environment, and the quality of life of the communities. Grantees were 

asked a series of questions related to the project and asked to respond according to the 

following scale:
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

September 2015 | 65

• None: Project had little to no impact on given measure

• Slight: Project impact was not large enough to reverse or stabilize measure’s 

trend

• Moderate: Project impact contributed to the stabilization or reversal of 

measure’s trend 

• High: Project impact was responsible for significant improvement in 

measure’s trend

Questions were asked regarding economic impacts, competitive impacts, environmental 

impacts, and quality of life impacts.

Ratings were sought to the following statements about long-term economic impacts of 

ARC projects:

1. Attracting new residents or stabilizing the area’s population

2. Attracting new jobs or increasing employment at existing businesses

3. Creating new sources of income or increasing income for local residents

4. Increasing local business sales or the value of business assets (such as equipment, 

real estate)

5. Increasing the value of household assets (such as homes, land, farms) for local 

residents

6. Increasing the value of community assets (such as community buildings, schools, 

infrastructure, parks)

Economic Measures

Analyzing the 123 survey responses, 97 respondents provided information for qualitative 

questions related to economic measures. Specifically, respondents believed that their 

projects had the greatest impact on attracting new jobs and increasing local business 

sales. Though the highest portion of “high” responses were for the increased value of 

community assets, only 30 of the respondents rated these projects as “high” or 

“moderate,” while 62 rated attracting new jobs and 55 rated increased local business 

sales as such. Projects were rated as having the least impact on attracting new 

residents. Given that the projects in this evaluation are focused on job creation and 

retention, these results are consistent with expectations. Results are depicted in Figure 

20 below.
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

66 | September 2015

Figure 20: Qualitative Survey Responses: Economic Measures (n=97)

Source: Survey response data

Competitiveness Measures

The second set of qualitative questions sought long-term outcomes related to the 

following competitiveness measures:

1. Improving the stability and sustainability of the local economy

2. Improving the efficiency of business operations or public services

3. Improving the productivity of students, employees, businesses, land, or other assets

4. Improving the skill level of the workforce

5. Increasing the viability of local businesses

6. Improving access to markets for local products, businesses, artisans, and 

entrepreneurs

7. Reducing the cost of doing business

An analysis of the survey responses shows that 97 respondents provided information 

related to these questions. Respondents believed that their projects had very significant 

positive impacts on stabilizing the local economy, improving the efficiency of business 

operations or public services, improving the viability of local businesses, and improving 

access to markets for local products, businesses, artisans, and entrepreneurs. 

Respondents indicated that their projects had minimal impact on reducing the cost of 

business. Figure 21 depicts the survey response results for competitiveness measures.

0

10

20

30

40

50

60

Attracted

new

residents

Attracted

new jobs

Created

new

sources of

income

 Increased

local

business

sales

Increased

value of

household

assets

Increased

value of

community

assets

None

Slight

Moderate

High
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

September 2015 | 67

Figure 21: Qualitative Survey Responses: Competitiveness Measures (n=97)18

Source: Survey response data

Environmental Measures

The third category of qualitative impacts related to environmental measures of the 

investments. Of the economic, competitiveness and environmental impacts, the 97 

grantees who provided input found their projects to be the least successful at improving 

environmental outcomes overall. The following are the topics posed as questions that 

were presented to the grantees:

1. Impact on improving air or water quality

2. Impact on improving quality and/or access to land and natural resources

3. Impact on waste reduction or improving waste management (reuse, recycling, 

alternative energy, biofuels)

4. Impact on improving energy security and independence

5. Impact on improving energy efficiency or conservation

The job creation and retention programs evaluated herein were not bound to serve 

environmental improvement. Their priority was to create and retain jobs, rather than 

improve the environment although, in some cases, this objective was met. Not 

surprisingly, impacts on environmental measures were typically reported as “slight” or 

“none.” The environmental measure with the greatest positive impact was improving 

energy efficiency or conservation. Environmental response results can be seen in Figure 

22.

 

18 Note that n=97 for all categories except “skill level of the workforce” and “access to markets for local products” 

where n=96.

0

5

10

15

20

25

30

35

40

45

50

Stability /

Sustainability

of local

economy

Efficiency of

business

operations or

public

services

Productivity of

assets

Skill level of

the workforce

Viability of

local

businesses

Access to

markets for

local products

Reduced cost

of business

None

Slight

Moderate

High
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

68 | September 2015

Figure 22: Qualitative Survey Responses: Environmental Measures (n=97)

Source: Survey response data

Quality of Life Measures

The final category of qualitative impacts related to the quality of life measures of the 

investments. The following are the topics posed as questions that were presented to the 

grantees:

1. Impact on improving access to culture, arts, or historic sites

2. Impact on improving civic life and governance

3. Impact on public health, safety, or wellbeing

4. Impact on improving public services and institutions

5. Impact on improving recreational opportunities

6. Impact on improving the diversity (age, gender, race, ethnicity, economic status) of 

the population

7. Impact on increasing knowledge and educational levels of the population

For these measures, 97 respondents provided information and reported that their 

projects had the greatest impact on increasing the educational levels of their population. 

Many projects had a focus on training workers, either through schooling or on-the-job 

training, so these results are consistent with expectations. The results for quality of life 

measures can be seen in Figure 23.

0

10

20

30

40

50

60

70

80

90

Improved air

or water

quality

 Improved

quality and/or

access to land

and natural

resources

Waste

reduction /

waste

management

Improved

energy

security and

independence

Improved

energy

efficiency or

conservation

None

Slight

Moderate

High
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

September 2015 | 69

Figure 23: Qualitative Survey Responses: Quality of Life Measures (n=97)19

Source: Survey response data

5.4 Summary of Data Analysis Findings

In general, the program evaluation element based on the survey of grant recipients 

yielded the following general findings:

• While the electronic survey elicited some response from grant recipients, the 

follow-up efforts improved the survey response rate three-fold.

• North Carolina and Pennsylvania accounted for the highest survey response 

rates with 73 percent and 52.6 percent, respectively.

• The grant funding created or retained nearly 2,200 jobs over the study 

period, excluding the Enterprise Development Programs. Business/training 

jobs accounted for the largest share of the job creation and retention. 

• Projects that responded to the survey leveraged an additional $179.5 million 

in private investment over the predictions. The Southern Appalachian Fund 

Technical Assistance and Operational Assistance grants accounted for 

approximately $100 million of this. Enterprise Development Programs 

leveraged $39 million more in private investment than anticipated. 

• According to the survey responses of grant recipients, job creation and 

retention projects as a whole were viewed as most successful at impacting 

economic measures, and least successful at impacting environmental 

measures.

 

19 Note that n=97 for all categories except “improved civic life and governance” where n=96.

0

10

20

30

40

50

60

70

80

Improved

access to

culture, arts,

historic sites

Improved civic

life and

governance

Improved

public health,

safety, or

wellbeing

Improved

public

services and

institutions

Increased

recreational

opportunities

Improved

diversity of the

population

Increased

education

levels of the

population

None

Slight

Moderate

High
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

70 | September 2015

6 Case Studies of Best Practices for Job 

Creation and Retention

The 15 case studies contained in this section:

1. Highlight a diverse group of inventive programs conducted over a range of funding 

cycles;

2. Document the strategies and approaches used to meet barriers or challenges;

3. Identify issues with outcome measures, techniques and means to track project 

achievements; 

4. Describe the outcomes and impacts of the programs; and

5. Advise communities planning similar programs of lessons learned in the course of 

the administration of programs and the grant process.

The programs selected for in-depth case study were chosen based on notable 

approaches and results. The Project Team identified a group of approximately 30 noninfrastructure projects distributed among Appalachian Regional Commission (ARC) 

member states based on project descriptions, information in the ARC.net database and 

preliminary survey results. The programs balanced job creation and retention projects. 

The Project Team and ARC staff culled the preliminary list down to 15 projects for the 

case study analysis. Due to non-response, some of the initially selected projects were 

replaced with alternative project options, resulting in the final list presented in this 

chapter.

In addition to jobs, these examples generated income and other quantitative outcomes, 

accompanying qualitative impacts, and policy implications for Appalachian residents and 

businesses. They enhanced economic development by attracting new industry, 

encouraging business expansion, diversifying local economies, and generating 

permanent, private-sector jobs.

Case studies were representative of each of the project-type categories as identified in 

Chapter 5.1.1. Nine of the 15 case studies were Business/Training related projects and 

two were Energy/Environmental projects. The remaining categories, Local Food, Child 

Care/Education, Community Building/Tourism and Planning/Research accounted for one 

case study each.

All but three of the thirteen states in the Appalachian region are represented in the 

selected case studies. The initial list contained options from every state, but the final list 

was unable to maintain this representation. In total, five cases are for projects that were 

implemented in North Carolina, more than other states in the Appalachian Region; two 

are based in Virginia and one each in Alabama, Georgia, Kentucky, Maryland, New York, 

Ohio, Pennsylvania, and West Virginia. The states without case studies are Mississippi, 

South Carolina, and Tennessee. Three of the projects were considered “Commission” 

projects, with funding directly out of ARC funds rather than a state’s allocation.
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

September 2015 | 71

The 15 case studies include:

1. A North Carolina program funding diverse grassroots programs promoting Triple 

Bottom Line projects. 

2. An Alabama program promoting export of state goods.

3. A Georgia program to restore a city’s downtown to its historical original look.

4. A Commission-funded Virginia program to develop an alternative energy curriculum.

5. A Maryland program creating business opportunities around historic hiking and 

biking trails.

6. A Virginia program installing exhibits and media presentations promoting regional 

arts, culture and history.

7. A North Carolina program creating manufacturing jobs for individuals with 

disabilities.

8. An Ohio program promoting farmers’ markets and local suppliers.

9. A New York program launching a reuse center.

10. A Kentucky program developing an energy boot camp.

11. A Pennsylvania program incubating small businesses, promoting international 

business development and procurement technical assistance.

12. A North Carolina program supporting the growth of clean energy economy.

13. A West Virginia program supporting an early-stage investment fund in high 

technology projects.

14. A Commission-funded North Carolina program improving curricula and training 

teachers in entrepreneurial education activities.

15. A Commission-funded North Carolina program promoting manufacturing and 

overseas sales of wood products and home furnishings.

Table 10 details the 15 case studies by grantee, project category, year of funding, 

funding level, and primary outcome. 
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

72 | September 2015

Table 10: Case Study Overview20

Grantee Project Title

(Grant Number)

Project 

Description

Project Type Year ARC 

Funds

Total 

Funds

Primary 

Outcomes

6.1 The 

Conservation 

Fund

Capacity building 

for Asset-Based 

Economies in 

Western North 

Carolina

(NC-16686-I)

To develop 

local AssetBased projects 

through a 

series of 

training 

workshop and 

hands-on 

technical 

assistance.

Planning / 

Research

2010 $100,000 $229,700 17 jobs 

created

6.2 University of 

Alabama

University of AL 

Export Trade 

Financing 

Program (EXTRA)

(AL-15573-C3)

Continuing 

support for a 

program that 

addresses 

access to 

technical 

assistance and 

finances in 

export 

marketing.

Business / 

Training

2010 $151,125 $302,250 92 jobs 

created; 

$6.7 

million LPI

6.3 City of 

Toccoa

Toccoa 

Downtown 

Revitalization 

Project (GA15259-I)

Revitalize the 

historic 

downtown 

business 

district.

Community 

Building / 

Tourism

2005 $300,000 $1,400,000 1,100 jobs 

retained

6.4 Mountain 

Empire 

Community 

College

Mountain Empire 

Alternative Energy 

Initiative

(CO-16396-I)

Develop a 

career Studies 

Certificate to 

articulate with 

an Energy 

Management 

Associate 

Degree 

program.

Child Care / 

Education

2009 $40,000 $82,386

6.5 The Progress 

Fund

Great Allegheny 

Passage 

Economic 

Development 

Program

(MD-16055-I)

To coordinate 

the Trail Town 

Program in the 

Maryland 

communities

Business / 

Training

2008 $100,000 $200,000 76 jobs 

created; 17 

new 

businesses 

created

6.6 Round the 

Mountain 

Southwest 

VA Artisan 

Network

Heartwood: 

Southwest VA 

Artisan Gateway

(VA-16005-I)

For state-ofthe-art 

iconography 

interior exhibits, 

media 

presentations, 

furnishing and 

equipment at 

The Heartwood: 

Southwest 

Virginia’s 

Artisan 

Gateway

Business / 

Training

2008 $1,000,000 $2,058,140

 

20 Note that the Mountain Empire Alternative Energy Initiative (6.4) did not directly have any reportable job or 

business creation impacts, though there have been 24 students enrolled in the program. Additionally, the 

Heartwood Southwest VA Artisan Gateway (6.6) does not have easily isolated impacts, and thus none have been 

noted in the table. 
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

September 2015 | 73

Grantee Project Title

(Grant Number)

Project 

Description

Project Type Year ARC 

Funds

Total 

Funds

Primary 

Outcomes

6.7 Marketing 

Association 

for 

Rehabilitation 

Centers 

(MARC)

MARC Custom 

Medical Products 

(NC-15696-I)

A program to 

develop 

manufacturing 

capacity to 

produce custom 

medical drapes 

and other 

products and 

generate funds 

for 

development 

programs for 

people with 

disabilities

Business / 

Training

2007 $200,000 $883,400 162 jobs 

created; 

150 jobs 

retained; 

$2.7 

million LPI

6.8 Ohio State 

University 

Research 

Foundation

Strengthening 

Appalachia’s 

Farmers’ Markets 

(OH-16104-I)

To provide lowand no-cost 

marketing and 

promotional 

strategies and 

templates for 

Appalachian 

Ohio farmers’ 

markets.

Local Food 2008 $75,000 $150,157 75 jobs 

retained

6.9 Cornell 

Cooperative 

Extension of 

Tompkins

Finger Lakes 

Reuse Center 

Start-up

(NY-16085-I)

To launch the 

Finger Lakes 

Reuse Center.

Business / 

Training

2008 $148,463 $300,444 9 jobs 

created

6.10 North 

Carolina 

REAL 

Enterprises, 

INC

Expanding 

Entrepreneurship 

in Appalachia –

REAL 

Entrepreneurship 

Curriculum 

Improvement

(NC-15807-I)

To revise the 

REAL 

Enterprise 

curriculum to 

include new 

technologies, 

more financial 

literacy, and 

information 

about emerging 

industry 

clusters and 

train new 

faculty.

Business / 

Training

2007 $82,600 $205,350 90 jobs 

created; 90 

new 

businesses 

created

6.11 Northeastern

Pennsylvania 

Alliance

Northeastern PA 

Alliance 

Enterprise 

Development

(PA-8291-C29)

Project will 

continue small 

business 

financing, 

international 

business 

development 

assistance, 

procurement 

technical 

assistance, 

support of PA 

Source Net and 

other ecommerce 

activities.

Business / 

Training

2010 $400,000 $800,000 169 jobs 

created; 

640 jobs 

retained

6.12 Land-of-Sky 

Regional 

Council

Building the Clean 

Energy Economy 

in Western North 

Carolina

(NC-16543-I)

Activities to 

support the 

growth of a 

clean energy 

economy in 

Western North 

Carolina

Energy / 

Environmental

2010 $100,000 $200,000 9 jobs 

created; 

173 new 

businesses 

created
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

74 | September 2015

Grantee Project Title

(Grant Number)

Project 

Description

Project Type Year ARC 

Funds

Total 

Funds

Primary 

Outcomes

6.13 West Virginia 

High 

Technology 

Consortium 

Foundation

WV High 

Technology 

Consortium 

Foundation 

INNOVA 

Commercialization 

Technical 

Assistance (WV15262-I)

Continuing 

support for an 

early-stage 

investment fund 

linked to a 

range of 

technical 

assistance 

efforts.

Business / 

Training

2005 $125,000 $250,000 22 jobs 

created

6.14 Kentucky 

Highlands 

Investment 

Corporation

Kentucky 

Highlands Energy 

Business Boot 

Camp

(CO-15789-I)

To develop and 

Energy 

Business Boot

Camp, 

providing 

intensive 

performancebased 

entrepreneurial 

training and 

mentoring to 

growing firms.

Energy / 

Environmental

2007 $75,000 $225,000 15 jobs 

created

6.15 North 

Carolina 

Department 

of Commerce

FY 2010-2011 

Wood 

Products/Home 

Furnishings 

Initiative (CO15268-I)

To assist 

companies in 

North Carolina 

and other 

states in the 

Appalachian 

Region export 

wood products, 

manufacturing 

supplies and 

home 

furnishing.

Business / 

Training

2010 $25,000 $75,000 Floor sales 

= 

$200,000; 

Expected 

sales = 

$8.5 

million

Each case study below is organized and presented in the following standard format: 

• Community Profile

• Project Description

• Project Planning and Implementation

• Economic and Community Impacts

• Lessons Learned

• Conclusions

The program descriptions are based on a range of information sources, including 

telephone and in-person interviews of program officials, follow-up calls, survey results, 

online reports and website content. Program documents were also utilized and included 

progress and final reports, as well as materials supplied by ARC, the grantees and other 

funding organizations. The sections on community profiles were sourced mostly from 

reports and tables compiled by ARC and available on their website, though some 

additional outside resources were consulted.

The case studies examine a range of issues including, but not limited to:

• How was the project conceived? What organizations were involved? What 

did it take to make the concept a reality?
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

September 2015 | 75

• What was the rationale for the project? What specific community and 

economic factors was it designed to address?

• What barriers or challenges did the project face? How were they overcome? 

How did project managers conceive these strategies?

• What has been the project’s impact in terms of direct jobs and income, spinoff activities, new relationships and capacity developed, and other community 

benefits?

• How is success being measured? What is the rationale for these metrics? 

How is the data collected?

• What were the key factors that made the project successful? Whose support 

was needed? What other public and private resources were required? 

• What were the staffing issues, if any?

• What was learned from the project and what advice might be offered to 

organizations developing comparable projects?

A summary of Key Lessons Learned concludes the chapter. These conclusions are 

compiled and consolidated from lessons identified in the individual case studies.

6.1 Capacity Building for Asset-Based Economies in 

Western North Carolina: Resourceful Communities 

Program

The purpose of this project is to assist The Conservation Fund’s Resourceful 

Communities Program (TCF/RCP) in supporting a network of grassroots organizations 

across North Carolina but primarily in economically distressed areas of the state. 

TCF/RCP provides re-granted funds to organizations for their projects by providing small 

grants, training and technical assistance. These funds come from the Appalachian 

Regional Commission as well as other private and public sources. In turn, these projects 

aspire to attract visitors, promote arts and culture, preserve the environment and improve 

energy efficiency. Target audiences include entrepreneurs, youth, residents and nonprofit organizations.

ARC awarded a grant of $100,000 to The Conservation Fund’s Resourceful 

Communities Program in 2010 to support a network of grassroots organizations in 

economically stressed areas of North Carolina. Local funds brought total program 

support to $229,700. This project is categorized as a Planning/Research project that 

focused on building community-level support for development and implementation of 

asset-based development strategies.

6.1.1 Community Profile

Twenty-nine counties in North Carolina are part of the Appalachian Region, a 205,000-

square-mile region that follows the spine of the Appalachian Mountains from southern 

New York to northern Mississippi. The 29 counties within the region and in NC are: 

Alexander, Alleghany, Ashe, Avery, Buncombe, Burke, Caldwell, Cherokee, Clay, Davie, 

Forsyth, Graham, Haywood, Henderson, Jackson, McDowell, Macon, Madison, Mitchell, 
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

76 | September 2015

Polk, Rutherford, Stokes, Surry, Swain, Transylvania, Watauga, Wilkes, Yadkin, and 

Yancey.

The 2012 population of the Appalachian region of North Carolina was just over 1.7 

million residents. Comparatively, per capita income in the region was only $34,215, or 

about 78 percent of the U.S. value, 90 percent of North Carolina and 95 percent of all 

Appalachia. Per capita income in the Appalachian counties of North Carolina ranged 

from a low of 62 percent of the U.S. per capita average in Graham County ($26,988) to 

92 percent in Polk County ($40,232).21

ARC defines the economic status of each county with one of five possible economic 

status designations based primarily on three indicators: three-year average 

unemployment rate, market income per capita, and poverty rate. The five status 

designations are: distressed, at-risk, transitional, competitive, or attainment. “Distressed” 

counties are the most economically depressed and “attainment” counties are the most 

prosperous. All but one of the Appalachian North Carolina counties was defined as atrisk or transitional; the exception being Graham County which received the lower 

distressed classification. In FY 2012, none of the counties in the North Carolina region of 

Appalachia received the higher economic status designation of competitive or 

attainment.22

In 2012, when both the national and Appalachian Region’s unemployment rate was 8.1 

percent, the counties of Appalachian North Carolina experienced a higher unemployment 

rate of 9.4 percent, about equal to the state rate. Unemployment rates in the Appalachian 

counties ranged from 7.3 percent in Henderson County to 16.8 percent in Graham 

County. Only three Appalachian North Carolina counties had lower unemployment rates 

than the national average.23

The poverty rate in Appalachian North Carolina from 2008 to 2012 was 17.9 percent; 

120.3 percent of the U.S. rate of 14.9 percent. The poverty rate for all of the Appalachian 

Region was 16.6 percent while that for North Carolina was 16.8 percent.24

6.1.2 Project Description

The Conservation Fund’s Resourceful Communities Program (TCF/RCP) supports a 

network of 300 grassroots and community organizations in low-wealth communities that 

advocate the “triple bottom line” (TBL) where outcomes promote sustainable economic 

development, social justice, and environmental stewardship. It provides opportunities to 

preserve landscape, lift residents out of poverty, and celebrate the state’s unique culture. 

TCF/RCP helps partners meet challenges by building on local assets—people, heritage, 

land and water—to create sustainable communities. The concept of the TBL is the 

foundation of TCF/RCP’s work.25

 

21 U.S. Department of Commerce Bureau of Economic Analysis; Local area Personal Income, 2012. Compiled by the 

Appalachian Regional Commission, March 2014.

22 County Economic Status Fiscal Year 2012, Appalachian Regional Commission.

23 Unemployment Rates, 2012; Appalachian Regional Commission.

24 Poverty Rates: 2008-2012; Appalachian Regional Commission.

25 Resourceful Communities – Our Approach. http://www.conservationfund.org/what-we-do/resourcefulcommunities/our-approach
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

September 2015 | 77

While the traditional strategy to build economic opportunity in these areas has been 

based on the extraction of natural resources, TCF/RCP recognized the strategy is not 

sustainable, nor were the impacts and benefits equitable. Instead, TCF/RCP helps 

communities:

• Capitalize sustainably on their significant natural resource base;

• Tap their cultural traditions of land and water stewardship and 

entrepreneurship; and

• Leverage environmental dollars, expertise and techniques to increase local 

wealth and improve the quality of life by “creating new economies” that 

protect, enhance and restore the natural resource base.

TCF/RCP helps these organizations carry on their work and ensure sustainable change 

through integrated solutions. All of the organizations in this asset-based development 

project were either working on a TBL project or working towards TBL projects. Initiatives 

included projects covering sustainable agriculture, alternative energy, youth 

development, eco/agricultural tourism, and others.

What makes the program unique is the breadth of support TCF/RCP provided for 

organizations applying for and administering projects that it funded. Funds have 

supported a wide range of projects, including eco-tourism, sustainable agriculture, youth 

leadership development, alternative energy, food distribution centers, green job training 

programs, paddle trails and more.26Another distinctive aspect of the program was the 

assistance provided to organizations seeking potential funding from other funding 

sources.

ARC funds were used by the grantee to develop local asset-based projects through a 

series of training workshops and hands-on technical assistance. Local projects then 

received implementation awards of up to $15,000 from the Creating New Economies 

Fund, to implement those strategies that have clearly articulated social, economic, and 

environmental benefits.

Regional training workshops were held, along with a state-wide gathering of peer 

communities active in the asset-based development field. ARC support underwrote the 

costs of personnel, meetings, travel, overhead, and the Creating New Economies Fund.

6.1.3 Project Planning and Implementation

The adverse shift in the economic environment over the past ten years spurred the 

Conservation Fund’s Resourceful Communities Program (TCF/RCP) to undertake the 

“Capacity Building for Asset-Based Economies in Western NC” project. The program 

proposed four major areas of work: conduct outreach to North Carolina ARC partner 

municipalities and community organizations to help them plan and develop asset-based 

development programs; award grants of up to $15,000 for projects and programs that 

promote the “triple bottom line”; conduct regional leadership training workshops to 

increase community capacity for asset-based development efforts; and provide one-onone assistance to individual groups to meet their needs. It also provided critical support 

to supplement local government services that had been cut back.

 

26 http://www.conservationfund.org/what-we-do/resourceful-communities/our-approach
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

78 | September 2015

The Creating New Economies Fund (CNEF) grant program was created to help partners 

implement TBL projects, providing seed grants of up to $15,000. Through ARC funding, 

TCF/RCP funded 12 projects totaling $85,000 from both 2011 and 2012 CNEF grant 

cycles. The funds were also designed to build organizations’ long-term capacities to 

access grant funding. Seven additional projects were funded through $49,000 in 

leveraged funds.

Many grassroots organizations are rurally isolated, lack resources and have little or no 

staff. These workshops allowed for cross pollination of ideas and networking of people 

across race, age, economic status and geography.

Through nurturing networks, organizations could benefit from the past success 

experiences of previous participants. Although TCF/RCP recognized these potential 

projects as unique, it maintained they could benefit from others’ proficiencies - through 

“convenings” (i.e., an annual gathering where grassroots partners can network, learn 

skills, discuss issues and connect with resources), peer learning, and site visits.

Projects included a green jobs training program, heritage tourism site development, 

training to participate in ecological restoration work, native plant propagation and 

business planning support.

Workshop topics ranged from technical assistance in general proposal writing skills to 

budgeting, economic development, setting objectives and goals, developing logic 

models, program evaluation, and others. Support requested from organizations included 

fundraising, board recruitment and leadership succession. Workshops were provided 

free of charge and lasted one day. Travel expenses (including meals and mileage) were 

available in some cases, as was lodging for select participants based on arduous travel 

times. Workshops ranged from individualized one-on-one assistance to small- and 

medium-sized workshops.

To obtain support, organizations completed a pre-screening procedure to determine if 

potential projects met TCF/RCP’s TBL criteria. The pre-screening application included 

information about the organization, the project idea, and how it promoted the TBL. 

Successful programs were then required to submit full proposals including information on 

stakeholder involvement, an evaluation plan, a detailed budget and how the project 

advanced the mission of the organization. TCF/RCP provided assistance throughout the 

process, including site-visits if requested by the organization or if suggested by 

TCF/RCP. Post grant award, TCF/RCP assisted organizations in securing additional 

funding, and in areas such as board and organizational development and strategic 

planning.

6.1.4 Economic and Community Impacts

Without ARC funding, this project would not have been undertaken. Essentially, the 

grantee attributes the outcomes of the program to ARC support and funding. Outcome 

measures were collected directly from the sponsored organizations.

Of 36 participants attending the 2011 technical assistance session, 23 submitted preproposals from Appalachian North Carolina requesting combined funding of $212,898. 

The 17 applicants invited to submit full proposals received feedback on their preproposals and were encouraged to seek additional support and feedback from staff as 
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

September 2015 | 79

they developed their full proposals. In 2012, 16 pre-proposals were submitted from 

western North Carolina applicants requesting a total of $172,800 in funding. This regranting program helped attract additional government and/or philanthropic funding.

Utilizing ARC funds, TCF/RCP awarded 12 grants for a total of $85,000 in 2011 and 

2012 that leveraged $820,524, and engaged over 2,000 people in their work. Seven 

additional programs were funded with $49,000 in leveraged funds during the 2011 and 

2012 grant cycles.

Additional economic outcomes included the creation of 52 part-time (8 permanent, 44 

temporary); and 17 full-time (1 permanent, 16 temporary) positions. Agricultural 

industries created or retained 36 part-time jobs (4 permanent, 32 seasonal) while 11 

part-time positions (6 permanent, 5 temporary) were created or retained in the services 

field.

As a result of this project, one business was retained and one improved. Ninety 

organizations located in 14 communities saw improvement, including new sources of 

income or increasing income for local residents, along with increased value of community 

assets. The grantee attributed these gains to the project. Programs improved the skill 

level of the workforce, access to local markets for products, businesses, artisans and 

entrepreneurs, and they supported cultural, historic sites and other amenities.

The project provided capacity building support to nonprofit organizations, via workshop 

trainings and direct technical assistance, at three workshops. There were a total of 93 

participants in these trainings. One-on-one technical assistance was provided to ten 

organizations. As a result, these organizations now have more engaged boards, new 

revenue streams, more strategic programming and are better equipped to face 

organizational challenges.

The project had a moderate economic impact in increasing local business sales or value 

of business assets, and increasing the value of household assets for local residents. 

Local business viability was modestly influenced, along with improved stability and 

sustainability of the local economy. Project impacts improved quality and/or access to 

land or natural resources, waste management, energy security and independence and 

energy efficiency or conservation. Quality of life issues, such as civic life, governance 

and recreational activities, along with the population’s knowledge and educational levels, 

were positively altered.

6.1.5 Lessons Learned

The lessons learned drawn from this program are drawn from interviews of program 

administrators, the results of a post-project evaluation survey conducted by an ARC 

contractor of the Non-Infrastructure grant recipients during the 2004-2010 grant period, 

and the Final Performance Narrative report completed by the grantee.27

• Need for increased coordination and “peer learning.” Many of the 

grassroots organizations TCF/RCP works with offer some truly innovative 

strategies but start from scratch when initiating projects. This is despite the 

reality that these organizations tend to face similar organizational challenges, 

 

27 Final Project Performance Narrative Project Activities and Accomplishments; April 1, 2010 – March 31, 2012. The 

Conservation Fund’s Resourceful Communities Program; ARC Final Report.
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

80 | September 2015

such as the need for diversified funding streams, weak board participation, 

and lack of leadership transition. Although TCF/RCP offers opportunities for 

organizations to learn together and from each other, more opportunities for 

“cross pollination” and learning would more efficiently address organizational 

challenges, disseminate lessons learned and share opportunities seized.

• Workshops can inform organizations about available resources for 

organizational development, project planning and other topics. Small 

grassroots organizations often lack the staffing or resources to pursue 

outside funding and have limited knowledge of federal, state or local sources 

that can be of assistance. Pooled information and shared lessons learned 

can assist these organizations, particularly those that face rural isolation 

and/or are located in economically stressed areas.

• Recognize programs that do not address traditional issues. Grassroots 

organizations that are not issue-based may struggle to secure funding for 

projects. TCF/RCP has found that many grassroots organizations are 

expanding into new program areas they have not previously undertaken –

namely health, food access and sustainable agriculture work to address 

significant community need not being met by local agencies. Additional 

programming and funding support for unique opportunities that fall outside 

conventional issues or for non-traditional partners is needed to shore up 

current limited availability.

• Demand for TCF/RCP programs and small grants has grown 

dramatically. Participation continues in regional leadership workshops and 

“Grass Roots Convenings.” These programs are increasing each year as do 

applications and requests for CNEF grants. TCF/RCP’s response has been 

to continue to offer their services at no-cost to participants and seek 

additional and diversified funding for its work.

• A triple bottom line approach creates more innovative projects with 

broader impacts. TCF/RCP uses the triple bottom line of sustainable 

economic development, environmental stewardship, and social justice as the 

basis for all its work, including the small grants program. The triple bottom 

line projects are innovative and far-reaching, while also addressing critical 

issues and needs in partner communities. Economic opportunities are 

provided, while the region’s critical resources are protected and the benefits 

extend to the most under-served communities. Rather than addressing a 

single community need, the triple bottom line approach provides a foundation 

to more systematically address inter-related challenges.

6.1.6 Conclusion

The four major goals of the program were: 

1. Conduct outreach to NC ARC partner communities to help them plan and develop 

asset-based development projects.
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

September 2015 | 81

2. Award grants of up to $15,000 for projects and programs that promote the “triple 

bottom line” of sustainable economic development, environmental stewardship and 

social justice through the CNEF program.

3. Conduct regional leadership training programs.

4. Provide one-on-one technical assistance to individual groups, tailored to their unique 

needs.

Overall, TCF/RCP achieved its projected goals. Though only one of two planned CNEF 

workshops was held in 2010, the project largely succeeded by training 35 participants at 

the grant writing workshop and an additional 30 community leaders participated in a 

targeted grant writing session for western small town members. The TCF/RCP also 

offered three regional leadership workshops in Western North Carolina for a total of 93 

participants during the project period. Additionally, ten partner organizations were 

provided one-on-one technical assistance. This assistance is ongoing and consists of 

mentoring and training.

Grant awards funded 19 projects for a total of $136,000 during two grant cycles.28 Of 

those, $85,000 in ARC funds enabled 12 grants for projects that leveraged $820,524 and 

engaged over 2,000 individuals.

TCF/RCP extends sincere appreciation for ARC’s on-going support. ARC’s investment 

has helped create substantive change in Western North Carolina by supporting local, 

asset-based economic development efforts in the region.

6.2 University of Alabama Export Trade Financing 

Program

The purpose of this Business/Training project was to deliver export technical assistance 

to small companies within Appalachian Alabama. The Export Trade Financing Programs 

(EXTRA) assisted small manufacturers in Appalachian counties in increasing 

international sales, and trained local bankers interested in providing trade financial 

services to small businesses within their markets.

ARC awarded a grant of $151,125 to the University of Alabama in 2010 to provide indepth technical and short-term assistance in export financing services to small 

businesses and to train and educate local bankers to provide trade finance services to 

their customers. Local funding brought the total program support to $302,250.

6.2.1 Community Profile

Thirty-seven counties in Alabama are part of the Appalachian Region. The counties in 

Appalachian Alabama are: Bibb, Blount, Calhoun, Chambers, Cherokee, Chilton, Clay, 

Cleburne, Colbert, Coosa, Cullman, De Kalb, Elmore, Etowah, Fayette, Franklin, Hale, 

Jackson, Jefferson, Lamar, Lauderdale, Lawrence, Limestone, Macon, Madison, Marion, 

Marshall, Morgan, Pickens, Randolph, St. Clair, Shelby, Talladega, Tallapoosa, 

Tuscaloosa, Walker, and Winston.

 

28 Ibid. p.16.
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

82 | September 2015

The 2012 population of Appalachian Alabama was about 3.1 million residents. The per 

capita income for the region was only $37,046, or about 85 percent of national per capita 

income. However, it slightly exceeded, by three percent, the per capita income of the 

state of Alabama and all of Appalachia. Per capita income by Appalachian Alabama 

counties ranged from 57 percent of U.S. average (in Bibb County and Coosa County) to 

105 percent in Shelby County (not included in this project) and 102 percent in Jefferson 

County.29

ARC defines the economic status of each county with one of five possible economic 

status designations based primarily on three indicators: three-year average 

unemployment rate, per capita market income, and the poverty rate. The five status 

designations are distressed, at-risk, transitional, competitive or attainment. “Distressed” 

counties are the most economically depressed and “attainment” counties are the most 

prosperous. In FY 2012, all but two of the counties in Appalachian Alabama were 

designated as distressed, at-risk, or transitional. One was deemed competitive and one 

attainment.30

6.2.2 Project Description

The purpose of the project was to provide in-depth and short-term targeted, one-on-one 

technical assistance in export financing small businesses in Appalachian Alabama, and 

education and training for local banks to learn how to serve local firms with export 

financing programs. Expertise was provided by the University of Alabama’s Alabama 

International Trade Center (AITC). Targeted businesses were new-to-market exporters, 

potential exporters who might have been approached by an overseas client interested in 

purchasing U.S.-made products, or existing exporters. The targeted support occurred 

from 2008 to 2011 with the goal of increasing international sales and sustaining export 

growth.

ARC funding paid for personnel costs, in-state travel, and general operating expenses 

such as supplies, subscription services to overseas media and technical publications, 

and a 15 percent indirect rate. Approximately one-half of ARC’s award went towards 

personnel, including independent contractors with highly specialized skills in credit 

analysis of foreign companies, government lending programs, and loan development.

6.2.3 Project Planning and Implementation

The impetus for this project was a gap in export finance services for small business 

exporters in Appalachian Alabama brought on by the 2007 to 2009 financial crisis, as 

large commercial banks curtailed commercial lending, consolidated operations and/or 

relocated out-of-state. As a result, banking expertise in the international trade division of 

local commercial banks became scarce or nonexistent at a time when more small 

businesses were getting involved or growing their export transactions.

 

29 U.S. Department of Commerce Bureau of Economic Analysis; Local Area Personal Income, 2012. Compiled by the 

Appalachian Regional Commission, March 2014.

30 County Economic Status Fiscal Year 2012. Appalachian Regional Commission.
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

September 2015 | 83

Alabama exports were $15.5 billion in 2010, up 25 percent from $12.3 billion in 2009, 

and up 112 percent over 2000 export figures.31 Demand was growing for international 

banking services – wire transfers, foreign exchange, letters of credit and export 

financing. The state trade office, the Alabama Development Office, initiated an 

aggressive trade promotion strategy to increase exports for small businesses, sparking 

growing interest from small firms.

Before the banking crisis, according to an AITC official, more than 150 banks near 

Birmingham, the state’s primary manufacturing center, provided know-how in 

international trade. Currently, only one regional bank has international banking staff and 

operations. Large commercial banks did not service the smaller transactions of the small 

business customers, leaving them without access to banks willing to extend credit on 

export transactions. Small local banks were not familiar with export loan guarantee 

programs. The need for local access to “hands on” technical financing assistance for 

small export businesses was evident, to boost export sales and provide entry to export 

financing services. Project sponsors identified a requirement for education and training of 

local community bankers.

The project goal was for AITC to identify 25 Alabama companies in the targeted 

Appalachian counties requiring assistance in basic export finance. In addition, training 

would be provided to 10 local and community banks and 50 local bankers who were 

interested in providing trade finance services to local small businesses.

AITC would assist local and community banks during the complete project period, 

consisting of 12 months. Months 1-6 of the project period would identify participants with 

export growth potential. During months 3-12 of the project period, in-depth assistance 

would be provided to 25 exporters.

Company candidates were identified from export assessments, site visits and referrals 

from AITC’s partner network, especially the state trade office. Other sources of 

identification included local chambers of commerce, logistics providers and banks. 

Manufacturers learned of AITC services through outreach events hosted by alliance 

organizations, while state-sponsored trade missions provided additional candidate 

companies. The project team networked with the private sector, exporting companies, 

and economic development organizations to generate potential referrals. Following the 

project team’s on-site interviews or export assessments for each potential candidate 

company, the number of companies was narrowed and 25 selected for in-depth 

assistance.

AITC provided services to 25 to 30 companies in a given year. Companies typically 

received between five and 100 hours of one-on-one assistance. No parameters for 

amount of service, in terms of hours or dollars expended per company, were developed 

in advance. Over the course of a year, on average, each company used about $5,000 -

$8,000 of program funds. Some remained with AITC over time; others received shortterm services. In some cases, those already involved in exporting had a particular 

difficulty with a transaction and AITC provided very short-term service to move that 

transaction forward. Another company might be looking to enter a new market or be 

 

31 Export Trade (EXTRA) Financing Program (Continuation). Alabama International Trade Center, Project Summary. 

The University of Alabama, 2013.
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

84 | September 2015

approached by a new partner company where more involved service over time was 

required. A third company may simply require intelligence about a particular market.

AITC trade counselors, both full- and part-time, provided the expertise; when available, 

export sales and logistics experts were recruited from the trade side of local banks. 

Others had remained in-state when their banks relocated to larger out-of-state cities such 

as Atlanta, Miami or Charlotte.

The program taught the basic day-to-day activities necessary to carry on international 

business, “Exporting 101,” along with finance-specific content. Basic topics included: 

getting companies to pay for your goods and services; lines of credit; how to access 

Small Business Administration (SBA) loan guarantees for exporters; what services were 

available from banks; how SBA loans worked; what payment terms they could offer; how 

to perform due diligence on foreign companies or the host country; and the programs 

available through the Export-Import Bank of the United States. Firms were also 

interested in information about new markets or how to evaluate a new foreign company’s 

credit worthiness. While relevant to current deals, the information also served as a 

framework for future business.

Beyond the basics, international finance-specific content was provided, consisting of:

• Training on export financing tools including risk factors, loan programs, credit 

insurance, and other transactional issues

• Meeting facilitation between the client and financial institutions

• Collection and review of financial statements on Alabama exporters and 

foreign buyers

• Assessment of country risk and country limitation factors

• Assessment of foreign buyer risk, financial statements, and provision of credit 

analysis

• Recommendations of specific export financing programs and services

• Assistance in the preparation of financing proposals and bank participation

• Hands-on assistance to secure financing

The most sought-after information focused on trade with Canada, Mexico or NAFTA 

trading partners. Additional requests centered on trade potential with China or the United 

Kingdom. However, some companies ventured into agreements with other so-called 

“risky” countries such as Argentina.

AITC addressed the lenders’ requirements on two levels. Staff met with bankers and 

provided individual technical assistance in one-on-one sessions, and AITC offered 

training programs for bank staff. For example, staff from AITC might meet with loan 

officers from a community bank and describe the resources available to assist exporters 

and clarify how credit could be extended (backed by government loan guarantees). AITC 

also worked with trade associations in the banking sector to develop continuing 

education sessions for bank personnel. It assisted in the design of curriculum targeted at 

lenders attending bankers’ conventions and presented sessions on the exporting needs 

of customers.
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

September 2015 | 85

6.2.4 Economic and Community Impacts

Outcomes were collected annually by AITC via direct interviews of their customers; 

information provided by project partners including the state trade office; and reviews of 

secondary data sources regarding export sales and existing industry development. 

Program outcomes were acknowledged as primarily the product of ARC’s intervention 

and funding. Had ARC not provided funding, program administrators believe the results 

would have been significantly scaled back.

In 2007, international trade in Alabama was a $32.8 billion industry with approximately 

2,300 firms actively engaged in trade and 80 percent represented as small business. 

Companies located in-state sold more than $14.4 billion overseas.32

There were 97 businesses improved during the course of the project. In the 

manufacturing sector, 234 jobs were created. Leveraged private investment totaled 

$18.6 million and export revenues increased $27.7 million. The program provided 

training for 107 individuals from lending institutions in Appalachian Alabama.

Access to markets for local products, businesses, and entrepreneurs was improved as a 

direct result of the program. In addition, new jobs or increased employment in existing 

businesses rose and new sources or increased income of local residents resulted. To a 

lesser degree, the program improved the stability and sustainability of the local economy 

and increased the viability of local businesses. Additional funding was acquired from 

ARC’s global initiative and the SBA STEP program as a result of the program and 

collaboration with the State Trade Office.

The program assisted manufacturing companies from diverse sectors, all located in 

Appalachian counties. Sectors and industries included agriculture, forestry, lumber mills, 

food processing, commodities and farming, industrial and/or packaging equipment, snack 

and pet foods, plastic components and industrial compressors. The businesses were 

located primarily in the five counties surrounding Birmingham, where approximately 60 to 

70 percent of the state’s manufacturing originates. Others were from the northwest 

section of the state near Mississippi, northeast towards Georgia or neighboring the 

Tennessee border.

6.2.5 Lessons Learned

The lessons learned from this program are drawn from interviews of program 

administrators and a post-project survey conducted by an ARC contractor.

• There is value in providing one-on-one, in-depth export finance 

assistance to manufacturers in the Appalachian Region. Various types of 

export assistance are available from several sources; federal, state and local. 

There is benefit within this mix for local in-depth, one-on-one export finance 

programs that target fewer companies with detailed assistance, rather than 

targeting many firms concurrently.

• A variety of events can motivate companies to enter the export 

business and follow-up assistance is advised. Overseas trade missions, 

domestic trade shows and regional workshops can increase businesses’ 

 

32 Brian Davis. A world of opportunities for Alabama business and banking industry, Alabama Banker, Summer 2008.
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

86 | September 2015

awareness of opportunities in the export market. By attending, businesses 

can meet potential customers, learn the overseas market, and gauge 

demand for their products and services. However, these meetings can also 

generate additional questions. A system should be in place to field 

subsequent general and financial questions generated by these encounters.

• Increasing bank engagement in exporting is challenging. The project 

hoped to have a greater impact in fostering community bank engagement in 

the export finance arena; however this outcome did not develop as 

anticipated. The economic problems affecting the nation and the banking 

industry during the grant period discouraged banks, at the time under stress, 

from looking beyond their local customer bases, according to program 

administrators. Nevertheless, they note, the project did increase awareness 

of community banks of the issues involved in international trade such as 

relevant populations, markets, economies and the importance of exporting in 

the global world economy.

• With assistance, small companies can get involved in exporting. The 

export market is not the exclusive purview of large companies. While 

resources and time are required for market entry, companies of any size can 

engage in international trade with some assistance. The costs of entrance 

are lower than ever. A modest level of exporting can become successful to 

where it becomes a routine and growing component of a company’s business 

model.

• Companies need realistic expectations and appreciate scale. Business 

executives sometimes underestimate the resources and time required to 

develop an exporting business. For most businesses, a multi-year 

commitment is required, often longer than anticipated initially. Relationships 

in the exporting arena take time to develop – the gestation period needs to 

be realistic. 

Companies often fail to concede the time it took to reach their current size in 

the domestic market, which may be relatively modest, involving only a small 

number of states. Experts suggest that to develop a viable export capacity, a 

commitment of at least one dedicated full-time staff person is required.

Initially, businesses can be unaware of the size of the international market 

they contemplate entering. Take, for example, a company looking to export to 

Mexico. Mexico City alone is a city of more than 19.4 million people; roughly 

75 percent of the population of the entire Appalachian region. A business is 

unlikely to set up an ongoing export business serving a market of this size 

without a substantial investment in staff, time and other resources. Expert 

assistance can illuminate these important parameters and help in the 

planning of the export business.

• Exporting assistance can lead to success. Some of the Alabama 

companies that received AITC assistance initially are currently among the 

state’s largest exporters.
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

September 2015 | 87

6.2.6 Conclusion

The University of Alabama Export Trade Financing Program successfully increased 

awareness of and involvement in export trade among many businesses located in 

Appalachian Alabama and the rest of the state. With expertise provided by the 

University’s Alabama International Trade Center, the Center filled a void created by the 

country’s economic crisis when larger banks reduced their support for overseas activities 

at a time when these support services were needed to grow markets and sales in a 

stagnating economy. The Center encouraged and assisted local community banks by 

furnishing international business expertise and aggressive trade promotion strategies to 

boost export sales and encourage entry of small businesses in the region.

International trade represents new markets for many small businesses. The grant 

provided service in basic export finance to 25 to 30 small businesses, approximately 20 

local and community banks and 50 local bankers each year. With support from those

experienced in this area, these small businesses learned about the market and 

expanded their opportunities.

Almost 100 businesses were improved during the course of the project and 234 jobs 

created. More than 100 bank personnel from banks located in Appalachian Alabama 

were trained as a result of this program.

6.3 Toccoa Downtown Revitalization Project

The primary goal of this Community Building/Tourism project was to revitalize the 

downtown historic business district of Toccoa, Georgia, by removing a colonnade of 

corroded concrete canopies supported by concrete columns. In 2005, ARC awarded a 

$300,000 grant supporting this revitalization project. An additional $1.1 million in state 

and local funding was obtained to supplement ARC funds.

6.3.1 Community Profile

Toccoa, Georgia, is a community of 8,400 residents in Stephens County, located in the 

Northeast section of the state about seven miles from the border of Georgia and South 

Carolina, about 90 miles northeast of Atlanta.

Manufacturing is the largest industry sector in the City, representing 32 percent of 

businesses. In June 2014, the unemployment rate was 7.2 percent compared to 7.8 

percent in the state. The 2012 per capita income of $18,329 in Toccoa was 27.6 percent 

less than the average for Georgia ($25,309) and 34.7 percent below the national average 

of $28,051. The poverty level in Toccoa, 27.8 percent of the population, was much 

greater than that in the rest of Georgia or the nation, 17.4 percent and 14,9 percent, 

respectively.33

In 2012, per capita income in Stephens County, Georgia was $32,056, or 73.3 percent of 

the per capita income for the entire U.S. The county per capita income also fell below the 

 

33 City-data.com and City-data.com: Poverty Rate Data.
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

88 | September 2015

rate for the combined Appalachian Region, the State of Georgia and the combined 

counties in all of Appalachian Georgia.34

6.3.2 Project Description

The Toccoa Downtown Revitalization Project was designed to revive the downtown 

business district of Toccoa, Georgia, by restoring the town to its original historic look. In 

the 1970s, Toccoa undertook $2 million of construction in the downtown area, 

transforming it into an outdoor-style mall, which was stylish at the time. The project 

featured the construction of cement canopies covering the storefronts and sidewalks. 

These features deteriorated over time and became dated. The 2004 undertaking 

involved removal of the canopies to increase the appeal of the historic downtown and 

boost business, tourism, and a sense of community pride.

6.3.3 Project Planning and Implementation

Planning for the project originally began in 1991 but lacked the political and financial 

support to begin construction until 2004. The canopies that had been erected in the 

1970s had cost approximately $2 million to construct, and the people of Toccoa were 

initially unwilling to support their removal. Over time, however, the condition of the 

canopies declined dramatically. The canopies were cracked and leaking, covered the 

storefronts making it difficult to tell whether businesses were open or closed, blocked 

light from entering businesses, and lacked character and charm to attract tourists and 

visitors. The closing of a major department store contributed to an even sharper decline 

in visitors and shoppers in the area, and left the downtown seeming even more run-down 

and abandoned. Population and business were both in decline.

The original storefronts remained underneath the cement canopies and aluminum plates 

that were installed in the 1970’s. Planners and supporters of the Toccoa Revitalization 

Project believed that restoring the buildings to their original historic appearance could 

increase the appeal of downtown Toccoa, which would in turn increase traffic to 

businesses, property values, out of town visitors to the area, and jobs. The concept of 

historical preservation, however, was not popular amongst community members due to a 

lack of education and information available on the subject.

Through the use of town hall meetings, public hearings, mailed pamphlets, newsletters, 

and presentations on the proposed plans, support grew in the community. The residents 

of Toccoa were shown mock-ups of how the buildings would look with their original 

storefronts restored, and they were educated on the benefits of having a historically 

preserved downtown area. They were allowed input in aspects of the planning, such as 

whether the street would be one way or bi-directional, and what kind of parking would be 

available. The education of the importance of historical preservation and the community 

involvement efforts led to strong support for the revitalization of the downtown.

Once the project had community support, a cost estimate was put together by a 

professional engineering and architectural company in 2004. Project organizers went in 

person to apply for an ARC grant at the local ARC Georgia branch. ARC was the first to 

provide financial backing for the revitalization project, which gave the project a kick-start 

 

34 Personal Income Rates, 2012. Appalachian Regional Commission.
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

September 2015 | 89

and provided it with the credibility it needed to obtain additional funding. The project 

received the remainder of the funding through other agencies: $500,000 through the 

Department of Housing and Urban Development’s Community Development Block Grant 

Program; $75,000 through the U.S. Department of Agriculture’s Rural Business 

Enterprise Grants Program; $340,000 through the State of Georgia’s Equity Fund; 

$300,000 from the Georgia Department of Transportation; and $50,000 through the 

Georgia Department of Community Affairs’ Signature Community Program. The balance 

of $705,000 was provided by the City of Toccoa. The actual cost of the project was 

approximately $2 million, on par with the original estimated cost.

Figure 24: Rennovation of Toccoa, Georgia

Source: Used with permission of Main Street Toccoa

6.3.4 Economic and Community Impacts

This project has succeeded in revitalizing and restoring downtown Toccoa. The canopies 

were removed and 65 storefronts are now restored to their historic appearance. Each 

property owner was provided with individual consultation and the drawings for the 

renovations were approved by the Historical Preservation Commission. In 2011, Toccoa 

was added to the National Park Service’s National Register of Historic Places, providing 

formal recognition of the area’s architectural and historical significance and worthiness of 

preservation. The Georgia Department of Natural Resources has also registered Toccoa 

as a historic district, and Toccoa was a top ten semifinalist in the National Trust for 

Historic Preservation’s 2008 Great American Main Street competition.

In 2008, there were 15 full time jobs, 29 part time jobs and 11 businesses created. There 

were also 55 full time jobs, 35 part time jobs and 30 businesses retained. As of 2013, 

there have been 1,100 jobs retained cumulatively. These are distributed among retail, 

finance and service sectors. Additionally, a total of $27 million private investment has 

occurred in the downtown district. Property values and population in downtown Toccoa 

have increased, with median house value increasing by 24.16 percent since 2000.35

From 2000 to 2010, the population declined by 8.4 percent to approximately 8,250.36

There is also training and technical assistance provided to tourism-based entrepreneurs 

and businesses.

 

35 http://www.usa.com/toccoa-ga-housing.htm#House-Value

36 http://censusviewer.com/city/GA/Toccoa
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

90 | September 2015

Figure 25: Toccoa, Georgia – Before and After Renovation

Source: Used with permission of Main Street Toccoa.

The restoration has also returned Toccoa to its historic street grid to improve auto and 

pedestrian accessibility. Widened sidewalks, added bike lanes, and a new walking trail 

through the downtown have increased pedestrian safety. The street grid has improved 

traffic flow and made businesses more accessible by automobile. There is also improved 

energy-efficiency through new streetscape lighting. To further assist in revitalizing this 

rural community, an Urban Redevelopment Plan was adopted in 2011. Its mission is to 

revitalize the U.S. Highway 123 gateway corridor, attract businesses to downtown, and 

redevelop grey fields and brown fields near the city center.

Figure 26: Currahee Military Museum in Toccoa, Georgia

Source: Used with permission of Main Street Toccoa.

Since the project’s completion, there has been a dramatic increase in visitors and 

investments because of the downtown’s historic feel.37 The City is currently working 

toward establishing an arts and entertainment district to enhance and promote vibrancy, 

and the old movie theater has been renovated to its historic appearance. There have 

been additions to the Currahee Military Museum since the project, as well as 

improvements to the golf course and the addition of a community swimming pool. There 

is now a mindset that historic preservation is important, with more residents taking care 

to preserve their homes during renovations and household improvements. The new 

Toccoa downtown has brought a sense of pride to Toccoa, and sparked an interest in the 

revitalization of the community.

 

37 Joint Development Authority of Franklin, Hart and Stephens Counties – concrete canopies. One Georgia Authority. 

January 26, 2009. http://www.onegeorgia.org/projects/9/Joint-Development-Authority-of-Franklin-Hart-andStephens-Counties--concrete-canopies

BEFORE AFTER
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

September 2015 | 91

6.3.5 Lessons Learned

The lessons learned from this program were drawn from interviews of program 

administrators, a post-project survey conducted by an ARC contractor and online 

resources.

• Keep community members informed and involved to gain and maintain 

support. Project leaders used varied techniques to engage community 

members throughout the project. The use of town hall meetings, public 

hearings, mailed pamphlets, newsletters, and presentations raised 

awareness about the plans and enlisted the support of community members. 

Support was maintained throughout the process by involving the community 

in decisions and important moments throughout construction. The community 

was allowed to vote on certain aspects of the project, such as parking, and 

held a groundbreaking ceremony when the canopies were taken down. Initial 

and maintained support from the community allowed the project to receive 

funding and resulted in a highly receptive and proud community once the 

project was completed.

• Develop relationships with locally invested grant administrators and 

leaders. The original project planners enlisted the assistance of a local grant 

administrator and leaders who were personally invested in the local 

community. By seeking financial assistance through the local ARC branch, 

the project leaders were able to build a strong relationship with a grant 

administrator who was dedicated to the local community and the project. 

They also hired a Main Street Manager who was extremely familiar and 

committed to Toccoa’s growth and success. These strong relationships 

ensured dedication and passion for the project.

• Use education as a means to inspire change/appeal to the history of the 

community. Project leaders used education in order to raise support for the 

project which resulted in changing the mindset of the community. By raising 

awareness about the importance of historical preservation and restoration, 

and by showing Toccoa residents the history that existed in their town, 

leaders successfully illustrated to community members the benefits of 

embracing their history. This resulted in much more than just support for the 

project: residents have continued to embrace Toccoa’s history in construction 

of multiple community buildings; raised awareness in home improvement 

projects; and generated a renewed interest in the history of Toccoa.

• Support existing business during construction. While the new downtown

was under construction, project managers did their best to ensure that 

existing businesses could remain open and were not negatively impacted 

because of the work. They organized to have construction done at night, built 

pedestrian bridges to make sure businesses remained accessible, 

communicated to Toccoa residents that existing businesses would remain 

open throughout construction, and worked closely with these businesses to 

make sure all of their needs were met. This allowed for a smooth transition 

from the old to the new downtown Toccoa without neglecting existing 

business.
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

92 | September 2015

• Change takes time. While the project was initially conceived in the early 

1990’s, construction did not begin until 2004. Project leaders worked 

tirelessly to gain the community and financial support necessary to launch 

construction, and did not give up when things were not moving as quickly as 

they hoped. With the perseverance, determination, and dedication of leaders 

over the years, the project was eventually completed successfully.

6.3.6 Conclusion

The Toccoa Downtown Revitalization Project fulfilled and exceeded its goals and 

expectations. Through communication with the community, support of businesses, strong 

relationships, and perseverance over time, the project was extremely successful in 

transforming downtown Toccoa, increasing business, and inspiring future improvements.

6.4 Mountain Empire Alternative Energy Initiative

The Mountain Empire Alternative Energy Initiative at Mountain Empire Community 

College used ARC funds to develop a curriculum in alternative energy and purchase 

instructional equipment to support the program. Students learned the latest technologies 

in alternative energy, preparing them to meet the needs of the emerging regional 

alternative energy sector. Seven new Energy Technology courses were created, 

eventually resulting in development of one Certificate program and one Associate 

Degree program.

ARC awarded a grant of $40,000 to Mountain Empire Community College for this Child 

Care/Education project in 2009. This grant was combined with local contributions of 

$42,386 for total funding to $82,386.

6.4.1 Community Profile

Mountain Empire Community College (MECC) is located in Big Stone Gap, a town of 

about 5,500 individuals in Wise County, in the mountains of Southwest Virginia. MECC is 

a comprehensive two-year college serving residents of Lee, Scott, Wise, and Dickenson 

counties, and the City of Norton.

Twenty five counties in the Commonwealth of Virginia are part of the Appalachian 

Region, a 205,000-square-mile region that follows the spine of the Appalachian 

Mountains from southern New York to northern Mississippi. The four counties served by 

MECC are all located in the Appalachian Region.

The 2012 population of Appalachian Virginia was about 765,500 residents. The per 

capita income of the region was only $32,701 or about 75 percent of national, 68 percent 

of state and 91 percent of all of Appalachia in per capita income. Per capita income by 

county ranged from 59.7 percent of U.S. average in Grayson County to 103.6 percent in 

Botetourt County. Three of the four counties served by the college realized per capita 

income of only two-thirds the U.S. average. The residents of Wise County, where MECC 

is located, realized a somewhat higher per capita income at close to three-quarters the 

U.S. average. Income data for City of Norton residents are included in the Wise County 

statistics.
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

September 2015 | 93

ARC defines the economic status of each county with one of five possible economic 

designations based primarily on three indicators: three year average unemployment rate, 

market income per capita, and poverty rate. The statuses are distressed, at-risk, 

transitional, competitive or attainment. Distressed counties are the most economically 

endangered and attainment counties are the most prosperous. Of 25 counties (and 

cities) in Appalachian Virginia, only Bath County received a rating of competitive and 

Botetourt County received a rating of attainment, reflecting relatively positive economic 

statuses. While no county received the most severe rating of distressed, the other 

counties all received ratings of at-risk or transitional.38 The four counties served by 

MECC were each designated at-risk.

In Appalachian Virginia, about 8 in 10 individuals 25 years or older have completed at 

least a high school education – a rate that falls below the average for the entire U.S.39

6.4.2 Project Description

In 2009, ARC funded the Alternative Energy Initiative at Mountain Empire Community 

College in Big Stone Gap, Virginia. The purpose was to develop an alternative energy 

Career Studies Certificate to increase the capacity of the southwest Virginia workforce to 

meet the needs of this emerging regional alternative energy sector. The Certificate would 

articulate with an Energy Management Associate Degree program offered at MECC. The 

project also included the purchase and installation of small-scale alternative energy 

demonstration equipment, including a wind turbine, on campus. ARC funds supported 

equipment and operational costs.

The program curricula included the latest advances in alternative energy technology: 

geothermal heat pump skills; the latest building performance evaluation techniques; 

photovoltaic (PV) installation of solar inverters and converters; and solar thermal 

technologies.

6.4.3 Project Planning and Implementation

In the wake of the economic crisis, which severely impacted Appalachian Virginia and 

the coal mining industry in particular, federal funds became available to support 

alternative energy programs. Since MECC already offered related training programs, this 

was an opportunity for the program to evolve in a direction compatible with its mission 

and fulfill one of its objectives: to demonstrate continuous improvement by embedding 

these latest technologies in degree programs in Heat Ventilation and Air Conditioning 

(HVAC) and Electrical. The timing of the grant, as alternative energy became a funding 

priority, coupled with distress in the local energy industry, made the program viable. 

MECC administration was confident the proposed addition could meet a key state 

criterion: in order to get a program approved by the Virginia Community College System 

(VCCS), the college had to demonstrate that graduates could find gainful employment in 

an existing industry in their geographic area.

Resistance to the project came from local mineworkers who viewed this emerging field of 

“alternative energy” as a threat to their livelihoods. From 2006 to 2013, the number of 

 

38 County Economic Status Fiscal Year 2012, Appalachian Regional Commission.

39 Education – High School and College Completion Rates, 2008-2012, Appalachian Regional Commission.
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

94 | September 2015

coal miners employed in Virginia declined by 14 percent, from 5,262 to 4,521. The large 

energy producing companies in the region offered no opposition – supporting 

technologies that would, increase the sustainability of their businesses in the long run. 

Thus, by replacing the unpopular terminology of “alternative energy” with less threatening 

language, for example “energy technology,” opposition by the coal miners to the program 

was mitigated.

Three comparable programs at educational establishments in the Commonwealth of 

Virginia had become inactive. By reviewing those programs, learning from their 

challenges and not repeating their missteps, MECC was able to sustain its program. This 

proposed curriculum was based on model curricula, the subject matter expertise of core 

project partners and instructional resources disseminated by the Advanced Technology 

Environmental and Energy Center, a center of excellence located in Iowa.40

MECC faced significant delays obtaining VCCS approval for the Energy Technology 

Associate Degree because the technologies were considered to be part of a 

new/emerging field. As a result, implementation of the degree program began later than 

anticipated, delaying graduate output.

MECC was unable to spend the entirety of ARC funds in the time allotted. The grant was 

targeted for professional development and construction of the wind turbine, but the grant 

expired before the windmill could be constructed. Thus, only $27,000 of the initial 

$40,000 was spent. ARC funds were used to leverage and match other grant programs 

over various time periods, and the windmill was constructed at a later date using funds 

secured from non-ARC sources. The MECC college administration estimates this 

program would have been delayed for a year or more without ARC funds.

Considering the total program time period, the amount of the grant was relatively minor 

compared to other funding sources. ARC’s grant primarily supported Energy Technology 

curriculum development and the purchase of small-scale instructional equipment. 

Nevertheless, ARC funds were judged to be instrumental to the program. Despite a 

number of extensions, the grant allowed MECC to complete all aspects of the program 

and leverage considerable matching funding. The additional monies from the Virginia 

Community College System, the Virginia Department of Mines, Minerals & Energy, and 

the Virginia Tobacco Commission, were used to significantly expand the scope of the 

Mountain Empire Alternative Energy Initiative by installing a larger-scale grid-tied 

alternative energy system on campus. This system includes the windmill, a solar array, 

and an energy monitoring system, which have been used for instructional purposes and 

have served to offset some of the college's ongoing energy costs. From the inception of 

the program to the present, total program funding exceeds $1 million.

6.4.4 Economic and Community Impacts

Over time, the grant resulted in the development of seven new Energy Technology 

courses. One Career Studies Certificate and one Associate Degree program was created 

and approved by the Virginia Community College System (VCCS). Students can earn an 

Associate of Applied Science with a specialization in Energy Technology, Certificate in 

Energy Technology with an emphasis in HVAC, or Certificate in Energy Technology with 

 

40 http://ateec.org/
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

September 2015 | 95

an emphasis in Electrical. Career paths from certificate to degree programs were created 

and allowed MECC to embed the latest technologies in the curricula.

Although program administrators make every effort to monitor what happens to students 

once they complete the program, they admit it is not always achievable. Student tracking 

shows that upon program completion virtually all students enter or return to the 

workforce. Although there have been only four Energy Technology Associate Degrees 

awarded, most of these students are employed in the region, in the industry or a related 

industry. According to tracking data available from MECC’s PeopleSoft Student 

Information System (SIS), the program served 253 students. The "students served" 

figure includes students enrolled in new Energy Technology courses during the grant 

period (13), as well as high school students participating in outreach efforts for the 

Energy Technology programs during the grant period (approximately 240).

Since inception, 24 students have enrolled in MECC's Energy Technology programs 

(Associate Degree or Certificate), and another 17 students have enrolled in individual 

Energy Technology courses within other degree programs, primarily HVAC & Electricity. 

MECC has awarded four Energy Technology Associate Degrees and 12 Career Studies 

Certificates. Unfortunately, as noted above, implementation of the Associate Degree and 

Certificate was delayed during the VCCS approval process. The Associate Degree 

program takes at least two years to complete, meaning that the program has barely been 

in place long enough to begin graduating students. The college anticipates that the 

number of credentials awarded from these programs will continue to steadily increase. 

All of MECC's Energy Technology Associate Degree graduates are working in the field, 

with all but one graduate working for HVAC employers within the MECC service region.

The program is facing new challenges as the industry and workers evolve. Power 

companies will not hire staff lacking a degree in a related field. Many students who enter 

the program are already employed. More recently the program is attracting students out 

of high school, which will present additional challenges. It is also attracting laid-off coal 

miners who are looking for expanded opportunities in other industries. These may be 

skilled workers, like welders, who are concerned about the future job market in mining 

and are looking to expand their opportunities into other industries and grow their skills.

An additional community benefit of the program is the annual Home Craft Days festival, 

hosted by MECC, which draws tens of thousands of visitors to the campus each fall. 

Since 2012, MECC has been able to power the festival (food and craft vendors, lighting, 

etc.) entirely with energy generated by the windmill and solar panels located on campus. 

Solar and wind energy generated on campus also provides power for parking lot lighting, 

tennis court lighting, and an instructional greenhouse. Furthermore, MECC's alternative 

energy monitoring system indicates that since it began operation in May 2012, more than 

75,668 pounds of CO2 have been saved.

MECC continues to invest in the success of the Energy Technology program. The 

college recently spent its own funds to construct a residential-style demonstration roof, 

adjacent to the existing alternative energy infrastructure, to support the teaching of 

proper installation methods for solar (photovoltaic and thermal) energy systems.

Though the project is trending toward success, a number of challenges were 

encountered during implementation. Because of program delays, the timing of the grant 

did not coincide with construction of the wind turbine. ARC’s grant expired before the 
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

96 | September 2015

windmill could be built. Additionally, time constraints specified in the grant forced the 

program to purchase equipment before the search to hire a program expert was 

completed. Therefore it fell upon existing faculty and energy partners to recommend 

equipment purchases. As a result, some equipment did not meet the goals of the 

program as well as it should, and was appropriated for other purposes.

In order to meet the state’s lowest bidder purchasing criteria, equipment for the program 

was purchased from multiple vendors. However a second conflicting regulation was not 

known to the program administrators. In order to be UL listed41, installed equipment must 

be tested and approved from a single manufacturer as a “set of equipment” or “working 

system.” Of course, this is not possible when purchases are made from multiple vendors 

and thus the equipment purchased could not be installed legally. The educators rightfully 

decided they would not install unapproved equipment on which to train students. This 

decision resulted in duplicate equipment purchases.

Hiring faculty was also a challenge. The program had difficulty finding candidates with 

the proper technical qualifications and therefore did not have a faculty person on board 

until a year into the program. Since grant funds have to be spent within a specified time 

period, hiring delays can result in negative consequences to programs.

Coordinating multiple funding sources, including federal sources, to achieve the program 

goals can be difficult. Each entity may have complicated funding mechanisms combined 

with short completion windows and high expectations for outcomes. As a result, this 

program may choose to only apply for equipment grants in the future, since the 

processes to secure equipment grants are often less burdensome. However, 

administrators do admit that even applying for equipment funding can involve a struggle 

to meet proposal deadlines, get new equipment or modifications in place, and 

demonstrate required outcomes.

In addition to finding the timelines from RFPs to proposal deadlines challenging, this 

grantee recommends a need for greater flexibility in program implementation. They note 

that when the potential grantee already has procedures in place that relate to the 

proposal, implementation is easier. But even when the proposal matches an ongoing 

situation, challenges can still exist, particularly if the grantee needs to secure permits or 

needs to go through an internal and/or external approval process. They point out that 

their institution’s proposal cycles and calendars may not coincide with those of the 

funding agency.

Changes in staff at the funding agency or grantee often lead to programmatic delays, 

especially when there is an approval process underway. Obviously, when someone 

leaves, the organization loses expertise and historical perspective. Most agencies will not 

train a replacement until the person is gone. During this delay, research supporting the 

proposal may become outdated, need to be updated or at worst, repeated. With 

deadlines looming, proposal materials can sit on a bureaucrat’s desk awaiting action for 

too long a time.

 

41 UL is an independent safety science company. UL Listing means UL has tested representative samples of the 

product and determined that it meets UL’s requirements. These requirements are based primarily on UL’s published 

and nationally recognized Standards for Safety. http://ul.com/marks/ul-listing-and-classification-marks/promotionand-advertising-guidelines/specific-guidelines-and-rules/
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

September 2015 | 97

6.4.5 Lessons Learned

The lessons learned from this program draw from interviews of program administrators

and results of the post-grant evaluation survey.

• It is easier to use a grant to add a feature to an existing program than to 

start a new program. Adding a new aspect to an existing program to make it 

more marketable in the workplace, for example, a new skill set, is less 

cumbersome than applying for a new program. However, potential grantees 

recognize that funding agencies often prefer to fund something new.

• Sometimes potential grantees must pass on ARC funding opportunities 

because they do not have an existing project on which to connect the 

grant. Though a program may have a strong desire to secure ARC funding, 

as citizens and taxpayers, grantees only want to undertake projects that will 

be successful and not waste the agency’s funds.

• Faculty should maintain communication with industry experts to 

identify which new skills are needed in the field and then add them to 

the training program. First and foremost, academic training programs must 

teach the basic skills of an industry so their graduates are marketable in the 

job market. However, they should communicate regularly with industry core 

partners and representatives to learn the new skills and knowledge needed 

for students to be successful. This program holds two advisory meetings 

each year with industry experts to identify new competencies to offer in their 

curricula.

6.4.6 Conclusion

The development of the Energy Technology Certificate and Associate Degree program at 

MECC provides new skill development opportunities for regional residents. The impact, 

in terms of number of students served and credentials awarded, is not yet significant 

enough to stabilize or reverse the devastating economic impact of closures in the mining 

industry. The Energy Technology program allows MECC to offer its service region with 

expanded skill development opportunities in emerging fields, and provides the region 

with a real-world demonstration of effective alternative energy use on the campus. In 

2013, the Southwestern Virginia Technology Council recognized MECC for excellence in 

manufacturing and technology education and environmentally friendly operations.42

6.5 Great Allegany Passage Economic Development 

Program

The purpose of this project was to create new business opportunities for residents of the 

Western Maryland counties around the Great Allegheny Passage Trail and the 

Chesapeake and Ohio (C&O) Canal Towpath.

 

42 MECC Recognized for Excellence in Technology Education and Energy Conservation. MECC News, 7/8/2013.
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

98 | September 2015

ARC awarded a grant of $100,000 to The Progress Fund of Greensburg, PA in 2008 for 

this Business/Training project. Local and state funds brought the total funding to 

$200,000.

6.5.1 Community Profile

The Great Allegheny Passage (GAP) is a 132-mile system of biking and hiking trails that 

connects Cumberland, Maryland to McKeesport, Pennsylvania (near Pittsburgh). ARC’s

grant was designed to create new business opportunities for residents of the Western 

Maryland counties around the Great Allegheny Passage Trail (shown in the figure below) 

and the Chesapeake and Ohio (C&O) Canal Towpath.

Figure 27: Great Allegheny Passage Map

Source: Used with permission of Allegheny Trail Alliance.

The Appalachian region is defined as a 205,000-square-mile region that follows the spine 

of the Appalachian Mountains from southern New York to northern Mississippi. 

Appalachian Maryland consists of Allegany, Garrett and Washington Counties, with a 

combined 2012 population of 253,046 residents. Per capita income in 2012 was $37,642 

or about 86 percent of the U.S. per capita income; 70 percent of Maryland and 105 

percent of all of Appalachia. The three counties; Allegany, Garrett and Washington 

realized per capita income of $34,547; $41,083 and $38,489 respectively or 79 percent; 

94 percent and 88 percent of the U.S. per capita average.43

ARC defines the economic status of each county with one of five possible economic 

designations based primarily on three indicators: three year average unemployment rate, 

market income per capita, and poverty rate. The statuses are distressed, at-risk, 

transitional, competitive or attainment. Distressed counties are the most economically 

endangered and attainment counties are the most prosperous. It defines the economic 

 

43 United States Department of Commerce, Bureau of Economic Analysis, Local Area Personal Income, 2012. 

Compiled by the Appalachian Regional Commission, March 2014.
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

September 2015 | 99

status of all three of these counties as transitional, the middle status. Each of the three 

counties in Appalachian Maryland was designated as “transitional” in FY 2012.44

In 2012, residents of Appalachian Maryland experienced an unemployment rate of 8.3 

percent compared to 8.1 percent for the entire U.S. and Appalachian region. It exceeded 

the Maryland state unemployment rate (6.8 percent) by 1.5 percentage points.

The Maryland section of the GAP trail begins in Cumberland and runs through Frostburg, 

among other Maryland cities and towns. Figure 5 shows the section of the trail in 

Maryland from Mile 1 to the border with Pennsylvania, just past Mile 20.

Figure 28: Maryland Section of the Great Allegheny Passage

Source: Used with permission of Allegheny Trail Alliance.

6.5.2 Project Description

This project leverages the existing resources of the work already underway in the 

Pennsylvania section of the Great Allegheny Passage (GAP) Trail to promote economic 

development through expanded business opportunities. ARC awarded a grant of 

$100,000 to The Progress Fund in 2008. State and local funds brought the total program 

support to $200,000. ARC funds directly financed a Maryland Regional Coordinator for 

the Trail Town Program focusing on the communities of Cumberland, Frostburg, and Old 

Town, Maryland. The outlined goals of the program are to promote business 

development, offer technical assistance to existing businesses, create jobs and assist 

the cooperative marketing of the region as a tourist destination. 

A two-year action plan was established as a guidepost for the Maryland Coordinator’s 

scope of work. Elements included organizational and capacity building, community 

planning and advocacy efforts, regional economic development strategies and trail-wide 

programs, and regional marketing and public relations strategies. Building on the Trail 

Town (Pennsylvania) work plan, the Regional Coordinator focused attention on the 

targeted Maryland communities to promote business development, offer technical 

assistance to existing businesses, create jobs and assist cooperative marketing of the 

region as a tourist destination.

 

44 County Economic Status Fiscal Year 2012, Appalachian Regional Commission.
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

100 | September 2015

6.5.3 Project Planning and Implementation

The program, which expands business and tourism opportunities in communities along 

the GAP trail, achieved success in its pilot programs and 2007 implementation in six 

communities in Pennsylvania. With Maryland communities and the Maryland Department 

of Planning expressing interest in the program, it seemed a natural fit to expand into 

Maryland. In 2009, the Maryland Department of Planning organized the expansion of the 

project into Maryland, which included marketing and planning of the trail building. 

However, as the business aspect was missing, The Progress Fund was kept on board to 

maximize the business and tourism development of the communities along the trail.

The initial projected outcomes of the project were to increase business startups and local 

business revenue, increase trail use, and generally improve amenities along the trail. 

Specific goals were to have five to six startups per year, increase trail usage by 10 to 15 

percent per year, and to track and survey businesses and trail users. The cost estimates 

for the project were calculated based on the existing project in Pennsylvania and were 

made up of the cost of personnel, contracts, travel, and facility rental. The Progress Fund 

looked at opportunities to offset some of the expenses through a variety of methods, 

such as marketing through local businesses for a reduced price. They also took into 

account the need for a project coordinator who knew the area well and would be paid full 

time to be working hands on within the communities.

ARC had assisted in funding the Pennsylvania branch of the project, so The Progress 

Fund looked to them again for financial assistance. ARC funding was crucial to the 

Maryland project as most of the other funding used for the Pennsylvania project was not 

available in Maryland. The project's aims and goals are in line with the mission of ARC, 

and the funding was essential for the project to be successful. ARC provided a total of 

$100,000 to the project, and without these funds the project would not have been able to 

be completed on such a large scale or attract as much additional funding. 

While the project can be considered highly successful, there were a few obstacles along 

the way. One such obstacle the project had to overcome was difficulties with the 

electronic trail counting. Due to the unpredictability of nature, there were problems with 

weathering and weatherproofing that led to inaccurate reports from the devices. Over 

time, they developed a box in which to secure the electronic counters to protect them 

from natural forces inhibiting the counting. A system was developed for how often data 

needed to be collected, and this streamlined system was implemented throughout the 

trail. 

The other major obstacle that was successfully overcome was the certification system for 

local businesses. Originally, The Progress Fund created two programs that businesses 

could participate in to increase their business: sustainability certified and trail friendly 

certified. There was, however, too much disconnect and businesses were mostly 

participating in one or the other instead of both. Organizers realized the inefficiency of 

having two programs, and combined the two into one blanket program. In this program, 

businesses could become part of the “Trail Town Certified Network,” which meant they 

were both sustainable and trail friendly. Once the program was reintroduced, it became 

successful and popular. 

The Progress Fund also developed a communication routine with both ARC and 

Maryland Department of Planning in order to keep track of where things were going and 
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

September 2015 | 101

make sure that goals were being met. In addition to monthly phone calls with both 

organizations, they also met with the Maryland Department of Planning in person twice a 

year to help get refocused and establish a strong and trusting relationship. This rigid 

communication schedule ensured that the project stayed on track and that all parties 

were pleased and on the same page with the project’s results and direction.

6.5.4 Economic and Community Impacts

The original goal of the Maryland branch of the Trail Town Program was to build on 

Pennsylvania’s Trail Town work plan, focusing attention on the targeted Maryland 

communities to promote business development, offer technical assistance to existing 

businesses, create jobs and assist the cooperative marketing of the region as a tourist 

destination. The program has undoubtedly achieved success in all of these areas.

There have been trail user surveys, business surveys, economic impact studies, and trail 

utilization and impact surveys completed, as well as yearly reports on trail usage 

patterns. There have also been a multitude of reports detailing future plans, assessments 

of Trail Town needs, as well as reports on related community interests such as public art, 

farmers’ markets, and hotel demand. The rigorous amount of research and reporting has 

provided the organization a great amount of detail on what they have done well, as well 

as areas where they can improve. 

With the success of the Trail Town Certified Network, businesses and trail traffic have 

increased. Nearly all of the original specific goals for the project have been met or 

exceeded. Trail traffic has increased by at least 10 percent every year despite 

technology issues with counting. While there were not always five business startups per 

year, overall there have been more than 11 new business and 40 new jobs created since 

2009. There have also been over 150 jobs and over 40 businesses retained due to the 

increase in trail use.

Businesses involved in the trail network have seen an increase in business, and 

businesses have started working together more to market each other and better 

accommodate visitors. Businesses who participated were rewarded with free marketing, 

as well as the opportunity to connect with other businesses that were involved 

throughout Maryland and Pennsylvania, allowing them to receive tips and pointers for 

improving their business. Participation in the Trail Town Certified Network has more than 

doubled since its transition from two programs to one. 

The implementation of town walk-throughs found areas that were in need of 

improvements and allowed planners to work more effectively. There have been major 

safety improvements, including bike planning and infrastructure, trail safety and signage, 

and street safety. There are now bike racks and pumps available along the trail, and 

businesses are increasingly going green. Kiosks have gone up in every town, including 

electronic kiosks in Frostburg and Cumberland. Overall, the project has met all of its 

goals despite any obstacles and achieved success in unexpected areas.

6.5.5 Lessons Learned

• Maintain regular communication with all invested organizations. The 

Progress Fund organized monthly phone calls with ARC and the Maryland 

Department of Planning (MDP), as well as twice yearly in person meetings 
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

102 | September 2015

with the MDP. This regular communication ensured that there was no 

miscommunication and kept the project on track at all times. There was clear 

communication regarding any obstacles being faced, which built a strong 

relationship of trust amongst all parties. Strong communication allowed the 

project to move forward efficiently and with the approval of all organizations.

• Do not hesitate to alter original plans to better suit the situation. While 

the original plan was to create two certification programs, it soon became 

clear that having two separate programs was creating confusion and few 

businesses were participating in both. The Progress Fund then decided the 

best route would be to rework the original plans and combine the two 

programs into one. This resulted in a dramatic increase in participation and a 

much higher rate of success. 

• Document lessons learned in order to avoid repeating mistakes.

Throughout the development of the project, all of the problems and solutions 

were documented. This ensured that, despite different organizations and 

employees working on any aspect of the project at any given time, there was 

a point of reference to avoid making the same mistakes twice. This saved 

time and effort while providing full transparency between all involved 

organizations.

• Some problems take time to fix; take the time to fix them. While there 

was an array of problems with the electronic trail counting technology, this 

was a crucial aspect to recording and reporting trail usage. Through trial and 

error, and with a lot of time and patience, they eventually developed and 

implemented a system that maximized the effectiveness of the technology. 

Despite the unexpected amount of time and effort that was put into the 

problem, the success achieved was well worth the effort.

6.5.6 Conclusion

The Great Allegheny Passage Economic Development Program’s expansion into 

Maryland was the logical next step after the program’s success in Pennsylvania. With 

support from the Maryland Department of Planning and ARC’s grant, The Progress 

Fund’s Trail Town Program successfully developed and increased business in local 

Maryland trail communities, increased trail traffic, and implemented the Trail Town 

Certified Network which was enthusiastically accepted. While there were some slight 

obstacles with technology and the original certification network, all of these problems 

were successfully overcome with perseverance, communication, and organization 

leading to overwhelmingly positive results.

6.6 Heartwood Southwest Virginia Artisan Gateway

The purpose of the grant was to fund the design, fabrication and installation of state-ofthe-art iconography interior exhibits, media presentations, furnishings and equipment at 

The Heartwood: Southwest Virginia's Artisan Gateway, highlighting the region's arts, 

crafts, mountain music, nature, culture and history.
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

September 2015 | 103

In 2008, ARC awarded a grant of $1 million to The Heartwood in Abingdon, Virginia; 

$500,000 was targeted for building construction and $500,000 to develop cultural 

displays. Local funds doubled the available resources bringing total program support to 

$2,058,140. This project is classified as a Business/Training project.

6.6.1 Community Profile

Southwest Virginia (SWVA) is a mountainous region in the westernmost part of the 

Commonwealth. The region is culturally distinct, known for its Appalachian culture, rich in 

music, craft and natural beauty. The major employment sectors are traditional, extractive 

industries, notably coal mining and timber, in addition to manufacturing and tobacco. The 

region has experienced economic distress and employment loss for decades.

Twenty five counties in Virginia are part of the Appalachian Region, a 205,000-squaremile region that follows the spine of the Appalachian Mountains from southern New York 

to northern Mississippi. Six of the counties include at least one additional city. Under 

Virginia state law, all municipalities incorporated as cities are independent of any county.

The 2012 population of Appalachian Virginia was about 765,000 residents. Per capita 

income in this region was only $32,701 which compared to about 75 percent of national, 

68 percent Virginia and 91 percent of all Appalachia. Per capita income by county ranged 

from 60 percent of the U.S. average in Grayson County to 104 percent in Botetourt 

County.

From 2010 to 2012 the average unemployment rate in Appalachian Virginia was 8.4 

percent; considerably higher than the 6.5 percent rate for Virginia, though just below the 

8.9 percent for both the U.S. and the entire Appalachian region. The Virginia counties 

with the lowest unemployment rates were Bath and Botetourt counties (5.8 percent and 

5.9 percent respectively); the highest rate (13.1 percent) was in Henry County, which 

includes Martinsville city.45

ARC defines the economic status of each county with one of five possible economic 

designations based primarily on three indicators: three year average unemployment rate, 

market income per capita, and poverty rate. The statuses are distressed, at-risk, 

transitional, competitive or attainment. Distressed counties are the most economically 

endangered and attainment counties are the most prosperous. Though no county was 

deemed distressed, the lowest possible designation, 23 of the 25 Appalachian counties 

in the Commonwealth were defined as either at-risk or transitional. That left only two of 

the counties in the region to receive the higher economic statuses of competitive (Bath 

County) or attainment (Botetourt County).

6.6.2 Project Description

In 2008, ARC awarded a grant of $1 million to The Heartwood: Southwest Virginia’s 

Artisan Gateway in Abingdon, Virginia. $500,000 was targeted for building construction 

and $500,000 to develop cultural displays. Local funding doubled the available resources 

to $ 2 million. The funds were part of Heartwood’s initial $17 million program allocation 

 

45 Alleghany County includes Covington city; Carroll County includes Galax city; Henry County includes Martinsville 

city; Montgomery County includes Radford city; Rockbridge County includes Buena Vista city and Lexington city; 

Washington County includes Bristol city; Wise County includes Norton city.
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

104 | September 2015

which came from the Commonwealth of Virginia, ARC, the Virginia Tobacco Commission 

and various localities.

The 29,000 square-foot facility, located off I-81 near Virginia Highlands Community 

College opened in July 2011. The structure, appearing like an abstract barn with a glassand-metal silo entrance, houses a regional artisans’ retail center and six interpretive 

galleries to showcase the talents of area artisans. The facility includes a large 

demonstration/performance space, office space, a cafe featuring locally-grown food, and 

seating for 150 diners. The project was spearheaded by 'Round the Mountain: Southwest 

Virginia's Artisan Network (RTM).

ARC funds were used for the design, fabrication and installation of state-of-the-art 

iconography interior exhibits, media presentations, furnishings and equipment 

highlighting the region's arts, crafts, mountain music, nature, culture and history. The 

exhibits include a large three-dimensional map image of the region with particular 

features highlighted for display, and audio-visual capabilities to see and hear the story of 

Southwest Virginia (SWVA)

6.6.3 Project Planning and Implementation

Heartwood evolved from a vision to reestablish the economic base of the area by 

capitalizing on the region’s unique assets. It would serve as a gateway to the region’s 

rich heritage of music, crafts and natural beauty; a cultural and community resource 

bringing assets and quality of place to attract visitors, retain residents and increase 

human capital.

Tourism was not the planners’ number one objective. Nevertheless, Heartwood Artisan 

Gateway was expected to attract more than 270,000 visitors, generate $2.2 million in 

revenue and generate 300 businesses in its first three years. Heartwood was part of the 

overall vision to develop a quality of life to attract entrepreneurs and high-tech 

businesses to the region.

SWVA shared one particularly troubling economic challenge with other distressed areas: 

a decline in number of college-educated young people – presumably the region’s “best 

and brightest.” An overall goal of the program was to attract white-collar, educated young 

people who would relocate their businesses or move to the area for non-traditional 21st 

century jobs. Most communities experience a relative balance of outgoing college-aged 

students and incoming college-aged residents. The numbers of young educated 

individuals who go to college and don’t return is countered by the number moving into 

the area. However, economically depressed areas discover that more well-educated 

young adults depart than reenter. There is a fear among local parents that their children 

are going to leave for college and not come back. The result is a net loss of collegeeducated young people. The college-educated population represents only 17 percent of 

the total population in this region compared to more than 36 percent of the residents in 

more affluent areas. For the entire U.S. the proportion is 29 percent.46 One yardstick of 

this project was to increase the number of college-educated individuals in the region.

 

46 ARC data and 2012 American Community Survey data.
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

September 2015 | 105

Human capital is seen as an integral part of economic development with the shift to a 

more knowledge-based economy. According to Richard Florida47 and other experts, 

when a community attracts the “creative class” and increases human capital, it sees 

greater economic growth. Those that do not will continue to decline economically.

The revitalization efforts centered around securing new businesses, generating new jobs, 

increasing the number of people employed and attracting tourism to the area. The local 

tax base would increase, positively impacting regional communities, businesses and 

workers.

Development officials preferred to attract year-round residents over short-term visitors. 

Their goal was to increase the number of individuals physically living in localities rather 

than boosting the number of seasonal residents or encouraging tourists to lodge one 

more night at a local bed and breakfast. They intended that individuals buy homes or 

cabins in the area rather than rent cottages for the summer, thereby building 

communities and developing pride in their towns.

One target of this mission was to attract advanced “anywhere businesses” –

establishments that could be located in any community so long as they had access to 

cutting-edge business features such as quality fiber optic connectivity and high speed 

internet. The hope was to attract owners of these types of businesses or entrepreneurs, 

lured by the region’s quality of life, including outdoor recreation (hiking and biking trails, 

rock climbing, and river sports), quality restaurants, cultural amenities, wineries, craft 

breweries and coffee shops. These businesses would advance the economy of the 

region and attract a high-level workforce.

Tourism, though not the primary program objective, was encouraged as a means to 

attract potential residents. Officials targeted what they considered “quality” tourism. For 

example, regionally stereotypical themed recreation was not encouraged and officials 

suggested there would be no public funds for it.

Developing a recognizable consumer brand “Southwest Virginia, Authentic, Distinctive, 

Alive,” was a key part of the overall vision. The catchphrase was modeled after the 

successful branding by West Virginia of “Wild and Wonderful West Virginia.” Heartwood, 

as the area gateway, would bring the concept to life.

Heartwood was the middle part of a multi-phase development program. The Crooked 

Road, linking the region’s musical destinations, may be considered Phase I of the 

program. Heartwood was Phase II, including the $1 million from this ARC grant. Phase III 

involving the ‘Round the Mountain artisan network was supported by a federal grant of 

$815,000, which included $500,000 from the Commerce Department’s Economic 

Development Administration, $215,000 from the United States Department of Agriculture 

and $100,000 from ARC. Combined with support from the Virginia Tobacco Commission, 

the Phase III program had resources of more than $1 million for a set of regional 

initiatives designed to develop Southwest Virginia’s “creative economy.”

 

47 Richard Florida, “The Rise of the Creative Class,” The Washington Monthly, May 2002, pp. 15-25. 

http://ipv4.os3.nl/_media/2009-2010/courses/icp/richard_florida_-_the_rise_of_the_creative_class.pdf
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

106 | September 2015

Resistance to the Project

Some state, regional and sub-regional tourist organizations felt threatened by 

Heartwood, which they perceived as competing for tourist dollars. There was a need to 

convince these entities that they shared common interests and the tourism market could 

support all of them. There was a belief that some local residents subscribed to a 

“mentality of scarcity.” They perceived the economic pie as fixed in size, therefore if 

someone got more of it, someone else must be getting less. Program administrators at 

Heartwood felt compelled to counter this mentality and promote what they considered a 

“mentality of abundance,” stating current revenues were sufficient and would grow over 

time with more than adequate returns for all partners.

There was skepticism among regional tourism organizations that Heartwood, a single 

location tourist site, would serve all 19 counties. Program administrators had to convince 

these local organizations that the program supported tourism throughout the region and 

the program funds would be used to cross-promote artists in music, food, crafts, and 

theater, along with other entities and ad-hoc groups.

Getting tourism organizations to agree with the program goal of encouraging people to 

move to the region was a challenge. Tourism organizations see their primary function as 

promoting tourism, not relocation.

A conflict between two area sub-regions spilled over to obstruct the Heartwood program. 

To the west of the Clinch River, the “coalfields” residents historically felt economically 

neglected, and begrudged their neighbors to the east in the “Blue Ridge” region. Past

efforts at regional cooperation met marginal success. However, the Crooked Road 

project, with Heartwood as its base, was the first to bridge this gap successfully. The 

collaboration persisted, paving the way for other successes, including the “Appalachian

Spring” and “’Round the Mountain” projects. The Heartwood project was the first to show 

that these two regions could address common challenges and opportunities, according to 

project administrators.

Concern was expressed that Heartwood might become an endpoint for visitors rather 

than a launching pad to artist outposts, depriving local artists of sales. Others considered 

the program elitist and too highfalutin for the area. Some thought the Southwest Virginia 

region did not deserve a facility as nice as Heartwood.

Program developers perceived unwillingness among some local government officials to 

support new projects, and in particular Heartwood and related programs. They were 

disinclined to fund projects with long-term payoffs that would not generate short-term 

revenues.

In addition, local governments and residents lacked the vision of planners – they were 

unable to picture their towns looking differently in the future. They could not asses the 

town’s assets and imagine them transposed. City planners had these skills – they could 

picture major changes, envision new hotels, restaurants and stores where currently there 

were only run-down buildings. They could envision relocating the town’s center, if 

required. It took financial evaluations of the project to show regional officials and others 

that growing revenues from the project were keeping property taxes low, something 

officials could get behind.
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

September 2015 | 107

Words can also be a problem. The term “sustainability” was controversial, and was 

avoided in materials and discussions describing Heartwood. To some, policies promoting 

the concept of “sustainability” are seen “as a sinister effort to undermine American values 

and install socialist policies that eliminate private property,” as described in a Southwest 

Virginia Tea Party newsletter in 2011. To avoid controversy, in reports, marketing 

brochures and program descriptions the term “self-reliant” was substituted.

Resistance also came from residents concerned over the level of taxpayer funds 

supporting the project. They also argued that prices for items at the Heartwood venue 

were out of range for local residents.

6.6.4 Economic and Community Impacts

Heartwood visitors may purchase products from regional artisans, with additional 

purchases made via the internet, on-site visitations to artisan studios, craft galleries and 

other regional venues. The facility incorporates space for visitor services where tourists 

can learn about the region's arts, crafts, music, outdoor, historic and other cultural 

assets, such as The Wilderness Road and The Crooked Road. The facility also provides 

assistance on travel arrangements, accommodation, route planning, and other general 

information. Heartwood provides a central location for artists to market their wares and 

reach a broad audience. It also serves as a gateway for exploration of the region. 

It is difficult to disaggregate impacts from the Heartwood facility from impacts resulting 

from preceding and follow-up projects, as programs had multiple funders and grants that 

overlapped and/or operated concurrently. Regionally, revenues from meals and lodging 

taxes increased by an average 60 percent and more than 100 businesses were created. 

The Crooked Road Music Trail, a project preceding Heartwood, has created an 

estimated 445 full-time equivalent jobs and infused $22.8 million in annual revenue to the 

region since 2004.

Heartwood returns 85 cents of every dollar generated through craft, local food, music 

and literature directly back into identifiable people, according to a local radio report aired 

in 2012.

Though not anticipated initially, the Heartwood facility has become a regional meeting 

place. It currently houses the Southwest Virginia Cultural Heritage Commission, ‘Round 

the Mountain: Southwest Virginia Artisan’s Network, The Crooked Road: Virginia’s 

Heritage Music Trail and the city’s tourist department. 

Though visitation has fallen short of anticipated goals, program managers aspire to 

increase tourism to meet program expectations. They emphasize that unlike adjoining 

states, a lack of signage on the interstate highways promoting the region hampers 

tourism. In their efforts to get signs posted, a question arose over which federal or state 

agency retains authority over the placement of highway signage. To date, the answer is 

unclear and signs remains un-posted.

An economic assessment by Friends of Southwest Virginia detailed tourism impact, 

meals and lodging tax revenue, and employment trends for SWVA from around 2004 to 
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

108 | September 2015

2012.48 Commenting on the results, project officials characterized the acceleration in 

growth as “phenomenal.”

Did the Heartwood and related projects move the needle by slowing or reversing the 

regional exodus of young college-educated individuals? The report demonstrates a 

slightly higher increase in 25-34 year olds with a college degree in the region compared 

to neighboring Western North Carolina from 2000 to 2012. The increases were not 

widespread throughout the region, as many localities reported declines. However, the 

report does note communities that experienced growth in residents with college degrees 

are similar to those communities with high travel expenses or that experienced high 

increases in travel expenditures. These areas also saw large increases in local meals 

and lodging tax revenue. The correlation shows that the same assets and quality of place 

that attracts visitors and tourism to certain communities in Southwest Virginia also 

attracts and retains residents and increases human capital. Furthermore, the authors 

argue that the quality of life in Southwest Virginia has played a role in attracting 

“anywhere businesses” to the region and allowed them to attract talented employees.

Heartwood’s administrators assert that the program brought multiple groups together in 

cooperation – a collaboration of theaters, farmers, crafts persons, musicians and others 

– who formerly saw themselves as competitors. Heartwood sees itself as a facilitative 

entity to bring about collaboration of stakeholders, fulfilling an important program goal. It 

is spurring job creation and new businesses while at the same time boosting local 

revenues.

6.6.5 Lessons Learned

Lessons learned were drawn from interviews with program administrators.

• Companies with little history in the region may relocate without notice.

A high-profile company responsible for creating hundreds of local jobs 

suddenly relocated its call-center overseas in response to lower workforce 

costs. Costs incurred by the locality to make infrastructure changes to 

accommodate its workforce were squandered.

• Issues of program identity must be addressed. The marketing campaign 

to support the program was misconstrued. Anecdotal information showed 

sizable numbers of people were aware of the Crooked Road music trail but 

believed incorrectly that it was located in West Virginia or Tennessee. 

• Regional economic development and tourism officials must understand 

that the program brand belongs to them, not the grantee. Though the 

program may be supported by state and regional entities, it is important that 

local officers appreciate it is not imposed on the region by the state, but 

involves local planning, local development, and local tourism organizations.

• The concept that music, crafts and outdoor recreation are part of a 

three legged stool and should be partners in tourism promotion is 

growing among tourism administrators.

 

48 Southwest Virginia, Authentic, Distinctive, Alive: SWVA Economic Analysis Report, Friends of Southwest Virginia, 

circa 2012. http://www.cppdc.com/Reports/SWVA%20Economic%20Analysis%20Report.pdf
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

September 2015 | 109

• Program benefits often take more time to generate than expected.

Despite the best intensions, some short-term outcomes do not pay off by the 

end of the grant period, but do become long-term successes. Granting 

agencies and evaluators should have faith in the risk, be flexible and have 

patience – and realize some programs take time to work.

• Business creation that yields multiple smaller entities may, in the long 

run, be a better investment and entail less risk than funding one large 

business that may or may be sustainable.

• Traditional measures of success will not apply to all programs.

Occasionally, more qualitative and/or long-term outcomes are better 

indicators of achievement. For example, a high number of jobs created may 

or may not be a good thing. Are multiple small enterprises better for the 

community than one large business that employs many workers at relatively 

low wages?

• The goals of the community are important and may not coincide with 

outcomes measured by program auditors. The community may want to 

stop the “brain drain” in their region, though that is not necessarily what 

auditors understand or investigate.

• ARC surveys need to be delivered to the correct individual. Sometimes 

the contact person, such as the Program Director, or Construction Manager 

cannot speak to the overall issues regarding the project.

6.6.6 Conclusion

Officials note that without ARC funding for Heartwood and its preceding phase, the 

programs would not enjoy today’s successes. Heartwood, they note with particular pride, 

laid the groundwork for subsequent initiatives including the Ralph Stanley Museum, 

dedicated to the local bluegrass musician and legend. ARC funding has been leveraged 

into $40 to $50 million in additional regional investment.49

Heartwood demonstrated that asset-based development could work by initiating a selfreliant program in a rural area suffering economic and physical distress but blessed with 

cultural assets and abundant natural resources. Heartwood serves as a blossoming 

tourist destination for heritage tourism and a positive force for the area.

Heartwood shows that a facility can serve almost 20 counties with a multitude of highly 

motivated collaborative partners. It boosted tourism, generates pride in the community, 

and serves as a model for other communities and states. To date, Heartwood and the 

related the series of projects have succeeded in creating hundreds of businesses, 

several thousand jobs and boosted local revenues.

 

49 Personal communication with Todd Christensen, Executive Director of the Southwest Virginia Cultural Heritage 

Foundation.
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

110 | September 2015

6.7 MARC Custom Medical Products, Western North 

Carolina

The Marketing Association for Rehabilitation Centers (MARC) is a 501(c)(3) nonprofit 

corporation whose purpose is to serve people with disabilities throughout a 21-county 

region of Western North Carolina. MARC serves by developing and implementing 

collaborative programs among its 14 member organizations located throughout this 

service area. Their services include food processing, commercial sewing, plastic 

thermoforming, embroidery services, form fill and seal, medical products, packaging 

services and product assembly. MARC was founded in 1978.

MARC’s Custom Medical Products (MCMP) program is an innovative rural development 

strategy deployed by MARC in July 2006. MCMP joins six MARC member Community 

Rehabilitation Programs in a partnership to manufacture custom disposable surgical 

drapes and other products used in the healthcare industry. The key objective for this 

venture is to create manufacturing jobs with a majority targeted for people with 

disabilities.

ARC has provided grant support to the MARC and specifically MCMP since 2006. The 

MARC grants were categorized as Business/Training. ARC has provided two grants 

totaling $400,000 ($200,000 each in FY 2007 and FY 2009) which were leveraged to 

provide an additional $2.6 million in grant support and direct “first money in” from MARC 

member organizations. The initial grant funding was matched by direct and indirect 

member investment of $838,000 over the duration of the project. Total funding was 

expected to approach $4 million from other sources, with the additional monies restricted 

to capital equipment and renovations.

Figure 29: MARC Surgical Drape Manufacturing

Source: Used with permission of MARC, Inc.

MARC serves as a catalyst for innovations that improve the economic and social quality 

of life for people with disabilities and disadvantages. MARC believes that all individuals 

have the right to be respected, to have equal opportunities, to work and to earn a fair 

wage. MARC's Board of Directors and staff support the concept that through meaningful, 

satisfying work, people are able to live more independently, successfully and 

productively.
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

September 2015 | 111

6.7.1 Community Profile

MARC includes 14 member organizations that collaborate in program planning, 

fundraising, implementation and review to achieve their joint goals. Twenty-nine counties 

in Western North Carolina are part of the Appalachian Region, a 205,000-square-mile 

region that follows the spine of the Appalachian Mountains from southern New York to 

northern Mississippi. All 21 counties served by MARC member organizations are part of 

the Appalachian Region. Table 34 lists the MARC programs and the counties they serve.

Table 11: MARC Members and North Carolina Counties Served

MARC Program Member Western North Carolina Counties Served

Ashe County ADAP Ashe

Caldwell Opportunities, Inc. Caldwell

Foothills Industries of McDowell County McDowell

Goodwill of Northwest North Carolina Buncombe; Madison

Haywood Vocational Opportunities, Inc. Haywood

Industrial Opportunities, Inc. Cherokee; Clay; Graham

LifeSpan Services, Inc. Wilkes

Polk Vocational Services, Inc. Polk

Rutherford Life Services, Inc. Rutherford

Transylvania Vocational Services, Inc. Transylvania

Vocational Opportunities of Cherokee, Inc. Jackson; Swain

Vocational Solutions of Henderson County Henderson

Watauga Opportunities Ashe; Avery; Mitchell; Watauga; Wilkes; Yancey

Webster Enterprises, Inc. Jackson; Macon; Swain

Source: MARC, Inc. and ARC.

Appalachian North Carolina is one of the poorest regions in North Carolina, in 

Appalachia and in the U.S. The 2012 population of the region was about 1.7 million 

residents who realized per capita income of only $34,215, which was about 78 percent of 

national, 90 percent of North Carolina, and 95 percent of all Appalachia per capita 

income.50 Per capita income ranged from around $27,000 in Graham County (61.7 

percent of U.S. average) to around $40,000 in Polk County (92.0 percent of U.S. 

average). Per capita market income, a measurement of income that excludes transfer 

 

50 United States Department of Commerce, Bureau of Economic Analysis, Local Area Personal Income, 2012. 

Compiled by the Appalachian Regional Commission, March 2014.
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

112 | September 2015

payments, was lowest in Cherokee County (around $16,000 or 45 percent of U.S. 

average) and highest in Forsyth County (around $32,000 or 87 percent of U.S. average).

The percent of the population that represents that they suffer from a disability ranges 

from nine to 22 percent and the MARC-county average of 16 percent is three percentage 

points higher than the state as a whole.51

ARC defines the economic status of all of these counties as at-risk, distressed or 

transitional. No counties in Appalachian North Carolina received the higher economic 

status of competitive or attainment.52

6.7.2 Project Description

MARC’s Custom Medical Products (MCMP) puts together and bundles medical products 

using customized work stations to optimize work flow in a controlled manufacturing 

environment. Among its accreditations and certifications are United States Department of 

Agriculture Food Packaging Plant certification, Total Quality System Audit (TQSA), ISO 

9001 (Manufacturers and Designers), ISO 13485 (Medical Products Manufacturers)53

and regulation and audits by the Food and Drug Administration. MCMP specializes in 

medical grade sewing and thermoformed products.

The 14 community rehabilitation programs in MARC’s consortium providing vocational 

rehabilitation, developmental services, and employment for people with disabilities, 

disadvantages and other barriers to employment have been Western North Carolina’s 

primary source of these services since 1978. The top objective of the programs is to 

bring an improved quality of life through work experiences. The program is designed to 

create more opportunities for individuals with disabilities and their caregivers while 

reducing unemployment and underemployment and dramatically improving their quality 

of life.

Figure 30: Automated Fabric Cutter in Use at MARC Custom Medical Products

Source: Used with permission of MARC, Inc.

The programs reflect the local culture, resources and distinctiveness of their 

communities, while structured along common program formats and adhering to the same 

state and federal guidelines. The most common services and programs among members 

 

51 United States Census Bureau, 2010-2012 American Community Survey.

52 County Economic Status Fiscal Year 2012, Appalachian Regional Commission.

53 ISO is an independent, non-governmental international organization with a membership of 162 national standards 

bodies. Through its members, it brings together experts to share knowledge and develop voluntary, consensusbased, market relevant International Standards that support innovation and provide solutions to global challenges. 

http://www.iso.org/iso/home.htm
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

September 2015 | 113

are Vocational Rehabilitation Services supported by the North Carolina Division of 

Vocational Rehabilitation and Adult Developmental Vocational Program services 

provided through the support of the North Carolina Division of Mental Health, 

Developmental Disabilities and Substance Abuse Services. Other services and programs 

include Child Day Care services to offender community re-entry programs.

A contemporaneous article in Today’s Medical Developments reported the program was 

designed to create or retain 250 jobs, primarily for people with disabilities in the 17 

westernmost counties of North Carolina. It was also intended as a base for rural 

entrepreneurial development through manufacturing in small towns located in those 

counties.

6.7.3 Project Planning and Implementation

One of the 14 MARC programs, Haywood Vocational Opportunities, Inc., operated a 

medical supply manufacturing business prior to 2005 and was experiencing capacity 

constraints. At a meeting of all MARC programs, the potential for expanding the medical 

supply business was discussed. Eleven members indicated interest in forwarding a 

collaborative effort in medical supply manufacturing and eventually the MCMP was 

formed in July 2005. Today, six of the 11 original members are part of the medical 

products manufacturing consortium, operating six facilities across Appalachian North 

Carolina.

The MCMP members commissioned a strategic marketing study to determine if the 

market for these products would continue to grow and if they would have an opportunity 

to serve this market. The team developed a business plan and proceeded to seek 

funding. Facility development and cost estimates were provided by their existing medical 

supply partner, Haywood Vocational Opportunities. Initial funding came from cash and inkind support provided by MCMP members. The full funding plan is shown in Table 12.
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

114 | September 2015

Table 12: MARC Custom Medical Products Funding Table, July 2006 – February 

2011

Funding Source Status Amount

MCMP Member Dues (Cash) Dues Paid $305,000

ARC MCMP Support Awarded-Completed $200,000

ARC Quality & Business Development Grant (MQ&B) Award in Process $200,000

Advantage West, EDI Grants Awarded-Completed

NC Rural Economic Development, R&D Grant Awarded-Completed

NC Rural Economic Development, ESI Grant Award in Progress $400,000

Golden LEAF Foundation Awarded-Completed $605,000

Golden LEAF Foundation, Economic Capacity Grant Awarded-Completed $250,000

Janirve Foundation Awarded-Completed $300,000

NC Department of Commerce, Economic Development Awarded-Completed $487,000

Z. Smith Reynolds MQ&B Support Awarded-Completed

Z. Smith Reynolds MARC Parity Initiative Award in Process $137,000

Total Cash from Funding Partners & MCMP Members $3,013,000

Total In-Kind from MCMP Members $1,552,144

Total Project 5-Year Investment of MARC & Funding $4,565,144

Source: MARC, Inc.

The MCPM program partners initiated the program in 2005 with an implementation plan 

and business plan that they expected to complete in three years. Cash dues and in-kind 

services from the partner organizations allowed them to begin fundraising and 

implementation. As one of their partners already operated manufacturing facilities in this 

industry, the implementation proceeded much like an expansion of an existing enterprise 

rather than a new enterprise. MCMP hired a president in 2006 to oversee the 

implementation process. The membership conducts monthly meetings to assess 

progress and plan future activities.

The initial membership in MCMP included 11 rehabilitation program organizations. Some 

determined that this project was not in their best interest for financial, staffing or other 

reasons and eventually the final group consisted of six organizations. Each opened 

facilities in their respective counties over the next three years. While each organization is 

independent, the collaborative nature of their process allowed each organization to 

benefit from the experience of the others.

The MCMP hired an outside consultant to assist with market research and business 

planning. While their collaborative social enterprise was different from most organizations 

in this industry, they needed to be able to compete on price and quality if they hoped to 
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

September 2015 | 115

retain customers. The fact that Haywood Vocational Opportunities had already 

developed a client base and understood the markets gave the five new producers the 

expertise they needed to compete. Initially, they operated on subcontracts with Haywood 

Vocational Opportunities, but eventually earned their own contracts.

Providing products in the medical supply industry required a focus on quality control. All 

products were required to meet high industry standards and clients needed assurance 

that such standards were met. Thus, the MCMP initiated a Quality Control Management 

System which has led to the issuance of nine ISO 9001 and 13485 Registrations. The 

former is the most widely used Quality Management System standard in the world, 

supported by eight universal management principles; the latter are the international 

Quality Management Systems for the design and manufacture of medical devices. 

MARC maintains a quality assurance audit system with certified auditors conducting 

independent audits of all administrative and manufacturing operations.

6.7.4 Economic and Community Impacts

The execution of their plan began to see results as staff were trained and facilities came 

on line. In 2005, the enterprise had about 750 employees and commercial sales of $36.8 

million. By FY 2014, the MCMP had added 282 new jobs and retained another 157 jobs. 

Revenue was now $58.7 million. Commercial sales of its products and services were 

used to generate excess funds for its member nonprofit organizations. The excess funds 

provided development programs for its client population and sustain program 

administration and management functions.

In March of 2009, the College of Business at Western Carolina University (WCU) 

conducted an economic impact study for the MARC enterprises.54 The WCU researchers 

evaluated 2008 MARC financial data, surveyed MARC members and surveyed MARC 

employees. This research estimated that:

• In 2008, MARC had an economic impact of $60.7 million on the 21 counties it 

serves

o This included $45.8 million direct impacts

o $7.5 million indirect impacts

o $7.4 million induced impacts

• The annual economic impact per job created was $58,814

• In 2008, jobs created by MARC had an average hourly wage of $10.39 

($11.46 in 2014 dollars)

• In the MARC region, there was a 50.1 percent higher occurrence of 

employment for persons with a disability than the national average

The MARC has achieved operating profits of about $2 million in each of the past five 

years. These funds are all reinvested in MARC programs in the 21-county service region.

 

54 Inhyuck Ha, Ph.D., Jessica L. Hollars, Economic Impact of the MARC on Western North Carolina’s Economy, 

Western Carolina University.
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

116 | September 2015

6.7.5 Lessons Learned

MARC is a successful collaborative social enterprise that has provided employment 

opportunities in the economically challenged region of Western North Carolina for 

disabled, disadvantaged and other workers. MCMP is an extension of the medical supply 

manufacturing enterprise operated by one member of MARC to five other members. 

Some of the lessons learned from the expansion include:

• Collaborative organizations provide support for expanded enterprises.

One member of MARC operated a successful medical supply manufacturing 

enterprise. This organization brought the opportunity to other members to 

expand this enterprise in their own regions. The existing market knowledge 

and technical experience allowed other collaborative partners to overcome 

traditional barriers to entry and start medical manufacturing operations in 

their regions.

• Develop a collaborative culture. The MARC collaborative approach 

extends to almost forty years of carefully negotiated agreements to achieve a 

positive outcome for all parties. MARC has worked closely with their 

counties, municipalities, local development districts, and state labor 

department to bring all ideas and perspectives to the table and distill them 

into a plan on which all stakeholders could agree. This collaborative 

approach is continuing as the project expands opportunities and creates jobs.

• Use of outside consultants assisted in developing a sound business 

plan. The MARC is a social enterprise managed mostly by professionals 

experienced in extending vocational and social support to segments of the 

population who have physical and socioeconomic challenges. It was a critical 

step to seek support from business professionals with experience in the 

medical manufacturing business to create a business plan that they could 

manage and measure progress against.

• Both enterprise and vocational training program benefits can be 

realized through a single training program. A critical service provided by 

the MARC members to its communities is vocational training. Operating 

medical manufacturing facilities allowed the training to evolve from classroom 

to on-the-job training to employment.

• Cost control and quality assurance are fundamental to the enterprise.

The cost structure of MCMP facilities can be higher than competitors due to 

the nature of its workforce. They must control cost and assure quality 

simultaneously. Another ARC grant is assisting MCMP members to receive 

and maintain ISO registrations.

6.7.6 Conclusion

The Marketing Association for Rehabilitation Centers (MARC) is a 501(c)(3) nonprofit 

corporation whose purpose is to serve people with disabilities throughout a 21-county 

region of Western North Carolina. MARC’s Custom Medical Products (MCMP) program 

is an innovative rural development strategy deployed by MARC in July 2006. MCMP is a 

collaborative social enterprise where six MARC member Community Rehabilitation 
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

September 2015 | 117

Programs manufacture custom disposable surgical drapes and other products used in 

the healthcare industry. The key objective for this venture is to create manufacturing jobs 

with a majority targeted for people with disabilities.

ARC has provided grant support to the MARC and specifically MCMP since 2006. The 

Commission has provided two grants totaling $400,000 which were leveraged to provide 

an additional $2,610,000 in grant support and direct “first money in” from MARC member 

organizations. This support contributed to the development of a training program for 

people with disabilities and the promise of permanent, rewarding employment.

6.8 Strengthening Appalachia’s Farmers’ Markets

The Ohio State University (OSU) South Centers worked with farmers’ markets managers 

to develop customized marketing strategies and tactics. Through one-on-one technical 

assistance, the program assisted Appalachian Ohio farmers’ market managers and their 

vendors/producers implement marketing best practices, create a regional identity, 

develop marketing plans and sustain their markets. Positive economic and social impacts 

were realized.

6.8.1 Community Profile

Thirty-two counties in Ohio are part of the Appalachian Region, a 205,000-square-mile 

region that follows the spine of the Appalachian Mountains from southern New York to 

northern Mississippi. 

In 2012, the more than 2 million residents of these 32 counties of Appalachian Ohio 

realized per capita income of only $32,893, or about 75 percent of national, 82 percent of 

Ohio, and 92 percent of all Appalachia in per capita income. Per capita income by 

county, compared to the U.S. average, ranged from 51 percent of the national average in 

Noble County to 88 percent in Clermont County.55

ARC defines the economic status of each county with one of five possible economic 

designations based primarily on three indicators: three year average unemployment rate, 

market income per capita, and poverty rate. The statuses are distressed, at-risk, 

transitional, competitive or attainment. Distressed counties are the most economically 

endangered and attainment counties are the most prosperous. In 2012, the 32 counties 

of Appalachian Ohio received ratings spanning the three lower designations: at-risk (nine 

counties), distressed (seven counties) or transitional (16 counties). None of the state’s 

Appalachian counties received either of the higher economic statuses of competitive or 

attainment.56

Between 2008 and 2012, roughly 338,000 individuals lived below the poverty level in the 

counties of Appalachian Ohio, experiencing a poverty rate of 17.1 percent, compared to 

14.9 percent for the U.S. (115 percent).57 In 2012, their 8.3 percent unemployment rate 

 

55 United States Department of Commerce, Bureau of Economic Analysis, Local Area Personal Income, 2012. 

Compiled by the Appalachian Regional Commission, March 2014.

56 County Economic Status Fiscal Year 2012, Appalachian Regional Commission.

57 Poverty Rates, 2008-2012, Appalachian Regional Commission.
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

118 | September 2015

was slightly above the U.S. average of 8.1 percent, exceeding the overall Ohio rate (7.2 

percent) by more than a percentage point.58

6.8.2 Project Description

The focus of the project was to provide resources, education, training and technical 

assistance to farmers’ markets managers, vendors and producers in Appalachian Ohio 

thereby increasing market share, sustainability and jobs. It also benefited consumers by 

providing greater access to healthy locally produced foods.

ARC awarded a grant of $75,000 to OSU South Centers for the project. Additional local 

sources brought the total project funding to $150,157. This project was categorized in the 

Local Food project type.

6.8.3 Project Planning and Implementation

By 2008, interest in farmers’ markets began to take off and questions from potential and 

recent businesses clients to the Ohio State University South Centers, part if its small 

business development network, escalated. Inquiries included how to start a farmers’ 

market, how to run a successful market, how to insure product safety, how to comply 

with state regulations on labeling and what were the best practices in designing 

marketing labels.

The markets were being run primarily by the farmers as direct outlets to consumers. This 

presented an opportunity for producers to sell directly at retail and earn full-fare 

consumer dollars, compared to the more traditional, though less-profitable paradigm, of 

selling to wholesalers and middlemen at a discount.

In addition, farmers’ markets were also managed by Main Street Programs, economic 

development organizations, health departments and others. Compared to those 

managed by farmers, these farmers’ markets enjoyed improved stability as a result of 

paid staff.

An issue most critical to the farmers’ markets was how to get the word out that the 

markets existed. This was particularly important for new operations. It was understood 

that the first few months of operation were decisive for success. If a farmers’ market did 

not attract enough vendors to supply it with product, consumers would stop shopping 

there. And if the market was unable to draw an ample supply of customers, the vendors 

would dry up. The program was designed to address this very real “Catch-22” situation; 

one that easily could result in the failure of the market.

Clients were identified through county extension programs, economic development 

agencies, local Chambers of Commerce, Main Street Programs and word-of-mouth.

Technical assistance included development of a marketing toolkit to help identify items in 

high demand, tactics to maximize profits by product placement, and how to implement 

effective promotional strategies. Face-to-face meetings, workshops and group training 

sessions were held with market managers and vendors to assist with implementation of 

best practices, marketing plans and additional areas of business management. 

Producers needed new marketing strategies, as some transitioned from tobacco or 

 

58 Unemployment Rates, 2012, Appalachian Regional Commission.
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

September 2015 | 119

commodity production, where product was sold to regional auction houses or grain 

elevators, to direct marketing to the end consumer. Customizable media materials 

included rack cards, press releases, flyers and bookmarks. The market manager could 

easily customize the materials by gathering local demographic information. Printing costs 

were assumed by the farmers’ market. Events calendars and a location map were also 

generated.

This project could not have been completed without ARC funding, though it was 

supplemented with funds from a United States Department of Agriculture Agricultural 

Marketing Service (USDA/AMS) farmer’s market promotional grant. ARC funds were 

used for staff time (for training, technical assistance and design of marketing materials) 

and to develop and print 3,000 copies of the popular map showing the locations of all 

farmers’ markets in the Appalachian Ohio (Figure 8).

Figure 31: Map of Locations of Appalachian Ohio Farmers’ Markets

Source: Used with permission of The Ohio State University South Centers

6.8.4 Economic and Community Impacts

The project assisted Appalachian Ohio farmers’ markets, local food producers and the 

public by leveraging its expertise and staff to meet program goals. It met its overall

mission to support small business development in the state and improve the lives of Ohio 

citizens by assisting in their education and economic conditions. The program afforded 

the opportunity to benefit farmers’ markets in distressed economic areas, an emerging 

group of clients for the organization.

Outputs and suggested improvements to the program are based on actual responses to 

a survey of participants and observations of the project team compared to defined 

program goals.
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

120 | September 2015

Twenty farmers’ markets in 20 communities were served, receiving 460 hours of 

technical assistance. Almost three-quarters (72 percent) of market managers indicated 

the resources provided were effective for their farmers’ market; 75 percent 

acknowledged increases in their knowledge of marketing; and 56 percent indicated they 

have implemented best practices learned in the course of the project.

The program improved 10 communities and 160 businesses. A total of 105 agricultural 

jobs were created or retained; 30 jobs were created, exceeding expectations; 75 jobs 

were retained. Six businesses were created and 160 retained.

Leveraged investment in excess of $70,500 topped expectations. Revenues, as a result 

of the program, were up $18,600; costs were reduced by $10,000.59 OSU South 

Centers was able to secure additional funding from the USDA/AMS Farmers’ Market 

Promotion Program in 2012 to continue to work with Ohio farmers’ markets, vendors and 

producers.

The communities where farmers’ markets are located realized additional non-economic 

benefits. Farmers’ markets provide a safe and non-threatening area where all members 

of a community can come together and interact. This helps build social capital for the 

communities, according to research at the national level. There is also evidence that 

farmers’ markets assist individuals with improvement of their diets (i.e. the consumption 

of more fresh fruits and vegetables) also benefitting communities.

6.8.5 Lessons Learned

The lessons learned were drawn from interviews of program administrators and survey 

responses from a post-program questionnaire conducted by an ARC contractor.

• Focus is needed on the incubator aspects for local food producers who 

direct market their products through farmers’ markets. Information and 

technical assistance on the resources available to take their business to the 

next level should be provided to this audience.

• A program focused on serving the individual producers in best 

business practices is recommended by the program administrator. This 

would benefit not only producers, but also the farmers’ markets in which they 

participate. The communities in which the markets are located and where the 

producers reside, and the consumers who purchase their fresh and healthy 

local foods would also gain.

• Farmers’ markets are mostly staffed by volunteers who may not have 

the time to complete the formal marketing plans, but nonetheless can 

benefit from the technical and marketing assistance. Many of the 

markets made strides in adopting new best practices despite not completing 

formal marketing plans.

 

59 To determine revenues increased, the number of farmers’ markets (10) times the average sales/market day/market 

($500) times the number of market days (24) times the reported percent increase in sales (15.5%) equals the 

revenues increased. Ten (10) farmers’ markets X $500 average sales/market X 24 market days = 120,000 X 15.5% 

reported increase = $18,600. Reduced costs or costs avoided were based on $1,000 to develop a custom 

marketing plan that the markets did not have to spend as a result of the project.
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

September 2015 | 121

• Farmers who manage farmers’ markets wear many hats, often working 

full-time in addition to their involvement in the markets. Getting farmers 

to set aside time for workshops, technical assistance or one-on-one sessions 

is problematic due to the many other roles they play. One way to secure time 

with these farmers’ market managers is to schedule sessions during the off 

season, though this will not completely resolve this concern.

• Not all managers can be counted upon to share information and 

materials with their vendors/producers. Future iterations of this or similar 

projects should consider contacting vendors directly, in addition to the market 

managers, or conducting presentations at the farmers’ markets annual 

meetings where the managers and vendors are present.

• Programs need to solicit and anticipate client needs. It is important to ask 

clients their needs in terms of content, though they may not know what they 

don’t know. One way to encourage client input is to remind them that the 

program is sponsored by their tax dollars and so they should want a say in 

how their monies are spent. 

• A less labor intensive mechanism is needed to update the printed map 

of farmers’ markets. Technology should be incorporated to collect updated 

information on farmers’ markets locations, product offerings and contact 

information.

6.8.6 Conclusion

This was the right program at the right time. The program was initiated when consumer 

and supplier interest in farmers’ markets was trending. The need to assist Appalachian 

Ohio’s farmers’ markets and their vendors/producers was great. The program increased 

the financial sustainability of these businesses, proved to be an incubator for local food 

production, allowed local producers to grow their small businesses, generated positive 

economic impacts and created jobs.

6.9 Finger Lakes ReUse Center Start-Up

The purpose of the project was to launch the Finger Lakes ReUse Center to create jobs 

in the deconstruction industry by disassembling housing, diverting regional assets away 

from local landfills, and putting regional assets such as architectural elements, building 

materials, and furniture back into productive reuse.

6.9.1 Community Profile

Fourteen counties in New York are part of the Appalachian Region, a 205,000-squaremile region that follows the spine of the Appalachian Mountains from southern New York 

to northern Mississippi.

In 2012, the more than 1 million residents of these 14 Appalachian New York counties 

realized per capita income of only $36,749 which was about 84 percent of national, 69

percent of New York, and 100 percent of all Appalachia per capita income. Per capita 
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

122 | September 2015

income by county, as a percent of the U.S. average, ranged from 69 percent in Allegany 

County to almost 89 percent in Tioga and Tomkins Cunties.60

The four Empire State counties primarily served by the program are located in the 

Southern Tier East Region of Appalachian New York. Three of four counties (Broome, 

Tioga and Tompkins), realized around 88 percent of the U.S. average in per capita 

income. Cortland County realized only 79 percent of U.S. average per capita income.

ARC rates each county one of five possible economic designations – distressed, at-risk, 

transitional, competitive or attainment - with distressed counties being the most 

economically endangered and attainment counties being the most prosperous. ARC 

defined the economic status in 2012 of all fourteen Appalachian New York counties as 

transitional – the middle rating.61

The Finger Lakes ReUse Center is located in the city of Ithaca, in Tompkins County. 

Ithaca is located approximately 225 miles northwest of New York City in the Finger Lakes 

region of New York. Ithaca experienced a slightly higher than state average population 

growth rate of 1.7 percent growth from 2010 to 2013 and is now home to approximately 

30,000 people. From 2009 to 2013, 46 percent of individuals in Ithaca were living below 

the poverty level compared to 15 percent for all of New York State.62

6.9.2 Project Description

In 2008, ARC awarded $148,463 to Cornell Cooperative Extension of Tompkins County, 

a 501(c)(3) non-profit educational organization. It is part of the Cooperative Extension 

land grant system, a partnership between county, state and federal governments that is 

administrated in New York State through Cornell University. Local and state funding of

$151,981 brought the total program support to $300,444.

The award created Finger Lakes ReUse, Inc., a non-profit 501(c)(3) organization to 

operate a ReUse Center and various programs to create jobs in the deconstruction 

industry by disassembling housing, diverting regional assets away from local landfills, 

and putting regional assets such as architectural elements, building materials, and 

furniture back into productive reuse.

Funding for the Center came from a variety of sources. ARC and matching funds each 

shared in the cost of funding a number of personnel for the first year of operation. 

Remaining ARC funds were put towards travel, supplies, and a majority of the equipment 

needed to initiate the operation. Future income to partially offset operating costs would 

be derived from sale of architectural elements salvaged from housing deconstruction 

performed by the Center, similar to other operations across the country. Operational 

income and continued support from the Solid Waste Division of Tompkins County Public 

Works would fund future years. 

The first deconstruction project undertaken by the ReUse Center was a pilot project to 

remove a wood-frame cottage on the Ithaca College campus. The building was taken 

 

60 United States Department of Commerce, Bureau of Economic Analysis, Local Area Personal Income, 2012. 

Compiled by the Appalachian Regional Commission, March 2014.

61 County Economic Status Fiscal Year 2014, Appalachian Regional Commission.

62 State and County Quickfacts, United States Census Bureau, 2015.
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

September 2015 | 123

apart by hand in five days. Fourteen of the 17 tons of materials were saved and either 

recycled or resold.

6.9.3 Project Planning and Implementation

Planning for the project began in December 2005 and Finger Lakes ReUse, Inc., was 

incorporated in 2007 as a 501(c)(3) nonprofit organization. The planning organizations 

included Tompkins County Solid Waste Management Division, Significant Elements63, a 

non-profit architectural salvage warehouse Cooperative Extension64, linking the research 

and extension efforts at Cornell University, the Cornell University Agricultural Experiment 

Station and the New York State Agricultural Experiment Station, provides the knowledge 

to maximize New York State’s agricultural and natural resources. Planning proceeded 

from 2006 to 2008 including but not limited to an initial phase of extensive data collection 

by Tompkins County Chamber of Commerce and TCAD (Tompkins County Area 

Development), and with Best-practices adoption. A summary report65 specified a suitably 

sized facility, accessible location, extended hours of operation, and proactive efforts to 

pick up and deliver materials would ensure maximum diversion and help provide a 

sustainable and community-friendly reuse operation.

Additional surveys were conducted during the planning phase to identify potential users, 

learn from other programs around the country and broaden the knowledge base of the 

staff. These included:

1. Survey of Selected U.S. Reuse Programs conducted by Kat McCarthy and Diane 

Cohen

2. Local Tipping Floor (RSWC) Observations conducted by Kat McCarthy and Sean 

Williamson

3. Locally Active Reusers Survey (local non-profits and individuals active in reuse) 

conducted by Diane Cohen and Jeff Klein

4. Generators Survey (Local Small Businesses) conducted by TC Chamber of 

Commerce

5. Generators Survey (Local Retailers) conducted by Rob Young, Cornell Cooperative 

Extension of TC, and Diane Cohen

6. Generators Survey (Local Manufacturers) conducted by TCAD

7. Educational Institutions Survey conducted by Diane Cohen

8. Collegetown Consumers and Generators Survey conducted by CU Certificate of 

Business Management students

In 2007, the program planners initiated a study of existing social and educational 

programs to identify potential partnerships. A five-year business plan was developed.

Finger Lakes ReUse embraced a Triple Bottom Line Mission to enhance the community, 

economy and environment through reuse of salvaged furniture, housewares and building 

 

63 www.significantelements.com

64 http://cce.cornell.edu/

65 A Reuse Center for Thompkins County, 2006 Surveys – Summary Report, December 8, 2006.
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

124 | September 2015

materials. The goals of the Center were to reduce waste and poverty, and create livingwage jobs and job training opportunities through reuse activities with a particular focus 

on individuals with barriers to employment.

The Finger Lakes ReUse Center, located in Ithaca opened on November 12, 2008 

featuring salvaged furniture, housewares and building materials. Deconstruction services 

began in June 2009. 

Cost and time are the two primary barriers facing the program. A deconstruction project 

may involve a team of three paid staff plus four volunteers for a period of two weeks or 

more. Thus, despite wanting to “do the right thing,” deconstruction compared to 

demolition may be cost prohibitive. However, deconstruction can be economically 

feasible for individuals in certain tax brackets where the tax write-off for donating 

materials makes it affordable. A knowledgeable appraiser, accurate records, a photo set 

and proper valuation of the re-use materials only (compared to the total value of the 

building) are some of the factors that are needed to document the project and satisfy the 

Internal Revenue Service. Typically, this is not practical for a business or university and 

works better for individuals. The program usually cannot make the economies work for 

the university where the prevailing training rate set by a trade union is in the $10/hour 

range.

Given the cost of deconstruction and the positive outcomes, the program hopes in the 

future to initiate a state-level lobbying effort to encourage subsidies for individuals who 

choose to take on these projects.

Asbestos removal is another barrier. The ReUse Center must abide by the same 

regulations as a specialized asbestos removal contractor. Asbestos occurs in materials 

of all kinds which must be tested and properly eliminated. This can be a costly process. 

Sometimes asbestos can be handled economically and safely by “demolition in place,” 

whereby the materials are hosed down with liquid wetting agents while the structure is 

torn down.

The program was implemented with little deviation from the proposal. The administration 

stayed true to their vision and the response from the community was positive. Despite 

being located in an academic community where strong opinions on all sides of issues are 

common, the reuse program claims support from the full political spectrum of the 

community and by design has taken a markedly non-partisan path.

The program was attributed mostly to ARC funding and would have been delayed for a 

year or more without it. ARC funds were used to purchase all supplies used in the 

deconstruction process and are still in use to date.

6.9.4 Economic and Community Impacts

Deconstruction, the “unbuilding” of a structure, is emerging as an economically viable 

alternative to demolition that saves natural resources, reduces waste, and creates jobs. 

Two general categories of deconstruction or selective dismantling have emerged. 

Selective deconstruction (or soft-stripping/cherry picking or non-structural) precedes 

demolition to remove easy, high-value materials such as paneled doors, lighting fixtures, 

“wavy” glass windows, cabinets, chairs, appliances and some hardwood flooring. Whole 
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

September 2015 | 125

house deconstruction includes soft-stripping but also saves the structure; framing 

lumber, sheathing, bricks, stones, etc.

From 2009 to 2011 this program improved 75 households; 25 small to medium sized 

businesses; 65 organizations and 8 communities. It served 320 students and provided 

local revenue, generated sales taxes, made affordable access to goods, technology and 

services, and allowed free or low cost materials to local non-profit organizations. The 

Center’s revenues increased $267,000 during the grant period and one business was 

retained.

The ReUse Center helped build and educate the community, improving energy efficiency 

and transforming local liabilities (waste) into assets (jobs). The program enhanced the 

local workforce by creating eight living-wage jobs and retaining five positions. Individuals 

were afforded job skill training opportunities with marketable and specialized skills 

through partnerships with outside organizations, especially to underserved communities 

and at-risk youth.

It also provided volunteer opportunities for the community, with multiple paths for 

volunteers to become involved. Volunteer orientations are held monthly, staggering days 

and times to attract as many individuals as possible. These include senior citizens, 

college students, retirees, adults with disabilities, and those on public assistance 

required as part of their obligation to work a specific number of hours. Between five and 

eight of the 17 full-time staff were hired from the public assistance cohort at a given time. 

As of May 2014, seven trainees have secured employment and 1,080 tons of materials 

were diverted.66

A substantial amount of material was diverted from the waste stream: 100 tons during 

the project period and an average of 150-170 tons per year since. The program claims 

70 to 90 percent of materials are prevented from going to landfills, though it often takes 

additional labor force to accomplish this type of deconstruction compared to traditional 

demolition. However, tax deductions are available to those who donate materials, 

mitigating the additional labor expenses. The reuse program offers considerable cost 

savings to the community. The material-related cost impact of the 1,300 tons of materials 

sold in the six years to 2014 has generated $2.3 million in used material sales and 

services. Disposal of this material would have cost the community $110,00067 and not 

provided the sales and services.68

The types of materials salvaged are related to construction, computers, electronics, 

household goods, sporting equipment, textiles and furniture. The Center was able to 

protect the environment by diverting challenging materials through deconstruction, 

refurbishment, repair and creative reuse with a reduction in waste and emissions. It 

provided the community with access to affordable reusable building materials, reduced 

construction and demolition waste entering landfills, and created jobs and job skills 

training opportunities.

 

66 The Community ReUse Center Model. Creating a Sustainable ReUse Infrastructure in New York State. Finger 

Lakes ReUse, Inc. May 19, 2014.

67 This assumes a cost of $85 per ton for disposal.

68 Why ReUse? Community ReUse Centers & Building a Sustainable ReUse Infrastructure. Finger Lakes ReUse, Inc. 

November 18, 2014.
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

126 | September 2015

The program attracted additional funding – including municipal funding for waste 

diversion and private foundation funding for environmental preservation. By 2012, the 

breakdown of revenue generated was 65 percent from business income and 35 percent 

contributed income. One year later, the program budget exceeded $800,000. Outcomes 

extending beyond the grant period include 18 jobs created (17 FTE); sales exceeded 

$400,000 in 2013 ($1.8 million to date); and service fees earned nearly $100,000. In 

2010 the organization won the Tompkins County Chamber of Commerce Not-for-Profit of 

the Year award. 

In addition, the program helped more than a dozen people transition off public assistance 

into living-wage jobs. More than 65 individuals work for the organization in various 

weekly shifts, including 17 FTEs, plus volunteers, job trainees and federal work study 

students receiving financial aid. 

The program has expanded since completion of ARC’s program. An eCenter Computer 

Refurbishing operation started in April 2010 featuring refurbished computers and 

electronics. The Ithaca Fixers Collective began in 2012; the ReSET Job Training 

program in 2013.

A subsequent store location is planned for 2015 pending completion of a second 

environmental assessment. While the original store is located in a shopping mall, the site 

for the new store, which will include a lumber yard, is a highly visible location in an urban 

neighborhood proximate to large home supply stores. Over time, the program hopes to 

open additional satellite stores in underserved locations.

Sharing knowledge is a priority of the staff at Finger Lakes ReUse Center. It has applied 

to ARC to fund an individual to train under them in how to begin a program in another 

community. The Finger Lakes program currently provides technical support for 

communities initiating their own reuse centers and has developed templates, start-up 

materials and financial examples for interested programs to adopt free of charge.

6.9.5 Lessons Learned

The lessons learned are based on staff interviews, speaker presentation materials and 

survey results from a post-program survey conducted by an ARC contractor.

• Program priorities may change. As administrators recognized that for 

many individuals working in the program was transforming their lives, the 

goal of helping at-risk persons get back into the community took on increased 

focus. The initial goals of the program were to launch the ReUse Center, 

create jobs and reuse regional assets, including building materials, furniture 

and architectural elements. Hence, the “human experience” is now a primary 

goal of this effort as are helping workers gain respect and giving people 

second chances.

• Multi-year funding would ease the administrative burden on staff. The 

grant application process is demanding, though administrators acknowledge 

it makes one think analytically about a myriad of program details. 

Nevertheless, not having to re-apply for support each year by writing and 

submitting a new grant proposal could free up program administrators’ time –
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

September 2015 | 127

by eliminating tasks in the grant writing process that are sometimes viewed 

as repetitive and instances of “reinventing the wheel.”

Grantees realize the reluctance of agencies to fund multi-year projects in 

case a project encounters problems. To make sure a program stays on track, 

funding agencies could simply withhold funding until the program rights itself.

• ARC funding provides affirmation which leads to increased community 

support. The grant led to positive responses from the community, lifted the 

program and helped it realize its vision. This affirmation made it possible to 

leverage additional program support.

• There is a huge, untapped need for additional deconstruction programs.

There are massive quantities of materials that can be reused but end up in 

the landfill. Despite this small program being considered a leader in the field, 

for reasons of cost and time it is unlikely to service an area more than one 

hour away.

• Location of the reuse center is key. A highly visible location in a high traffic 

area will further sales of reusable materials. When customers can see the 

center and its wares from the road, particularly when the road is located 

nearby or on the way to other retail home center stores, the volume of reuse 

sales will improve significantly.

• Success components at program origination include: 

o Determining business structure

o Identifying stakeholders

o Developing sufficient support

o The importance of partnerships

o Identifying the board of directors

o Developing a business plan

o Site location

o Inventory management and point-of-sale systems

o Minimizing overhead

6.9.6 Conclusion

The Finger Lakes ReUse Center was able to protect the environment by diverting 

challenging materials through deconstruction, refurbishment, repair and creative reuse 

with a reduction in waste and emissions. It provided community with access to affordable 

reusable building materials, reduced construction and demolition waste entering landfills, 

and created jobs and job skills training opportunities. The ReUse Center seeks to 

become a stepping stone to make these programs viable and available to other 

communities.
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

128 | September 2015

6.10 Expanding Entrepreneurship in Appalachia – REAL 

Entrepreneurship Curriculum Improvement

The North Carolina Rural Entrepreneurship through Action Learning (NC REAL) provides 

experiential entrepreneurship education for schools, community colleges, and adult 

entrepreneurship programs, and is adaptable to other contexts. This grant allowed the

NC REAL to update and modernize its entrepreneurship curricula, encourage additional 

individuals to use the REAL curricula and provide training scholarships as needed. The 

REAL Entrepreneurship curriculum is now a subsidiary of Sequoyah Fund, a community 

development finance institution, which purchased it from NC REAL in 2015. This project 

is classified as a Business/Training project and the case study refers to grant funding in 

FY 2007. 

ARC awarded a grant of $82,600 in 2007 to NC REAL, which helped to secure additional 

local funding of $107,750 and state funds of $15,000 bringing total funding to $205,350.

6.10.1 Community Profile

Twenty-nine counties in North Carolina are part of the Appalachian Region, a 205,000-

square-mile region that follows the spine of the Appalachian Mountains from southern 

New York to northern Mississippi.

In 2012, the population of Appalachian North Carolina approximated 1.7 million 

residents. Their per capita income was $34,215, lower than all of the comparison 

regions. The per capita income was about 78 percent of the national average, 90 percent 

of state average and 95 percent of the entire Appalachian region average. Per capita 

income by county ranged from a low of 62 percent of the U.S. average in Graham 

County to a high of 92 percent in Polk County.69

Appalachian North Carolina experienced a 2012 unemployment rate of 9.4 percent, 

almost equal to the entire state (9.5 percent). The rate in Appalachian North Carolina 

exceeded both the U.S. rate and the rate for entire Appalachian region, both at 8.1 

percent. Sixteen of the 29 Appalachian counties in North Carolina experienced 

unemployment rates in double digits, the most severe in Graham County (16.8 percent). 

The 17.9 percent poverty rate experienced by residents of Appalachian North Carolina 

from 2008 to 2012 exceeded the national rate by three percentage points. Only four of 

the counties fell below the poverty rate for the U.S. The county with the most severe rate 

of poverty among the state’s Appalachian counties, 28.8 percent or almost twice the 

national poverty rate, occurred in Watauga County.70

ARC defines the economic status of each county with one of five possible designations 

based primarily on three indicators: three-year average unemployment rate, market 

income per capita, and poverty rate. The five economic status designations are: 

distressed, at-risk, transitional, competitive, and attainment. “Distressed” counties are the 

most economically depressed and “attainment” counties are the most prosperous. In FY 

 

69 United States Department of Commerce, Bureau of Economic Analysis, Local Area Personal Income, 2012. 

Compiled by the Appalachian Regional Commission, February 2014.

70 Unemployment Rates, 2012; Poverty Rates, 2008-2012. Appalachian North Carolina United States Department of 

Commerce, Bureau of Economic Analysis. Compiled by the Appalachian Regional Commission, February 2014.
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

September 2015 | 129

2012, ARC defined the economic status of all twenty nine counties in Appalachian North 

Carolina as distressed, at-risk, or transitional, the three lowest status designations.71

6.10.2 Project Description

The North Carolina Rural Entrepreneurship through Action Learning (NC REAL) was a 

relatively small training program based in Raleigh, which used action learning to expand 

entrepreneurship in Appalachian North Carolina and across the state. The staff of three 

trainers from NC REAL taught their clients using a “train the trainers” model; training 

instructors and educators who, in turn, would be teaching the REAL curriculum to 

students and entrepreneurs in their schools, institutions and local communities. The 

REAL program encourages development of sustainable enterprises by helping 

individuals develop the skills required for successful business ventures.

Though this case study focuses on program enhancements made during the 

Appalachian North Carolina phase of the program, the REAL program curricula is taught 

in 43 states and foreign countries, hundreds of high schools, post-secondary schools, 

community and online-based programs. More than 400 elementary and middle school 

facilitators also use REAL curricula.72 In order for an institution to offer REAL training, its 

instructors must have successfully completed the training, be certified by the REAL 

program and own a copy of the curriculum.

The FY 2007 ARC grant funding allowed NC REAL to undertake a periodic update and 

modernization of its curricula, expand trainer education and grow its entrepreneurship 

education programs. The curriculum revision replaced out-of-date references (such as 

obsolete references to video cassette tapes) in the 20-year-old coursework, integrated 

modern online tools and technologies and brought in contemporary electronic 

enhancements. The update also added additional financial literacy to the coursework and 

enhanced information about emerging industry clusters. ARC funding laid the 

groundwork for current specialized curricula offerings (in subject areas such as 

agriculture, healthcare and craft artistry), and expansion into new communities. In 

addition, 40 new teachers from the Appalachian region were trained at REAL Summer 

Institutes, with scholarship support provided as needed. In general, ARC support 

underwrote costs of personnel, supplies, contract support, and training scholarships.

The following is a description of the curriculum as stated on an archived website from the 

REAL program from around the time of ARC’s grant:

The REAL Entrepreneurship curriculum consists of a printed guide of 

core activities and a USB Flash Drive with supplemental activities and 

resources designed for those setting up hands on REAL 

Entrepreneurship programs in their schools or community organizations 

The curriculum includes group and individual activities business planning 

journals an integrated technology component and a course and program 

planning section. Note The curriculum is licensed only to teachers who 

attend a REAL Entrepreneurship Institute.73

 

71 County Economic Status Fiscal Year 2012. Appalachian Regional Commission.

72 About REAL, http://ncreal.org/what_is_real

73 http://ncreal.org/our_programs (2014-02-24)
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

130 | September 2015

6.10.3 Project Planning and Implementation

With special emphasis on rural communities in North Carolina, NC REAL’s mission was 

to develop entrepreneurial talent through hands-on education and to advance the 

formation of sustainable enterprises. A key goal of the program, which began in 1985, 

was to encourage more individuals in Western North Carolina to teach using the REAL 

curricula, including instructors at community colleges and high school organization 

facilitators. NC REAL used a train-the-trainer model, allowing three REAL trainers to offer 

12 to 15 full-week training sessions across the state. The curriculum was adaptable to 

community-based organizations, including those assisting Hispanic and Native American 

populations, and to current or potential entrepreneurs.74

At its core, REAL was designed to teach potential entrepreneurs the ins-and-outs of 

running a small business. REAL offered experiential education to a wide range of 

audiences, spanning from students and youth in elementary and middle schools, to postsecondary students and adults. By teaching their teachers in the REAL curriculum, for 

example, students in turn, become able to implement school-based enterprises and 

small businesses in their home communities. One primary goal of the REAL 

entrepreneurship curricula was for students to develop feasible business plans through 

which they could assess their entrepreneurial potential and research, plan, set-up and 

operate businesses of their own design.

Scholarships were often available to instructors who met program qualifications and 

developed plans describing how they would use their training to train others. Employers 

usually picked up the tab for those without scholarship support. In fact, many educational 

institutions required faculty to attend the REAL courses as a regular part of their

professional development, especially new hires. Instructors who successfully completed 

the program earned their certification in the REAL curricula and continuing education 

units (CEUs), if appropriate.

Though ARC grant funding focused on the in-state program, the REAL program is now 

taught in 43 states and foreign countries. NC REAL was responsible for developing the 

original entrepreneurship curricula, which was adopted by at least two other states. Over 

time, these other states dropped by the wayside, and by 2007, the period covered by this 

grant cycle, NC REAL had secured all rights to the entrepreneurial training program. 

Currently all institutions, domestic and foreign, must purchase the curricula from NC 

REAL.75

NC REAL was able to identify potential trainers through its community relationships with 

small business centers, community colleges and high schools, and follow-up with former 

instructors. Additional sources were utilized to identify potential instructors for the 

program’s more recent specialty curricula development. For example, personnel were 

identified to teach the REAL agricultural curricula through cooperative extension 

programs (supported by the United States Department of Agriculture), county agencies 

and/or councils of government. Crafts persons and artists to teach the program 

coursework designed for artists were identified through art and economic development 

groups at community colleges. With few exceptions, most participants resided in-state.

 

74 How does REAL Work? http://ncreal.org/what_is_real/how_real_works_cfed

75 See introduction concerning sale of the program in 2015.
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

September 2015 | 131

NC REAL officials acknowledge this program would not have taken place without ARC 

funding. By facilitating the curriculum update, the grant laid the groundwork for the 

subsequent REAL Agricultural Entrepreneurship Curriculum funded across the state and 

other curricula designed for new targeted populations as described above. Leveraged 

funding from Z. Smith Reynolds supported a strategic planning project to assist NC 

REAL determine its next five years.76

6.10.4 Economic and Community Impacts

Since the onset of the NC REAL program in 1985, more than 12,000 entrepreneurs have 

received intensive training and more than 500 North Carolina high school, postsecondary and community-based organization educators and administrators have 

participated. NC REAL offers its experiential education and professional development 

courses as seminars and week-long institutes. More than 1,500 entrepreneurs, bankers, 

accountants, and other professionals have provided assistance to NC REAL’s 

participants and graduates. The program creates 2.5 jobs per REAL business start-up, 

generating $3 million in sales per month across the state.77 The REAL program is taught 

in 43 states and foreign countries. The modules are available to community colleges, 

small business centers, and community development centers statewide. The program 

website provides course registration information and workbooks for purchase.78

During ARC’s funding period, from 2008 to 2011, 18 organizations, 24 communities and 

5,000 students were served. More than 250 instructors were trained in REAL 

Entrepreneurship. Eighteen organizations and 28 communities were improved and more 

than 150 courses were offered in North Carolina alone. These results are based on 

reports of REAL instructors from across the region, contacted two times a year. 

However, program administrators note that since REAL offers the coursework to the 

trainers, rather than directly to the individuals who will be using it directly in the 

community, data on the number of businesses created or enhanced is anecdotal and 

may not be comprehensive. 

NC REAL does, however, track directly the courses offered and student enrollments. NC 

REAL entrepreneurial training programs have created thousands of new jobs across the 

state through its training programs and curricula. It has also fostered networking among 

business owners who have taken the course. 

ARC funding assisted NC REAL to offer new, expanded specialized entrepreneurial 

curricula for craft/artists, agricultural/farmers and healthcare workers. In some cases, the 

NC REAL program was approached to find out if it could assist special populations, such 

as laid off agricultural workers. In this case, staff recognized the specialized needs of this 

population, so rather than tweaking the curricula, NC REAL developed a new set of 

courses specifically designed to address their unique situation. In this manner the 

program evolved and was able to reach partners and train new groups of individuals not 

served previously.

 

76 NC REAL reports in 2012 that a $35,000 grant from Z. Smith Reynolds Foundation complements a previous 

$28,000 commitment by the Foundation to strategically plan for the future. 

http://archive.constantcontact.com/fs031/1109430909585/archive/1110008793236.html

77 About REAL. http://ncreal.org/what_is_real

78 http://www.entrekeys.com/
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

132 | September 2015

The updated curriculum and instructor training allowed additional community colleges to 

offer an entrepreneurship course and increased the variety of courses offered. The 

program provided comprehensive professional development program for instructors (inservice seminars, site visits); experiential, activity-based entrepreneurship curricula; 

course materials; evaluation and documentation strategies; and school-based 

module/training. It is currently taught in every community college in Western North 

Carolina.

In a post-program interview, grant officials noted a program obstacle - reluctance among 

facilitators to employ the program’s evaluation system. This made it problematic to 

conduct a proper appraisal of the NC REAL grant program. The inability to track progress 

against program goals was disappointing to NC REAL officials, despite recognition that 

ARC valued the use of concrete measures to gauge success. Despite revamping its 

evaluation system, the low response rates prohibited the grantee from proper evaluation 

of the program.

Another issue cited by program staff was limited success in getting communities to 

support what they called a “culture of entrepreneurship.” One way the NC REAL 

encouraged this mentality was to ask trainees to discuss what was missing from their 

communities and address how entrepreneurs could resolve these deficiencies. For 

example, participants were asked to contemplate why their community was overly 

dependent on tourism and to try to come up with entrepreneurial approaches the 

community could leverage to meet the challenge. 

The grant met its primary goals of updating and modernizing the REAL curriculum and 

expanding the REAL program to new and increasing partners in Appalachian NC. A lessvital goal, to strengthen groups of partners, was deemed somewhat less successful by 

the grantee. For example, program administrators described how turnout for training at 

an Alabama site turned out to be less than anticipated. Here the eventual outcomes were 

characterized as modest, limited to a church-sponsored cookbook, a program fundraising 

event and the experience of attending the program.

Nevertheless, ARC funding allowed the NC REAL to grow as an organization and 

expand its reach. NC REAL evolved as a viable entrepreneurial education model 

program, modernizing its training curriculum and working in multiple states and countries. 

The program was able to develop new focus areas with new partners such as 

agriculture, healthcare and craft artistry. The results were more than 12,000 

entrepreneurs gaining intensive training and the participation of hundreds of educators 

and administrators experiencing the REAL entrepreneurial training program.

6.10.5 Lessons Learned

The lessons learned are drawn from interviews of program administrators and survey 

responses from a post-program questionnaire conducted by an ARC contractor.

• Securing partnerships at program outset is essential. Support from 

partners, often considered a mere formality, should be taken seriously 

beginning with the proposal stage of a program. Letters of support from 

partners enhance credibility allowing outreach to additional local and regional 

communities, and particularly communities that can provide specialized 

expertise needed for the success of the current program or future programs. 
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

September 2015 | 133

• Routine updates can foreshadow unforeseen opportunities to evolve 

and serve new partners. Bringing the curriculum up-to-date to include 

newer technologies and subject matter ensured it could be quickly modified 

later on. This allowed the program to reach new communities, including 

healthcare workers, farmers and craft persons/artists. 

• Consider how to track data early in the process. It is important to inform 

participants that providing follow-up tracking data is expected. Consensus on 

measurements and processes can boost response rates during the 

evaluation phase of a program.

6.10.6 Conclusion

NC REAL’s mission to provide timely, relevant entrepreneurship education was 

advanced by updating the REAL Curriculum. This revision allowed expansion into new 

communities and made possible the development of specialized curricula. At the time of 

this grant, the NC REAL program was at a crossroads. It had to update and modernize 

its REAL Curriculum to stay relevant. ARC funding made this possible and allowed REAL 

to expand its offerings and expand its training of new facilitators. ARC funding was the 

impetus that got the organization moving to where it is now; an evolving program 

positioned for the future. The skill levels of the workforce, the viability of local business 

and access to local markets were improved as a result of this program.

6.11 Northeastern Pennsylvania Alliance Enterprise 

Development

In 2010, ARC awarded a $400,000 business development grant for technical assistance 

to the Northeastern Pennsylvania Alliance to continue to serve area businesses and 

industries with small business financing, international business development assistance, 

procurement technical assistance, e-commerce activities and its network of service 

providers.

6.11.1 Community Profile

In 2012, almost 6 million Pennsylvania residents (45 percent) lived in the state’s 52 

Appalachian counties. Seven of these counties – Carbon, Lackawanna, Luzerne, 

Monroe, Pike, Schuylkill, and Wayne – comprise the Northeast Pennsylvania Local 

Development District (LDD). LDDs assist counties by strengthening local partnerships, 

addressing common issues and responding to critical concerns.79 Almost 18 percent of 

residents of Appalachian Pennsylvania lived in the seven counties served by the 

Northeast PA Development District (NEPA).

Fifty-two counties in Pennsylvania are served by LDDs, multi-county organizations 

providing services in such areas as community and economic development, 

transportation, international trade, strategic planning and much more. Counties share 

information through their LDD, respond to common concerns, and develop solutions to 

today’s critical issues. For over 40 years Pennsylvania’s LDDs have taken the lead in 

 

79 www.nepa-alliance.org/about/local-development-districts-ldd-history
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

134 | September 2015

community and business initiatives, coordinating and delivering services on behalf of 

their member counties as well as the Commonwealth. NEPA is the local LDD for ARC in 

Northeastern Pennsylvania. All seven LDDs are designated by ARC.

In 2012, the residents of Appalachian Pennsylvania realized only about 91 percent of the 

average per capita income of the entire state. However, Pennsylvanians in general, and 

those in Appalachian Pennsylvania, fared slightly better than their comparison cohorts in 

terms of average per capita income. The average per capita income for the state of 

Pennsylvania slightly exceeded that of the U.S. (103.1 percent). Similarly, the average 

per capita income in Appalachian Pennsylvania exceeded that of the entire Appalachian 

region ($41,082 to $35,849).

In contrast, six of the seven counties in the NEPA service region fell below the average 

per capita income of Appalachian Pennsylvania Of the seven counties in the NEPA 

service region, only Lackawanna County realized higher average per capita income than 

the overall average of Appalachian Pennsylvania. The average unemployment rate for 

the Appalachian Pennsylvania counties in 2012 matched that of the Appalachian Region 

(7.9 percent). Of the seven NEPA counties, only one, Wayne County, had an 

unemployment rate below that of the average for all Pennsylvanian Appalachian 

counties.

ARC defines the economic status of each county with one of five possible economic 

designations based primarily on three indicators: three year average unemployment rate, 

market income per capita, and poverty rate. The statuses are distressed, at-risk, 

transitional, competitive or attainment. Distressed counties are the most economically 

endangered and attainment counties are the most prosperous. Of the 52 counties of 

Appalachian Pennsylvania, all but nine were designated transitional, the middle rating. Of 

the remaining nine counties, seven were designated as competitive, and one each was 

distressed or at-risk. ARC rated all seven counties in the NEPA region as transitional.80

6.11.2 Project Description

The project allowed Enterprise Development Programs to assist local businesses, 

community based organizations, local and county governments and not-for-profit 

organizations. According to program officials, clients were small businesses with fewer 

than 100 workers, and in fact, most had fewer than 50 employees.

The program provided technical and marketing assistance in three core areas:

1. Government Procurement Assistance

2. Business Finance Assistance

3. Export Marketing Assistance 

Government Procurement Assistance

The program brings buyers together with Pennsylvania suppliers of services and 

products through its PA SourceNet. This system aims to maintain the network based on 

high quality reliable information. Data are reviewed by a qualified marketing specialist 

 

80 County Economic Status, Fiscal Year 2012, Appalachian Pennsylvania. 

http://www.arc.gov/reports/region_report.asp?FIPS=42999&REPORT_ID=37
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

September 2015 | 135

and accuracy is verified prior to enrollment. More recently, the program was expanded to 

link state companies with prequalified international trade leads.

The program helps small businesses sell products and services to federal, state or local 

government agencies through NEPA’s Procurement Technical Assistance Center 

(PTAC). It also assists companies who wish to bid on federal contracts and helps 

government agencies locate local regional suppliers.

Business Finance Assistance

NEPA’s Business Finance Sector provides direct loans to eligible underfinanced small 

businesses, from a wide range of eligible sectors, with expectations that jobs will be 

created or retained. The finances help pave the way for these businesses to secure 

additional investment from banks and other sources of private and public funding. The 

loans feature low, fixed interest rates, lower down payments and standard loan term 

maturities. NEPA also makes available microloans for business start-ups, small 

businesses and entrepreneurs. 

Export Marketing Assistance

NEPA is a primary point of contact for regional businesses engaged in international 

trade. It provides information on the benefits of international trade and international 

business advice to new and experienced exporters to grow sales and expand markets. 

Other Services

As the LDD, NEPA develops maps and datasets using the latest geographic information 

system (GIS) technology. The data are used for planning, resource management, 

infrastructure analysis, emergency response and site selection. Additional communityeconomic development support services include rail line retention, entrepreneurship 

development programs (EDP), modeling to identify business opportunities, and regional 

events, among many other services. 

NEPA is a 501(c)(4) not-for-profit organization directed by the following entities:

• Volunteer Board of Directors

• Local government and county officials

• Other not-for-profit executives

• Development organizations

• Private industry individuals

The $400,000 funding in 2010 from ARC was used for operational expenses. According 

to the program director, additional sources of funds included the U.S. Economic 

Development Administration and various state funding agencies. Additional state, federal 

and local support brought total funding to $800,000.

6.11.3 Project Planning and Implementation

The Northeast PA Alliance Enterprise (NEPA) was founded in 1964 to serve the seven 

counties on Northeastern Pennsylvania. NEPA was a non-profit, private, regional 
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

136 | September 2015

community and economic development agency providing services in business financing, 

government contracting assistance, international trade assistance, non-profit assistance, 

transportation planning, research and information, and local government service. ARC 

funding began in 1981-1982. At that time, the program was comprised of a financial 

component only. These initial funds were used for staffing and staff training 

opportunities.81 According to the program director, as the program expanded subsequent 

funding was used for more general operating expenses.

Pennsylvania’s economy was slowing by the mid-1990s and business leaders sought to 

influence the Commonwealth’s direction in the world economy. Team Pennsylvania 

(Team PA) was launched as an idea in 1997 allowing government and industry leaders 

to work together in the state’s job growth and business climate and collaborate with like 

organizations to ensure the Commonwealth’s economic development.82 The 

Northeastern Region of the state’s economy solidly relied on small business, 

concentrated in the services, retail trade and manufacturing sectors. From 1998-2007 

employment in the manufacturing sector fell 26.0 percent.83

When ARC was slated for cutbacks and eventually for elimination by the Reagan 

Administration in the early 1980s, leading to cutbacks in funding, Pennsylvania 

developed a series of reports entitled “Choices for Pennsylvanians.” They focused on job 

generation and assisting small businesses with recommendations in a half-dozen key 

areas (including small business finance, new business incubation, assistance to 

entrepreneurs, and others) and concentration on special areas called Enterprise Zones. 

In Pennsylvania, factors driving the designation of Enterprise Zones included high 

unemployment rates, persistent poverty and blight, deficient infrastructure, closures of 

major plants, and poor business climate, among others.84

The current program, according to program officials, was based on the framework 

developed by Walter Plosila, at the time a former deputy secretary in the Pennsylvania 

Department of Commerce and Director of the Governor’s Office of Planning. He was a 

pioneer in the development of business incubators, which began around 1964 but did not 

take off until the early 1980s.85

As the LDD for the region, it could focus on programs deemed most important to the 

area. Initially, the Northeast PA Alliance only could support a finance program and funds 

to hire and train staff. 

6.11.4 Economic and Community Impacts

The following outputs were reported to ARC. The grantee expected the program would 

generate 200 new jobs. It created 169 positions. About 585 jobs were predicted for 

retention. It retained 640 jobs. The program expected to realize $4 million in leveraged 

 

81 http://www.nepa-alliance.org/about/mission/

82 http://teampa.com/about

83 Northeastern Pennsylvania Enterprise Development Program Report – Chapter 1: Project Summary.

84 Wong, Sidney. Local Enterprise Zone Programs and Economic Development Planning: A Case Study of California 

and Four Mid-Atlantic States, 1996. Pennsylvania Institute for Urban Research collection.

85 Petrucci, J. Evolving Innovation: Research Parks as the Next Generation of Business Incubators. Keystone Edge, 

July 9, 2009.
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

September 2015 | 137

private investment though none was reported to ARC for this particular reporting period. 

However, information obtained from the grantee directly indicates the program exceeded 

expectations, leveraging more than $5.4 million from the private sector from October 

2009 through September 2010.86

According to ARC data, during the previous full grant year, the grantee made 14 loans 

totaling $1.49 million, leveraging $6.5 million in private funds and served 220 companies 

in its technical assistance programs. As a result of the grantee’s assistance, area 

companies experienced a $23 million increase in government procurement contracts. 

The total reported job impact was 365 jobs created and 740 jobs retained.

6.11.5 Lessons Learned

Lessons learned were gleaned from interviews of program administrators. 

• Hire the correct type of staff. While this advice is applicable to many types 

of programs, it applies especially to finance-related programs. Program 

officials recommend that when hiring, seek individuals who are able to adapt 

and learn, and can function in different situations. It is also important to pay 

them what they are worth.

• Temper expectations. Do not set expectations too high at the beginning of a 

program. Outputs take time to develop and reach potentials.

• Develop outcomes that can be documented. It is important to be able to 

capture and document outcomes to report back to funding agencies. Jobs, 

new sales and new markets are good examples of metrics that can prove a 

good investment to program funders and hopefully convince them to increase 

funding in the future. The ability to document private investment and 

subsequent jobs helps prove a program’s worth and illustrate to stakeholders 

the program’s accomplishments.

6.11.6 Conclusion

The Northeastern Pennsylvania Alliance program (NEPA) continues to provide business 

finance services, a procurement outreach program for companies interested in marketing 

their products or services to the federal, state and/or local governments and an 

international trade program to small businesses in the seven NEPA service counties of 

Northeastern Pennsylvania. In addition, it offers a geographic information system (GIS) 

program, information services, education and training, and entrepreneurial initiatives, all 

supported by ARC funds. 

The program has generated thousands of jobs since its inception and millions of dollars 

in leveraged private funds. Though this case study report focuses on one year of funding, 

the program’s longevity and outcomes are testaments to the its success.

 

86 ARC changed the reporting timeframe in 2010. Some measures may cover only 9 months ending 6/30/2010. 
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

138 | September 2015

6.12 Building the Clean Energy Economy in Western 

North Carolina

In 2010, ARC awarded a $100,000 business development grant to Land-of-Sky Regional 

Council, a multi-county local planning and development organization in Western North 

Carolina. Additional federal and state funds doubled the total to $200,000. ARC support 

underwrote personnel, indirect costs, travel, supplies and contractual expenses. 

The purpose of the project was to initiate activities to support the growth of a clean 

energy economy in the western region of the state. Actions included development of a 

website to network clean energy businesses, hosting training sessions for local area 

businesses and residents on clean and renewable energy, developing strategic energy 

plans for local governments and forming a regional leadership group to guide clean 

energy efforts and identify regional branding opportunities.

6.12.1 Community Profile

Twenty-nine counties in North Carolina are part of the Appalachian Region, a 205,000-

square-mile region that follows the spine of the Appalachian Mountains from southern 

New York to northern Mississippi.

The 2012 population of Appalachian North Carolina was about 1.7 million residents. 

Their average per capita income was only $34,215, about 78 percent of national, 90 

percent of state and 95 percent of all Appalachia in per capita income. Average per 

capita income by Appalachian North Carolina County ranged from 62 percent of the U.S. 

average in Graham County to 92 percent in Polk County.87

Joblessness and poverty were significant issues in the region. The unemployment rate in 

Appalachian North Carolina in 2012 was 9.4 percent, about equal to the state (9.5 

percent), compared to 8.1 percent for both the U.S. and the entire Appalachia region. 

Sixteen of 29 Appalachian counties in the state experienced double digit unemployment 

rates, the most severe in Graham Country (16.8 percent). The 17.9 percent poverty rate 

in the region, from 2008 to 2012, was three percentage points above the national rate 

(14.9 percent). Only four counties fell below the national average. The most severe 

poverty rate among Appalachian North Carolina counties in occurred in Watauga County 

(28.8 percent); a rate almost twice that of the entire U.S.88

ARC defines the economic status of each county with one of five possible economic 

designations based primarily on three indicators: three year average unemployment rate, 

market income per capita, and poverty rate. The statuses are distressed, at-risk, 

transitional, competitive or attainment. Distressed counties are the most economically 

endangered and attainment counties are the most prosperous. ARC defines the 

economic status of all of these twenty nine Appalachian counties in North Carolina as at-

 

87 United States Department of Commerce, Bureau of Economic Analysis, Local area Personal Income, 2012. 

Compiled by the Appalachian Regional Commission, February 2014.

88 Unemployment Rates, 2012; Poverty Rates, 2008-2012. Appalachian North Carolina, United States Department of 

Commerce, Bureau of Economic analysis. Compiled by the Appalachian Regional Commission, February 2014.
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

September 2015 | 139

risk, distressed or transitional. None of the counties in the region received the higher 

economic status of competitive or attainment.89

6.12.2 Project Description

Land-of-Sky Regional Council (LOSRC) is a multi-county, local government planning and 

development organization in Western North Carolina. There are 16 regional councils in 

North Carolina. Regional councils are volunteer associations of local governments taking 

on many of the decision making, planning, management and technical services to local 

governments which are best addressed on a regional level.90 Regional Councils are 

regarded as local governments without taxing or police authority 

LOSRC initiated the Building Clean Energy Economy in Western North Carolina project 

in 2009 to promote clean energy and entrepreneurship in the region. The project 

supported partners with information, expertise and implementation tools to develop the 

clean energy economy in the 31 counties of Western North Carolina (WNC). Support 

included regional leadership, technical assistance and development services for local 

governments and communities. An important step was the development of an interactive 

website to support regional networking, assisting local governments in planning and with 

knowledge, branding, marketing, and business outreach. The program incorporated 

collection and analysis of data, and establishing a leadership group of high-level 

stakeholders. Regional partners included: AdvantageWest, Blue Ridge Entrepreneurial 

Council, AB Tech, Blue Ridge Community College, and the six Western North Carolina 

Councils of Government. ARC funds were judged as critical to the program, with 

administrators acknowledging that the program would not have existed without ARC 

support.

Given the significant economic challenges of the region, a key priority was job creation 

via development of alternative fuels and green energy initiatives. The instability of the 

traditional fuel industry together with an aging energy infrastructure, rising energy costs, 

growing joblessness, falling wages and the loss of manufacturing and construction sector 

jobs were all contributing to hardships in the region. Moving towards a clean energy 

economy was seen as an important step in resolving these circumstances.

6.12.3 Project Planning and Implementation

The evolving atmosphere of clean energy, emerging clean energy techniques and 

economies, and growing environmental awareness encouraged development of this 

program. It was conceived by the region’s Comprehensive Economic Development 

Strategy to foster clean energy planning and entrepreneurship and aligned with the 

priorities of regional planners seeking a means to establish clean energy businesses in 

Western North Carolina, broaden present ventures and provide guidance. The program 

filled this void by providing leadership, planning mechanisms, entrepreneurial proficiency 

and an overarching structure to move these efforts forward.

 

89 County Economic Status Fiscal Year 2012, Appalachian Regional Commission.

90 Land-of-Sky.org. Frequently Asked Questions.
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

140 | September 2015

Heretofore, the grantee, Land-of-Sky Regional Council (LOSRC), had worked on a 

campaign to develop clean energy, clean air and clean vehicles. LOSRC and the 

regional economic development corporation AdvantageWest developed the 31 county 

clean energy initiatives to include all six regional councils of government in the district 

and economic development partners. Prior to ARC’s grant, partial funding was obtained 

from the U.S. Department of Commerce, Economic Development Administration allowing 

some partners to start early and actively move forward on this vision for more than a 

year. This early funding allowed project partners to refine the vision, establish roles and 

prioritize project activities.91

An administrative body was required to guide the effort. To establish this Leadership 

Group, the Project Advisory Group vetted an initial list of more than 70 individuals. It 

agreed on a final list of 32 participants from diverse industry backgrounds and 

geographic areas, after networking the candidates and meeting with a number of 

stakeholder groups. Participants came from diverse segments of the communities. They 

included business, service providers, local governments and councils of governments, 

education and workforce development, nonprofit businesses, public and private business 

developers, recruiters and marketing experts and utilities, among others. The Advisory 

Group developed a vision and roles for the Leadership Group. Figure 18 shows the 

structure of the Leadership Group and Clean Energy Network for this project.

The Leadership Group provided expertise, guidance and materials for marketing plans, 

branding programs, strategies for economic development, and workforce and 

infrastructure expansion. Composed of multiple partners, they sought to raise awareness 

about the importance of clean energy policy issues, grow local jobs and businesses, and 

garner support for the WNC clean energy sector. Their stated purpose was, “to create a 

unified umbrella of leadership, strategy and promotion of WNC’s clean energy 

economy.”92

The structure and focus of the Leadership Group was refined during the grant period. Its 

role morphed from primarily advisory to a more action-oriented body. Preserving dollars 

generated by clean energy within the WNC region was a program priority. To this end the 

Leadership Group focused on “internal recruitment” – growing local businesses while 

also working to draw industries to the region to bolster the clean energy cluster.93

Central to the program was the development of a Web 2.0 platform; networking the 

cluster of clean energy businesses, supporting blogs and business profiles, facilitating 

social networking and communicating updated content. As the program developed over 

the course of the grant, the content of the Web 2.0 portal, AdvantageGreenNetwork.org, 

expanded and its memberships increased. 

The EvolveEnergy Partnership was developed during the grant period to foster 

innovation, financing, policy and workforce tools. It supports expansion of existing clean 

 

91 Land of Sky Regional Council. Progress Report Quarter 1. Building the Clean Energy Economy in Western North 

Carolina. July 30, 2010.

92 Progress Report Quarter 1, July 30, 2010.

93 Ibid.
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

September 2015 | 141

energy businesses, creation of new ventures and clean energy commercialization and 

partnerships among the cluster of clean energy businesses.94

6.12.4 Economic and Community Impacts

The Building the Clean Energy Economy in Western North Carolina program brought 

together individuals, businesses and groups that would be unlikely to collaborate had this 

program not existed. The high-level staff and regional leaders on the Leadership Group 

encompassed diverse geographic and industry backgrounds. Success of the program 

hinged on connections made possible by ARC’s grant. 

As a result of the program, new to the region businesses, such as Sierra Nevada 

Brewing Company and other like-minded companies who share the clean energy 

philosophy, were recruited. Like Sierra Nevada Brewing, some enterprises were related 

to the craft beer and cider brewing industry, while others represented philosophicallysimilar businesses, such as outdoor gear manufacturing and those involved in additional 

areas of outdoor recreation. The number of clean energy businesses involved in the 

program expanded and grew. New and existing businesses involved in the clean energy 

sector became strong constituents of the local economy.

The Web 2.0 platform fostered networking among the clean energy business cluster, 

expanding to include the 31-county region. The platform incorporated active blogs, 

profiles of businesses, social networking, and quarterly updates. By the third quarter of 

funding, unique visits were up 94 percent, memberships had more than doubled, and 

email newsletters, blog posts and news articles related to clean energy were increasingly 

posted.95 Networking facilitated by the web platform assisted businesses finding 

resources in siting, working capital, marketing networks and additional services to spread 

the word out about their products and get their goods to market. It also broadcast rules

and regulations pertaining to clean energy and renewables.

The EvolveEnergy Partnership, an initiative of business and community leaders to make 

WNC a model of economic and environmental success through support of clean energy 

innovation and industry development, formally launched in 2011. It can trace its roots to 

the planning, framework, strategic initiatives, research studies and branding resulting 

from this grant. Uniting the regional partners, it is responsible for creating more than 250 

jobs, securing more than $5.5 million in investment, supporting nine new clean energy 

startups, training more than 625 professionals, and generating in excess of $56,000 in ad 

value and $280,000 in public relations value for Western North Carolina’s clean energy 

economy.96

The Building the Clean Energy Economy in Western North Carolina program came 

together around a common vision, developed strategies and created a unified brand. It 

developed outreach to businesses and individuals committed to the clean energy 

economy, and supported recruitment and tactics. The program fostered business 

networking forums, and encouraged attendance and sponsored events for site 

consultants. Program funds supported lead generation meetings and outreach, training 

 

94 http://evolveenergyp.com/about-the-project/history-accomplishments/

95 Progress Report Quarter 3, January 31, 2011.

96 http://evolveenergyp.com/about-the-project/history-accomplishments/
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

142 | September 2015

seminars and outreach to media, including newsletters, blog postings, fact sheets and 

surveys. They produced applications for recruiting fellows and interns. Grant funds were 

used to subsidize presentations at professional meetings and develop tracks of interest 

in related subject areas. The program supported resource directories, grant tools for 

green projects, continuing education seminars and business training. It delivered 

business support services through incubation, at times in conjunction with local colleges. 

The program planned to create nine jobs, a goal accomplished by the end of the grant 

period. It also predicted the creation of three new businesses. However, at program 

completion, it generated 173 new businesses. The program served 134 businesses, 38 

organizations and 31 communities. 

At the end of Phase I of the program in 2011, Western North Carolina had the highest 

concentration of clean energy companies in the state with more than 475 businesses; a 

clean energy company in every WNC county; and annual industry growth rates 

significantly outpacing state and national averages, among other measures of success.97

6.12.5 Lessons Learned

The lessons learned are drawn from interviews of program administrators, survey 

responses from a post-program questionnaire conducted by an ARC contractor, online 

information and report documents, such as progress reports.

• Resist over-scaling the program. The very broad scope of the program 

was an issue the grantee faced from the onset of the project. This program 

covered a large geographic area with varying needs and concerns. Regional 

participation and demand varied, resulting in mixed levels of support for the 

program. The grantee suggests the program administration and outcomes 

might have benefited from a more compact scale combined with a more 

defined program focus.

• Develop a long term strategy. Program leaders did not anticipate how the 

program would fare or if it could continue to exist when the outside funding 

expired. According to program officials, toward the end of the program, 

administrators struggled over whether the program could survive without 

grant funding. In the end, it did not. AdvantageWest was scheduled to close 

at the end of summer 2015.

6.12.6 Conclusion

The Building the Clean Energy Economy in Western North Carolina grant created 

connections among like-minded companies who shared the mindset of a clean energy 

economy in Western North Carolina. These businesses shared values of conservation, 

healthy economy, and quality of place. The program facilitated the relocation of new 

companies and service providers to the region who shared these values.

The success of the program lies in creating a foundation that remains in place today, 

though the program itself may no longer be up and running. That may be acceptable in 

 

97 Building the Clean Energy Economy in Western North Carolina Phase I Final Report, Land-of-Sky Regional 

Council, 2012.
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

September 2015 | 143

the life-cycle of the organization. The enduring legacy of the program is that it has 

become mainstream and is incorporated into the culture of the region.

6.13 West Virginia High Technology Consortium 

Foundation INNOVA Commercialization Technical 

Assistance

This project allowed the West Virginia High Technology Consortium (WVHTC) to expand 

its activities in order to accelerate product commercialization by continuing support for an 

early-stage investment fund, and linking this new source of capital to a range of technical 

assistance efforts. The INNOVA Commercialization Group (INNOVA) investment fund 

made small investments (under $100,000) in companies with new technologies that 

required additional market research and commercial testing. INNOVA also partnered with 

public and private sector technical assistance organizations to help bring these new 

technologies to market.

In 2005, ARC awarded a Business/Training related grant of $125,000 to WVHTC to 

support early-stage investment and technical assistance to high-technology start-up 

small businesses in West Virginia. Additional funds brought the total funding to $250,000.

6.13.1 Community Profile

All fifty-five counties in the state of West Virginia are part of the Appalachian Region, a 

205,000-square-mile region that follows the spine of the Appalachian Mountains from 

southern New York to northern Mississippi. 

The 2012 population of West Virginia (and therefore the Appalachian region of the state) 

was just over 1.86 million residents who realized average per capita income of $35,082, 

about 80.2 percent of the U.S. average, and just below the average for the entire 

Appalachian region (98 percent). Per capita income ranged from a low of $22,412 in 

Doddridge County to a high of $44,660 in Kanawha County, or just over half (51.2 

percent) to approximately twice (102.1 percent) the U.S. average, respectively. 

The 2012 unemployment rate for the state (and therefore Appalachian region of the 

state) at 7.2 percent was slightly better, that is lower, than the 8.1 percent both for the 

U.S. and the combined Appalachian region. Clay County experienced the highest 

unemployment rate in West Virginia (12.2 percent) or about 151 percent of the U.S. 

average, while Monongalia County (4.8 percent) had the lowest jobless rate in the state, 

about 59 percent of the U.S. average. 

In 2005, ARC used a four category county economic classification system to define the 

economic status of each county based primarily on three indicators: three year average 

unemployment rate, market income per capita, and poverty rate. The four economic 

status designations were distressed, transitional, competitive, and attainment. Distressed 

counties were the most economically endangered and attainment counties were the most 

prosperous. At the time of program funding, no county in the state received the highest 

economic status designation and only two received the second highest designation. 

Nineteen were designated distressed, the lowest rating, and 34 transitional, the second 
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

144 | September 2015

lowest rating.98 From 2005 to 2009, the poverty rate for the state was 17.6 percent or 

about 131 percent of the U.S. average. 

ARC added a fifth economic status designation, at-risk, in FY 2006, between the lowest 

designation distressed and second-lowest transitional designation.99 In 2012, only two of 

the state’s fifty-five counties received the highest or second-highest economic status 

designations. Eight were designated as distressed, 13 were deemed at-risk and 32 were 

rated as transitional.100

6.13.2 Project Description

Headquartered in Fairmont, West Virginia High Technology Consortium (WVHTC) is a 

501(c)(3) non-profit company that helps technology companies reach commercial 

success. The importance of diversifying the economy of West Virginia provided the 

impetus for the program. 

As traditional manufacturing, coal mining and labor intensive jobs declined due to 

technical advances required to increase productivity, both Senator Robert C. Byrd and 

Congressman Alan B. Mollohan recognized the opportunity to grow high technology 

enterprises within West Virginia. WVHTC was formed in 1990 to provide business 

support services to develop high-tech businesses in the state. The program offers 

entrepreneurial and business development training, consultation, professional resources, 

and investment funding to early stage technical businesses. The organization is 

comprised of three segments: Technology Park, Research and Development Group, and 

INNOVA Commercialization Group.

WVHTC INNOVA Commercialization Group (INNOVA) is a venture development 

organization focused on technology. It manages the primary startup-stage capital 

investment fund for entrepreneurs and new enterprises.

Along with preparing growing companies for future venture capital investments, INNOVA 

also determines and provides the appropriate resources and assistance a new business 

needs to evolve. It delivers a variety of entrepreneurial, educational, training, and 

networking events to start and grow a nascent business.

The length of program service varies from approximately six months for companies that 

are almost ready to enter the marketplace (and only require completion of paperwork and 

legalities), to roughly seven years for companies that require more extensive assistance. 

Depending on the situation, services over time can be curtailed or more ongoing. 

Continuing services may involve INNOVA taking a seat on a client company’s board of 

directors. According to program staff, INNOVA stays involved until the company is solidly 

entrenched in the marketplace, i.e., “a going concern.” Once the appropriate loans are 

paid off, INNOVA then drops out of the picture.

 

98 County Economic Status, Fiscal Year 2005,Appalachian West Virginia. 

http://www.arc.gov/reports/region_report.asp?FIPS=54999&REPORT_ID=26

99 http://www.arc.gov/research/SourceandMethodologyCountyEconomicStatusFY2002FY2005.asp

http://www.arc.gov/research/SourceandMethodologyCountyEconomicStatusFY2006.asp

100 County Economic Status, Fiscal Year 2012,Appalachian West Virginia. 

http://www.arc.gov/reports/region_report.asp?FIPS=54999&REPORT_ID=37
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

September 2015 | 145

6.13.3 Project Planning and Implementation

Beginning in the 1980s, West Virginia’s economic lynchpins, the coal industry and 

traditional manufacturing, were in decline. It became obvious to Senator Robert C. Byrd 

and Congressman Alan B. Mollohan that diversification of the state’s industries was 

required. A technology park was developed in Fairmont, a relatively centralized location 

in state. 

The intention was to use models such as Innovation Works in Pittsburgh101 and 

JumpStart in Ohio102 to develop a ‘software valley’ in West Virginia. Five federal 

agencies, including the Criminal Justice Information Services of the FBI, the National 

Energy Technology Laboratory, The National Aeronautics and Space Administration 

(NASA), The National Institute for Occupational Safety and Health (NIOSH) and the 

Department of Defense, Biometrics Fusion Center (BFC) were located within this 

technology park.103 These federal entities provided a fertile launch environment for 

promising West Virginia small businesses to subcontract to these federal agencies.

By the 1990s, the need for an organization to assist these inexperienced small 

businesses became apparent. The talented engineers and technical persons who 

founded these enterprises often lacked a basic background or experience in business. 

Funding became available for a resource to provide business development and formation 

expertise along with legal, accounting, and financial assistance. Funds for small early 

stage investments were also on hand, including a 2002 ARC grant to set up and staff the 

organization.

These businesses also required risk capital. If the business founders’ funds became 

exhausted before the company was mature enough to obtain more formal sources of 

capital, these companies could fail. Seed stage investment funds were needed to sustain 

them until more formal and larger investments could be obtained. New high technology 

ventures require knowledge and funding to grow; they require management and financial 

capital-- both difficult to obtain in early stages of a business’ development.

ARC was one of the earliest supporters of INNOVA. The 2005 funding, combined with 

funds from the earlier ARC grant, allowed statewide reach and the opportunity to serve 

small businesses or anyone with an idea, in a structured fashion. It also allowed INNOVA 

to leverage additional funds. With those funds, INNOVA provided finance professionals, 

marketing, operations experts, and technical assistance, either directly or from third 

parties at reduced costs.

INNOVA program administrators were unable to recall obstacles in the implementation of

the program. They also reported the planning process was straightforward.

6.13.4 Economic and Community Impacts

ARC funding facilitated additional investment which made it possible to reach the number 

of individuals the program ultimately assisted. The funds also helped match program 

 

101 https://www.innovationworks.org/

102 http://www.jumpstartinc.org/

103 North-Central West Virginia’s Technology Industry: A Pathway Through the 21st Century. Anderson Economic 

Group, LLC, 2006.
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

146 | September 2015

funds from other programs, as discussed below. The grant stipulated assistance be 

provided to high tech companies only. Nevertheless, INNOVA looked beyond high 

technology companies and used its expertise to assist other companies by referring them 

elsewhere or introducing them to their network of groups that could provide assistance. 

WVHTC staff uses its knowledge of West Virginia to access and employ statewide 

resources.

In January 2012, INNOVA, the West Virginia Jobs Investment Trust (WV JIT), and the 

Natural Capital Investment Fund (NCIF) began participation in the West Virginia Capital 

Access Program (WV CAP), an investment program made possible through the U.S. 

Department of the Treasury State Small Business Credit Initiative (SSBCI) funding. This 

program provides West Virginia access to $13.1 million in federal dollars to fund new 

small business investment programs. Based upon projected demand, 58 percent of the 

total funds ($7,651,010) are allocated to the Seed Capital Co-Investment Fund to be 

invested by INNOVA, NCIF and WV JIT. All funds must initially be invested within two 

years. This increased deal flow also brings with it increased requirements for Technical 

Assistance to ensure “investment readiness” and due diligence prior to investment, and 

loan and financial packaging.

The observation was made and upheld that through statute, ARC funds may be used as 

a match for other federal funds. INNOVA queried if this would be the case for the federal 

SSBCI program. With the assistance of Charles Howard, ARC General Counsel, a case 

was presented to the U.S. Department of the Treasury. The subsequent positive 

affirmation from U.S. Treasury allowed three of the eight WV CAP participants to 

enhance their pool of investment funding available under the program by freeing up ARC 

funds for utilization as approved matching funds.

Over the course of the program, $1.8 million in additional funds were raised from the WV 

Economic Development Authority, the Claude Worthington Benedum Foundation, 

INNOVA internal funds and the WV Capital Access Program, funded by the U.S. State 

Small Business Credit Initiative, SSBCI).104 Portions of each, totaling $893,825, were 

directly used for investment alongside $536,000 of ARC investment funds. If ARC funds 

were not available, INNOVA believes it would have been considerably more difficult to 

attract additional investment, supportive investment, and reach the number of people it 

did through the matching program to SSBCI. 

The availability of INNOVA services and funding reached companies through word-ofmouth communication and referrals from other organizations. In essence, companies 

and individuals contacted INNOVA for assistance. As funding levels increased, so did 

INNOVA’s opportunities to assist, resulting in more companies using INNOVA’s 

resources.

Driven by ARC funding, the north-central West Virginia technology industry had a total 

economic impact of $5.2 billion on the state.105 In addition, it fostered direct construction 

economic impacts of more than $111 million and indirect economic impacts of over $200 

 

104 The West Virginia Capital Access Program (WVCAP) is the state’s program designed to increase small business 

access to capital. The program was announced in December 2011 when the United States Department of the 

Treasury approved the state’s application for State Small Business Credit Initiative (SSBCI) funding.

http://www.wvcommerce.org/info/west-virginia-edge/wv-edge-issue1-012/small_businesses_score_big/wvcap.aspx

105 Ibid.
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

September 2015 | 147

million. Other benefits resulting from the projects included more stable regional 

employment, an increase in educational levels and income, lowered unemployment 

rates, and additional research activities and funding. While the program predicted serving 

200 companies, it assisted more than 500 companies, 350-400 during ARC’s funding 

period.106

The INNOVA investment funds made it possible to approach other funding groups and 

foundations for additional funds. These dollars were spent on supplementary services to 

assist the client businesses such as legal, patent protection, accounting, marketing, and 

finance. Approximately $100,000 was used to offset professional services costs and get 

the small businesses “investment ready.” ARC funds made it possible to approach and 

attract additional resources to get support funds from foundations. Though ARC funds 

could not be used for direct labor expenses, more employees could be brought on by 

leveraging the monies for professional services, leading to “exponential growth,” 

according to program officials. 

INNOVA invested in 22 companies and provide services to additional companies which 

allowed them to enter new markets. Examples are available on INNOVA’s website. 

Program administrators acknowledge that initial job creation projections were overly 

optimistic. However, the program had significant impacts on individuals and businesses 

during the funding period. The following data are conservative, based on the time of 

investment only, and not five years forward as some other projects do: 

• 22 investments in companies 

• 45 full-time jobs created 

• 8 part-time jobs created

• 144 jobs retained 

The program has experienced significant growth in demand as demonstrated by an 

increasing number of applicants for assistance. To date, INNOVA has worked with more 

than 400 companies, placed over $1.6 million of direct investments into 17 West Virginia 

companies, is recognized as a primary source of seed capital in the state and has raised 

about $2.3 million in its seed stage investment capital fund.107

6.13.5 Lessons Learned

The lessons learned from this program are drawn from interviews of program 

administrators and report documents either provided by program administrators or 

located online. 

• Collaboration and communication among investment groups can 

improve efficiency. When individuals and companies apply to multiple 

funding agencies, collaboration and communication can facilitate the grant 

application process for potential grantees. Processes can be integrated 

whereby applicants need only complete one common funding application 

rather than multiple detailed applications, much of which can be redundant. 

 

106 Program Director Interview

107 http://www.wvhtf.org/businessdevelopment/innova.php
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

148 | September 2015

Collaboration also aids funders by identifying organizations in the grant 

application pipeline and the amount of assistance each requires to become 

self-sufficient. It allows funding agencies to coordinate and distribute their 

tasks to more efficiently provide grantees the funds needed to bring them to 

marketplace and stand on their own. 

INNOVA also provides workspace for two other support organizations in its 

facility. With resources available under one roof, these organizations can 

take a company from initial development through the investment stage – all 

at one time. The key is collaborating to efficiently use time and resources and 

ultimately service more companies.

• ARC funds can leverage additional funds from private investors. The 

power of leveraging funds became apparent to INNOVA as it was able to use 

ARC funds to get other funding agencies to also step up to the plate. The 

ability to use ARC as matching funds doubled available resources.

• Funding direct labor can be problematic. Because of various restrictions, 

finding the funds to directly hire personnel is a challenge to funding agencies.

• Many nascent high technology companies require assistance to 

navigate challenges before their final product is ready for market.

Professional and technical services represent a most important opportunity 

for assistance. There is considerable demand for assistance among new 

high-tech entrepreneurs and businesses before they become self-sufficient. 

As one program official stated, “There are so many opportunities and few 

resources.” For many new enterprises, intensive due diligence in the 

application process and mentoring from experienced experts creates a 

strategic path to commercial success.

6.13.6 Conclusion

West Virginia’s High Technology Consortium Foundation’s INNOVA Commercialization 

Group is an essential contributor to the state’s tech-based economic development 

movement. It has been a catalyst for high technology growth and innovation, helping 

many start-up businesses succeed by providing critical missing elements such as 

commercialization assistance services, seed-stage investment capital, and 

entrepreneurial education programs, training, business support services and networking 

events.

6.14 Kentucky Highlands Energy Business Boot Camp

The purpose of the project was to assist new, renewable or alternative energy-related 

entrepreneurs, researchers and inventors move towards successful businesses and 

sustainability. The Kentucky Highlands Energy Business Boot Camp offered intensive 

performance-based entrepreneurial training and mentoring programs for these entities to 

develop profitable, fundable and sustainable business models and create employment 

opportunities in the region. This program was classified as an Energy/Environmental 

project.
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

September 2015 | 149

6.14.1 Community Profile

Kentucky Highlands Investment Corporation (KHIC) was founded in 1968 to stimulate 

growth and jobs in a nine-county region of Southeastern Kentucky. KHIC’s mission is to 

provide and retain employment opportunities in Southeastern Kentucky through sound 

investments and management assistance.108 In 2003, KHIC expanded its service area to 

twenty-two counties. It includes the counties of Bell, Clay, Clinton, Cumberland, Estill, 

Harlan, Jackson, Knox, Laurel, Lee, Leslie, Letcher, Lincoln, Madison, McCreary, 

Owsley, Perry, Pulaski, Rockcastle, Russell, Wayne and Whitley.

In 2012, 54 counties in Kentucky were part of the Appalachian Region, a 205,000-

square-mile region that follows the spine of the Appalachian Mountains from southern 

New York to northern Mississippi. Each of the 22 counties covered in KHIC’s expanded 

service area were included in the 54-county ARC legislation. Employment in the area 

relied heavily on coal mining – more than twice the dependency of ARC counties in West 

Virginia and Pennsylvania.109 This strong reliance was compounded by a lack of job 

opportunities in other fields.

Employment in the coal industry had been in decline for years. By 2008, coal mining 

accounted for only one percent of Kentucky employment. Even in Eastern Kentucky 

counties, mining jobs ranged from three to 23 percent of the employment base and faced 

substantial long-term unemployment and poverty rates exceeding thirty-three percent.110

The region was challenged economically when the grant was awarded. The 2012

population of the Kentucky Appalachian region was about 1.2 million residents. These 

residents realized average per capita income of only $28,928, about 66.1 percent of 

national and 81 percent of both state and the entire Appalachian Region in average per 

capita income.111

ARC assigns each county one of five possible economic designations – distressed, atrisk, transitional, competitive, or attainment- with distressed counties being the most 

economically endangered and attainment counties being the most prosperous. In 2012, 

three-quarters (76 percent) of the 54 counties in Appalachian Kentucky were designated 

distressed, the lowest economic status. The remainder were defined as at-risk (17

percent) or transitional (7 percent). Not one of the counties in Appalachian Kentucky 

received either of the two higher economic status ratings.112

6.14.2 Project Description

The Kentucky Highlands Energy Boot Camp was an intensive, entrepreneurial 

performance-based training and mentoring program for developing businesses. It 

attracted entrepreneurs, law makers, public agency officials, angel investors, energy 

 

108 Kentucky Highlands Investment Corporation. About-khic. www.khic.org

109 Roenker, Jonathan M. The Economic Impact of Coal in Appalachian Kentucky. 

http://cber.uky.edu/Downloads/Roenker02.htm

110 The Impact of Coal on the Kentucky State Budge, Executive Summary. http://www.maced.org/coal/exesummary.htm

111 http://www.arc.gov/reports/region_report.asp?FIPS=21999&REPORT_ID=53

112 http://www.arc.gov/reports/region_report.asp?FIPS=21999&REPORT_ID=37
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

150 | September 2015

business researchers and the public to its two symposia, one at the end of each energy 

business training cohort. The program was conducted in conjunction with Technology 

2020’s Center for Entrepreneurial Growth.113

Program goals included creating a fundable, sustainable and profitable business model 

to generate jobs in the region. There were no fees to participating businesses, each 

receiving stipends to offset costs associated with classroom participation, ongoing 

strategic planning, field exercises, marketing execution, travel and time away from their 

businesses.

Training incorporated classroom learning, coaching meetings, networking events and a 

regional energy symposium. ARC funds underwrote contractual costs. Companies were 

required to attend four half-day training sessions focused on building a sustainable and 

fundable business. Mentoring and materials were provided to complete program 

assignments. Attendance at networking or market-specific events was compulsory, with 

stipends covering travel and costs provided by the program.

Companies were obliged to prepare presentations describing essential components of 

their businesses. These were used at the symposium and meetings with partners, capital 

sources, local, state and regional organizations. Businesses met with a marketing firm to 

develop materials and later present an overview of their company at the symposium. 

They were tasked to develop a marketing plan, identify target markets, pinpoint partners 

or channels to deliver their sustainable business model and develop a financial plan and 

capitalization strategy. Businesses were coached and required to apply for applicable 

grants and programs pertaining to their business. 

The program sought to identify businesses with the highest likelihood of success. A 

maximum of six slots were available per round. Companies were vetted through a 

screening process that included accomplishments or ideas, background biographies and 

resumes, references, on-site visits and interviews. Two additional individual 

entrepreneurs were invited per session and received a limited stipend amount.

Initially, the program expected to conduct two Boot Camps with four to six participants at 

each, yielding eight strategic plans. It hoped to create or sustain 12 to 16 jobs over 

roughly the subsequent two years. Four networking events were planned with each Boot 

Camp attracting 24 participants at each of two symposia.

6.14.3 Project Planning and Implementation

In 2006 and 2007, Kentucky Highlands Investment Corporation (KHIC) started to see the 

necessity and opportunity for economic diversification in the Kentucky Highlands region. 

While the local economy depended heavily on mining, KHIC recognized that forward 

thinking would be required in order to diversify. They wanted to fill a pipeline with new 

businesses that would be sustainable and profitable while also creating jobs. They saw a 

latent ability within the service area that could tap into the energy sector. KHIC wanted to 

commit to entrepreneurial development so they paired up with venture development 

organization Tech 2020, which was interested in helping small business owners turn 

 

113 Created in 1995, Technology 2020 is a public-private partnership whose mission is to grow new businesses and 

high quality jobs by capitalizing on the unique technology resources of the Tennessee Valley Corridor and to 

advance the region’s reputation as a nationally recognized center of technology-based entrepreneurship. 

http://www.tech2020dev.org/
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

September 2015 | 151

ideas into a reality. The companies had a previous relationship, both recognized the 

need, and together had the resources to create change.

KHIC and Tech 2020 wanted to develop an accelerated program to help launch 

businesses, and developed the plans for the Energy Business Boot Camp. While 

originally planned to last 10 to 12 months, it was eventually decided that a six month 

program had a better chance of attracting participants. They also added a $7,500 stipend 

to attract participants: an incentive for them to get out of the office and network. KHIC 

planned two Boot Camps with six participants each, with each camp including two 

networking events and a concluding symposium at the end. They allowed themselves 

two months for the marketing and selection process, and slated the first camp to start in 

November 2007.

The cost estimates for the project were based on similar past KHIC investments. KHIC 

negotiated with Tech 2020 and researched what granting agencies were supporting such 

projects. They estimated the cost to be $225,000 for the program over 18 months. The 

cost was split three ways between ARC funding, the Kentucky Cabinet for Energy, and 

KHIC investment. As the Kentucky Cabinet for Energy’s grants are limited, and KHIC 

limited the amount they could spend on the Boot Camp, ARC funding was crucial. This 

project would not have been undertaken without the grant. 

KHIC used several techniques that they had used in the past to ensure the project’s 

success. The experienced KHIC staff understood the importance of having the right 

people in the right roles to provide the highest level of expertise and understanding for 

the corresponding project. Bob Wilson took the lead on this project, leveraging his years 

of experience leading entrepreneurial companies, contacts within Tech 2020, and energy 

expertise and knowledge. Project leaders also knew, however, that while people are 

important, the project design is important as well. Wilson’s experience allowed him to 

lead a team in order to create a solid plan that, in conjunction with the people running it, 

led to a successful project. KHIC had a reputation for taking on challenging projects, but 

had gained respect for constantly improving and taking responsibility for any mistakes. 

This led to strong relationships with other organizations, including ARC and the Energy 

Cabinet. By keeping lines of communication constantly open, KHIC built strong and 

trusting connections. They kept agencies apprised of the project through regular phone 

calls, as well as bi-yearly visits to the Cabinet to update them, thank them, and explain 

any discrepancies. KHIC also had a lot of encouragement from their own board, which 

allowed them to step out and try something new while knowing they were being 

supported. 

While the Boot Camps were executed successfully with overwhelmingly positive results, 

there were a few challenges KHIC faced along the way. The main obstacle was 

recruiting entrepreneurs to participate, as many did not want to travel and take time away 

from their companies. As 22 counties were included in the recruitment pool in order to 

ensure enough participants, and some locations were as much as a four-hour drive 

away, the stipend was not always enough to entice people to sign up. KHIC understood 

that, as they were recruiting from a generally older demographic, they would need a 

more “boots on the ground” approach to appeal to the targeted entrepreneurs, as 

opposed to using technology and relying on electronic communication. They sent 

representatives to the properties of prospective participants in order to build a trusting 

relationship, as well as relying on their reputation as a legitimate company with 
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

152 | September 2015

connections and resources. KHIC also overcame budgetary restrictions by cutting down 

on unnecessary and redundant costs, as well as by closely considering what costs were 

most important in reaching the goals of the program. In the end, the program came in onbudget and the Boot Camps were successful.

6.14.4 Economic and Community Impacts

The Energy Business Boot Camps achieved both measurable and immeasurable 

success. Both Camps ran smoothly and with overwhelmingly positive feedback from the 

participants. Both of the keynote speakers were highly successful and recognized 

leaders in their fields, and the networking events were beneficial to all who participated. 

As for the goals set during the planning stages, all projected outcomes were either met 

or exceeded. 

Participants went on to apply what they had learned during the Camp to their 

businesses. Survey feedback said that participants had never had such an effective, 

intensive, and beneficial assimilation of good information and networking experiences in 

such a short time, and that it really propelled their thinking about business. Participants 

won $3.36 million in leveraged private investment. Much of this was from grants, which 

was improved during the grant writing seminars during the Camp. Moreover, eight out of 

ten participants were successful in receiving loans for their businesses. 

One standout participant was Nathan Hall, who went on to receive the Thomas J. 

Watson Fellowship which funded a year of study around the world. Hall travelled the 

globe researching the use of biofuel internationally and developing his research on 

switch grass and forest sustainability as a means to develop fuel. He now speaks about 

his travels and research, leading and inspiring other entrepreneurs in the field of energy. 

Another outstanding product of the Boot Camps was Genesis Development, Kentucky’s 

first green energy company that provides and educates on alternatives to coal. Countless 

other participants went on to incorporate the green energy knowledge into their 

businesses, succeeding in creating opportunity for economic diversification in the 

Kentucky Highlands region and creating new business that is sustainable and profitable. 

Ten businesses were improved from 2007 to 2012, seven created and three retained, 

according to program officials. A dozen jobs were created and 12 more retained. More 

than $3.36 million in leveraged private investment was realized. In addition, $1.714 

million in energy technology grants was awarded to participating companies.114

Companies heavily involved in research and development were able to increase their 

cash assets as a result of this program. It also increased the diversity of businesses in 

the region.

6.14.5 Lessons Learned

Lesson learned were collected from interviews of program officials.

• Understand and adapt to your target demographic. During the recruitment 

process, KHIC had a specific demographic in mind for participants in the 

Boot Camps. Problems arose, however, as they realized this slightly older 

 

114 Program Survey
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

September 2015 | 153

and more traditional group preferred in-person communication, made difficult 

due to the large geographic spread of the participating region. By 

understanding what would build this group’s trust, and by changing their 

methods of recruitment, KHIC managed to successfully recruit enough 

participants for the program.

• Communicate openly and honestly with funders. Through consistent and 

transparent communication, KHIC managed to build strong and lasting 

relationships with funders. KHIC had regular phone conversations as well as 

in-person meeting where they shared not only progress, but also any 

setbacks. This created a trusting relationship that allowed the project to 

recover from setbacks and have successful end results, as well allowing for 

more potential funding into the future.

• Cut costs by keeping the end goals in mind. While KHIC had originally 

planned for a much larger budget, they realized much of the planned 

spending was unnecessary. By only spending what was necessary to 

achieve the goals of the project and cutting out all frivolous spending, the 

project managed to come in on-budget while keeping funders within their 

constraints. 

• People are important, but so is project design. As an established 

organization, KHIC had many experienced employees within the company. 

By selecting leaders with relevant and substantial experience, such as Bob 

Wilson, the project was provided the significant advantage of years of 

accumulated knowledge. It was not only having the right people on board, but 

having them in the right seats that led to the creation of a solid and 

achievable project design that was certain to realize success.

6.14.6 Conclusion

The program created a vibrant network of energy-related entrepreneurs and partners 

working to meet market needs in this economically challenged region of Kentucky. 

Success was advanced by including a stipend allowing participants to get away from 

their workspaces to attend the Boot Camp and related events and to complete program 

requirements. This was a necessary and important inducement for participants and 

would have been unlikely, or may have not occurred at all, without ARC funds.

6.15 North Carolina Department of Commerce FY 2010-

2011 Wood Products/Home Furnishings Initiative

In 2010, ARC awarded an amended business development grant to the International 

Trade Division, State of North Carolina Department of Commerce for $25,000 to expand 

trade opportunities for North Carolina and other states in the Appalachian Region. The 

program was designed to promote the Asia/Pacific Rim initiative to assist companies 

export wood products, manufacturing supplies and home furnishings. 
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

154 | September 2015

ARC funds from this Business/Training grant were used to purchase booth/pavilion 

space at the China International Furniture Manufacturing and Raw Materials Exhibition 

(FMC China). ARC funds totaled $75,000.

6.15.1 Community Profile

Twenty-nine counties in North Carolina are part of the Appalachian Region, a 205,000-

square-mile region that follows the spine of the Appalachian Mountains from southern 

New York to northern Mississippi. 

The 2012 population of Appalachian North Carolina was just over 1.7 million residents 

who realized average per capita income of only $34,215, approximately 78 percent of 

national, 90 percent of North Carolina and 95 percent of all Appalachia in per capita 

income. Average per capita income by county ranged from 62 percent of U.S. average in 

Graham County ($26,988) to 92 percent in Polk County ($40,232).115

ARC defines the economic status of each Appalachian county with one of five possible 

economic designations based primarily on three indicators: three year average 

unemployment rate, market income per capita, and poverty rate. The statuses are 

distressed, at-risk, transitional, competitive or attainment. Distressed counties are the 

most economically endangered and attainment counties are the most prosperous. All 

Appalachian counties in the state but one was defined either as at-risk or transitional, the 

second lowest and middle ratings. The exception was Graham County which received a 

distressed designation, the lowest possible classification. In FY2012, none of the 

counties in the region received either of the two higher economic status designations of 

competitive or attainment.116

Only three counties in Appalachian North Carolina experienced unemployment rates 

below the national average in 2012. That year, the unemployment rate in Appalachian 

North Carolina was 9.4 percent. It exceeded both the national and entire Appalachian 

region’s unemployment rate of 8.1 percent and was about equal to the rate for the entire 

state. Unemployment rates in the individual ARC counties ranged from 7.3 percent in 

Henderson County to 16.8 percent in Graham County.

6.15.2 Project Description

The program goal was to recruit 14 to 20 wood, supply and home furniture companies 

each year to participate in the trade shows held at the Appalachian Pavilion of the China 

International Furniture Manufacturing and Raw Materials Exhibition (FMC China) or its 

earlier incarnation Interzum Guangzhou. These shows continue to be the largest and 

most comprehensive woodworking machinery and accessories trade fairs in all of Asia. 

FMC China provides an international sourcing platform for buyers and manufacturers, 

covering one million square feet. Small businesses wishing to attend on their own do so 

at considerable expense. 

The grant helped to offset costs to participating businesses. ARC funds were used to 

purchase the pavilion and floor space, significantly cutting down on individual costs. The 

 

115 United States Department of Commerce, Bureau of Economic Analysis, Local Area Personal Income, 2012. 

Compiled by the Appalachian Regional Commission, March 2014.

116 County Economic Status Fiscal Year 2012, Appalachian Regional Commission.
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

September 2015 | 155

cost of the pavilion can range from $2,500 to $40,000, according to program officials. 

Travel for the grant team is state-funded and applied to the required support match. ARC 

funds covered half the required participation fee. Participating companies pay the other 

half, which goes towards the required matching funds. Freight, lodging, food and 

transportation costs are also divided between the businesses and ARC.

ARC’s Export Trade Advisory Council (ETAC) office works as a team to promote the 

show in ARC states and encourage attendance by companies from the region. As grant 

participants, companies have an opportunity to learn about overseas markets without 

having to manage the somewhat intimidating logistics of international travel on their own. 

Without ARC support, at least some of the participating businesses may not have been 

able to afford overseas travel and therefore denied the opportunity to learn first-hand the 

potential benefits of expanding into this market.

The ARC-sponsored Appalachia USA Pavilion at the 2010 Interzum Guangzhou home 

furnishing show presented the opportunity for companies in Appalachian counties to 

participate while mitigating financial risk to their businesses. One goal of the program is 

for companies attending with ARC support to recognize the return-on-investment 

opportunities of international trade and, in the future, choose to fund their subsequent 

participation without ARC assistance.

A catalog of various services provided by the team to participating companies included 

availability of an on-site interpreter, pre-arranged appointments with potential customers, 

travel logistics from the Appalachian region to Shanghai China, and product shipment, 

among others. The North Carolina representative headquartered in the Hong Kong office 

handles much of the marketing, and provides additional services to participants such as 

translating business cards, brochures and marketing materials. Costs incurred by 

attending businesses count as part of the matching requirement. Travel for the state of 

Mississippi staff contingent also counted towards the match.

6.15.3 Project Planning and Implementation

The program was conceived by ARC’s Export Trade Advisory Council (ETAC), a panel 

composed of one or two members from each state in the Appalachian Region with 

expertise in exporting. Approximately 10 years ago, this council considered divergent 

sectors affecting multi-state regions. It concluded lumber, home furnishings and 

manufacturing were common and prevalent to the regions. The wood and home 

furnishing sectors were chosen for a pilot program whereby these products were 

exhibited as a unit at international trade shows. ETAC unanimously approved the 

program which became a model for other states, products and sectors. The trade office 

of the North Carolina state government volunteered to lead the project, as they had an 

office overseeing these products.117 The grant focused on the Asia/Pacific Rim where 

hardwoods and softwoods from the region were previously established in the market. 

The grant was supervised by the trade manager from the North Carolina Furniture Export 

Office headquartered in Highpoint, NC, where many furniture manufacturers were 

located and by the North Carolina Asia Pacific office located in Hong Kong. North 

Carolina sends its furniture representative to the show, at the state’s expense. For the 

 

117 Note that some other states, including Mississippi, also had offices overseeing these products.
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

156 | September 2015

wood and home furnishing sector, the funds go exclusively to North Carolina. Other 

states have taken the lead for additional business sectors, such as mining, though North 

Carolina remains the standard for the more recent programs, having successfully grown 

its program over a multi-year period. 

The first of these ARC grants was executed in September 2008, to support a regional 

Asia/Pacific Rim export initiative featuring Appalachian furniture/home furnishings and 

wood/forest products. ARC provided $25,000 with a required match of $100,000 in cash, 

contributed services or in-kind contributions from program participants. The funds were 

used to support the delegation to the China International Furniture Fair/Interzum in 

Guangzhou, China in March 2009. 

In 2009, the award was amended. ARC provided an additional $25,000 funding support 

with a required additional match of $50,000, bringing the total match to $150,000 through 

September, 2010. Again, funds supported the delegation attending the Interzum/China 

show in Guangzhou, China in March 2010. 

The third year of funding realized an amendment in August 2010 for an additional 

$25,000 from ARC, bringing its total contribution to the program to $75,000. Matching 

funds increased by $50,000 setting a total program match requirement of $200,000. This 

time the grant supported a delegation to the Furniture Manufacturing & Supply China in 

Shanghai, China in September, 2011. ARC funds helped reduce exhibition costs for the 

participating Appalachian businesses. A 27-member delegation representing 

Appalachian furniture and wood product businesses attended the show, generating more 

than $36 million in new Appalachian export sales.118

The grant agreement was amended once again with additional ARC financing of $25,000 

from July 2011 to the end of 2012. This brought ARC total funding to $100,000 and 

increased the matching funds by $25,000 to a total of $225,000. It supported the 

delegation to Furniture Manufacturing & Supply China in Shanghai, China in September 

2012.

One program official likened the program administration to a “well-oiled machine.” No 

specific obstacles were reported by program officials either in the planning or initiation of 

the grant. Program managers noted that the multi-year program runs smoothly and that 

ARC staff is particularly easy to work with and available to solve any issues that occur.

6.15.4 Economic and Community Impacts

The program works not only for the benefit of North Carolina but for the good of the 

entire ARC region. It is an exceptional example of cooperation among ARC states. For 

instance, data are shared among the state delegations, including lead generation, sales 

information, and other business metrics. The grantee collects, analyzes and distributes to 

participants the number of leads generated, immediate sales from the show, contracts 

signed and anticipated sales in the subsequent 12 to 18 months based on who the 

representatives met at the show and other factors.

In 2009, expected sales totaled $14 million over the subsequent 12 months. In 2010, 

floor sales reached $200,000, with expected sales of $8.5 million. More than 4,200 

 

118 ARC News Brief, September 2011, Appalachian USA Delegation Attends Shanghai Trade Event.
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

September 2015 | 157

containers of product were sold. Assuming an average container value of $10,000, $42 

million in sales were realized, pushing total sales for the year to exceed $50 million.

In 2009, the grant supported travel to two shows in China Guangzhou. The first was 

attended by three companies (and/or states); the second by 13. Each company had its 

own built-out space in the sizable pavilion (roughly 2,600 square feet). North Carolina 

and Mississippi shared space at this show, but now each has its own designated area. 

Exhibit space can also be divided by company, client or distributor as needed. In March 

2010, 14 companies attended the shows; in 2011 there were ten. Private areas are also 

provided, allowing business representatives to negotiate contracts, logistics and other 

business issues.

China has become North Carolina’s second largest export market with sales in excess of 

$2.8 billion in 2013.119 The nation has been one of North Carolina’s top growth markets 

for the past 10 years, generating tens of millions of dollars in sales. 

Businesses are quick to realize the opportunities inherent in this market. Officials note 

that even before the show ends, many novice companies attending the shows indicate 

interest in returning the following year. However, the grantee has set a program goal that 

once companies attend with ARC support and are introduced to this market they attend 

on their own in subsequent years. Based on many years of experience, program officials 

report that most companies would not delve into these international markets without ARC 

funding to reduce their costs, nor would they be willing to manage these considerable 

and complex international logistics on their own.

6.15.5 Lessons Learned

Lessons learned were gleaned primarily from interviews of program officials.

• Maintain a representative on site. Though the show is held in Shanghai, 

the director of the program stationed in Hong Kong plays a critical role in 

managing the logistics for clients. The added expense of the director’s travel 

to the show is well worth the cost, program officials agreed. North Carolina 

program staff has strong relationships with all export partners that serve the 

state. If a point-person has any difficulties with a client, state officials are 

available to intervene. Relationships are essential, and it is likely that not all

states have fostered these kinds of connections.

• Realize the work and financial commitment involved in managing 

international programs. Should a new agency be appointed to spearhead 

this type of program, administrators should have an understanding of the 

amount of work involved. Plus, they should realize this can be considered “an 

unselfish act” on the part of a state, as the benefits accrue beyond their state 

boundary, while any costs are incurred by the state alone. For instance, if no 

companies participate, the state is still responsible for costs of the pavilion 

and space. The commitment is considerable, as several individuals both instate and overseas work on these programs year round. Gaining government 

support for a project that “benefits the greater good” rather than one’s own 

state is sometimes difficult to achieve in state legislatures. 

 

119 U.S. Exports to China by State, 2004 to 2013. The U.S.-China Business Council.
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

158 | September 2015

• Find the right mix of people to work on these programs. The skills 

involved in managing and running these programs go beyond international 

trade. To be successful, the expertise of the staff must cover a wide range of 

skillsets.They include marketing, business logistics, and other areas of 

expertise.

6.15.6 Conclusion

The logistics of international trade can be intimidating, especially to small- and mediumsized businesses new to overseas sales. This grant successfully facilitated small- and 

medium-sized businesses in ARC states increase their exports to China and the Pacific 

Rim. Companies were encouraged and assisted in marketing their products at one of the 

largest furniture and wood products show in Asia, held in Shanghai, China. By providing 

business assistance, education about opportunities, and logistical support, this longrunning program facilitated businesses’ return-on-investment through international trade.

The program provided across-the-board assistance to existing new-to-export companies, 

many not aware initially of the potential impact of international sales on their businesses. 

It encouraged entry into new markets by paving the way to attend the China show at 

reduced cost and risk to their businesses. Benefits accrued to companies in all 

Appalachian states, not just in North Carolina.

6.16 Key Lessons Learned

The “Lessons Learned” section can be a beneficial tool for project managers developing 

comparable future projects. It summarizes shared problems encountered in the course of 

the programs and how the managers were able to resolve them. Future administrators 

may avoid similar hurdles that may occur in the course of upcoming projects by applying 

these lessons to their unique situations.

The Key Lessons Learned were compiled by reviewing the 15 individual case studies. 

They harvest shared experiences from the unique programs that seem to overlap a 

number of projects. 

The lessons from ARC’s Job Creation and Retention Program Evaluation fall into four 

categories: Community engagement, program flexibility, outcome measures, and the 

grant process. The Key Lessons Learned are presented below.

6.16.1 Engage, Educate and Involve the Community

A number of program administrators stressed the importance of securing community 

support and goodwill for their programs through engagement. They noted community 

buy-in for programs can be reinforced in many ways. These include tapping into regional 

pride; awakening its sense of history, evoking shared needs and underscoring economic 

opportunities. Sometimes involvement in a program can be encouraged by reminding the 

community that their taxes fund the program and consequently it is their responsibility to 

insure good stewardship of those dollars. 

In the City of Toccoa, Georgia, for example, the community rallied around a vision for 

downtown revitalization based on the City’s historical significance and community pride. 

Using town hall meetings, public hearings, pamphlets, newsletters presentations and 
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

September 2015 | 159

mockups, citizens were kept informed of each phase of construction. Soliciting input on 

the project and inviting participation in ceremonial events fostered community 

engagement and support for the program.

Sometimes a program encounters resistance, compelling administrators to supply 

additional justification and instruction before the community champions the agenda. The 

Heartwood: Southwest Virginia’s Artisan Gateway faced initial opposition from 

established artistic organizations and attractions that perceived the venue as competition 

for tourist dollars. Program administrators were able to prove these skeptics wrong, 

garnering support for the program by assuring local artists of the mutual benefits 

Heartwood could provide, including opportunities for cross promotions and as a staging 

area to visit local artisans. Local officials who were at first skeptical that Heartwood could 

serve not only the county in which it was located, but multiple surrounding counties were 

also persuaded by its success.

In New York, the Finger Lakes ReUse Center involved partners and stakeholders 

beginning with the initial planning process, building support based on the mutual 

environmental and economic needs of the community. Community involvement was also 

a key factor in the success of the Maryland Great Allegany Passage (GAP) Historical 

Trail Program which built support by engaging locally invested leaders and grant 

administrators early in the grant process.

6.16.2 Change is Good – Be Flexible

Program attainment can occur even with mid-course modifications. Based on 

observations by program administrators, the New York Finger Lakes ReUse Center 

changed program priorities from environmental and economic bases to a program which 

focuses on so-called human experiences. 

A number of ARC funded academic programs improved, added to or modified curricula 

based on input from experts in the field. The results were graduates better prepared to 

join rapidly changing job markets. By tapping existing market knowledge and technical 

experience, program offerings were broadened, attracting students from additional fields 

of study. 

The GAP Economic Development Program in Maryland modified its business certification 

program mid-course by combining two processes making it more efficient, popular and 

successful. 

In Kentucky, the Energy Business Boot Camp changed its methods of recruitment to 

attract the target demographic and maintain the group’s trust. The program also cut costs 

and spending to achieve the project goals.

6.16.3 Mind Your Measures

It can be challenging to determine where one program begins or ends, and whether a 

specific program generated identifiable outcomes. In a number of instances, programs 

and funding cycles overlapped, making it difficult for grantees to specify outcomes 

resulting solely from ARC funding. In some cases it was difficult for administrators to 

determine if outcomes were the result of programs funded by more than one agency, 

during multi-year funding cycles, or began before ARC’s grant period got underway. Staff 
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

160 | September 2015

changes exacerbated these issues and program history was lost as individuals intimately 

involved in the programs were not available to interview. 

So called “soft” outcomes, like “community pride” or “historical interest” are difficult to 

quantify and measure. It can also be problematic to agree on the variables to tally when 

measuring concepts like “the growth of tourism.” In addition, some outcomes take longer 

to develop than the length of the grant period – and outcomes may occur or continue 

beyond a project period and in fact after the program evaluation has been completed. 

Administrators are sometimes unable to track outcomes, for example in instances where 

program graduates move out of the region and lose contact with the program.

In some cases, program outcomes prioritized by the administrators or grant partners may 

not coincide with those of the local community. In Heartwood, for example, the 

community hoped to slow or halt the outflow of educated young people from the 

community. Program administrators, on the other hand, were more concerned with 

attracting new residents and new high tech businesses to the area. At the Ohio Farmers’ 

Markets, the program was successful in bringing healthy food alternatives to the region 

despite the participants not necessarily completing the formal marketing plans and 

strategies stressed by the administrators in the grant.

Larger outcomes were not always agreed to be more desirable. Positive outcomes, such 

as job creation, that were the result of one or a few large businesses, were viewed by 

some as more risky and potentially less sustainable than the fewer jobs created by more 

numerous but smaller businesses.

Program goals sometimes varied from those of government partners. Whereas 

politicians most desired short-term revenues, some program administrators and 

associates prioritized more long-term payoffs.

In at least one instance, the administrator’s need for follow-up data was not impressed 

sufficiently on program participants, resulting in a lack of data to adequately gauge the

success of the program. In the future, it was agreed to more compellingly advise 

participants that submitting outcome data was a condition of participation in the program. 

Other issues of measurement ensued. In the Maryland case, electronic counting devices 

on the hiking and biking trails did not work as anticipated. In Toccoa, Georgia, there was 

a 10-year delay in beginning the renovation construction. Nevertheless, administrators, 

including one in Pennsylvania, stressed the importance of collecting the right type of data 

that can inform and hopefully impress the entities that provide the funding.

6.16.4 Fine-tune the Grant Process

A number of program administrators reported challenges navigating the proposal writing 

or grant administration processes, and offered some practices that might be adjusted. At 

least one conceded the difficulty in writing a grant proposal for a new program. It noted 

from experience, that submitting a proposal supplementing an existing program was 

more efficacious. However, some requests for proposals had to be passed over, since it 

was not always feasible to identify a program upon which to add another grant.

Staff changes at the granting agencies could be frustrating for applicants. Historical 

knowledge was lost, relationships had to be re-started, and new personnel had to be 
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

September 2015 | 161

trained. In some cases, new staff simply made errors or delayed submitting materials, 

which led to administrative delays and/or funding denials.

Unanticipated hold-ups in approval processes could delay the spending of grant funds 

within the permissible project timeframe, resulting in having to return uncommitted funds 

to the funding agency. Such was the situation surrounding the approval for a wind turbine 

at the Mountain Empire Community College in Virginia. Also, as a grant period deadline 

approached, that program felt compelled to purchase program equipment before it could 

hire an appropriate faculty person. This resulted in missteps in the purchase of highly 

technical program equipment which turned out to be inappropriate for the project at hand, 

but fortunately could be used for another project.

Other grant administrators noted challenges coordinating multiple funding sources, overscaling the scope of the program, and the short timeframes in which to get proposals and 

reports completed and submitted. One grantee regretted that officials had failed to 

sufficiently plan for the end of grant support.

7 Rural Policy Interviews

7.1 Background

The purpose of this section of the report is to provide context to the findings of this 

evaluation. Appalachia is a unique region with various assets and challenges. To design 

appropriate recommendations and solutions for this Region, it is important to first 

understand the key challenges to development, desired outcomes, and what types of 

solutions are likely to work. 

In an effort to gain insight into these concepts, experts in the field of rural development 

were contacted for discussions regarding the challenges, opportunities and strengths of 

areas like Appalachia. These experts included practitioners, researchers, and funders. 

The eleven individuals who participated were asked about the key challenges to the 

Appalachian Region, promising industries, and best practices for measuring the success 

of an investment in the Region. A few of the relevant experts were also asked about their 

thoughts on the role of outside funding in the Region. A list of 17 questions120 was used 

to guide the discussions with the development experts. The discussions were conducted 

as 30 to 60 minute phone interviews with each expert over the span of five weeks. 

In all, 18 development experts were contacted and interviews were held with eleven of 

these individuals. The first nine experts were selected by the HDR team, in conjunction 

with ARC, based on available contacts and experiences working in rural development.

The remaining nine experts were recommended by the first set of experts interviewed. 

The majority of experts (eleven) were first contacted by ARC through an initial 

introductory email121 explaining the background of this evaluation and why the HDR team 

wanted to speak with them. From there, the team followed up with the development

experts to schedule individual phone interviews and email the specific questions.

 

120 The full list of questions and the experts interviewed can be found in Appendix XX of this report.

121 The remaining seven experts were contacted first directly by the HDR team.
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

162 | September 2015

7.2 Summary of Discussions

The conversations with development experts covered three main themes: key challenges 

for rural communities in the United States as a whole and Appalachia in particular; target 

industries that the Appalachian Region should focus on; and how to measure the 

success of investments in rural communities. A few relevant experts were also asked 

about the roles of public and private funding entities in Appalachia. The remainder of this 

section discusses the interview findings.

7.2.1 Challenges in US Rural Communities

A primary focus when evaluating the best approach to allocating resources for rural 

development is identification of the key challenges and the context surrounding these 

concerns. Two key areas were discussed: major challenges faced by rural communities 

when trying to improve their well-being and how these challenges have changed and are 

expected to change over time.

Major Challenges

The respondents indicated three main challenges to rural economic development: lack of 

leadership and vision for future development; lack of stable, high-paying jobs; and a 

reduction in investment in rural development. 

The first challenge identified is that rural communities are often faced with a lack of 

leadership and vision for the future direction and growth of the region. Ideally there 

should be local leadership to develop growth strategies for each community, based on 

that particular community’s assets and needs. Without this leadership and direction, it is 

difficult to achieve any long term development. 

The second challenge noted is the lack of stable, high-paying jobs making it difficult for 

Appalachia to have a steady or growing economy. With the recent outflow of many 

manufacturing companies and associated loss of many well-paying jobs, it has been 

difficult for local workers to find replacement jobs with comparable pay. Instead workers 

are forced to accept the first job available, which is often lower-paid.

The third challenge that experts pointed out is a reduction in government investment in 

rural development, possibly due to the lack of rural advocacy at a national level. With 

high unemployment and a small tax base to draw from, these rural communities have 

limited financial resources of their own and rely heavily on state and federal funding. 

Without these funds, the communities’ aging infrastructure is becoming more dilapidated 

and there is little money to build additional infrastructure needed for quality of life 

improvements. The diminished government investment is also exacerbating the growing 

disparities between rural and urban areas. With the loss of many traditional 

manufacturing jobs in rural areas, and as more opportunities and more government 

money go to urban areas, experts are concerned that without changes, there will be little 

economic opportunity left for people who feel connected to and remain in these rural 

communities.

In addition to these three primary challenges, some experts also discussed the 

challenges associated with brain drain and attracting youth to rural areas. Rural areas 

are often unappealing to younger populations, who leave for bigger cities, which have 
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

September 2015 | 163

greater access to the knowledge sector and a variety of well-paid, high-skilled jobs. The 

lack of a high-skilled labor force and the brain drain associated with younger adults 

leaving the area create a unique set of challenges. Rural areas, with their remoteness 

and lack of broadband and transportation connectivity, tend to attract older, retiring 

populations who seek the slower pace of life and beauty of nature associated with these 

areas. This demographic typically does not contribute to the workforce.

Evolution of Major Challenges

Experts agreed that these challenges for rural areas have been persistent over time. 

Despite major economic improvements that have occurred over the years, in part due to 

ARC’s investment, the Appalachian Region is still lagging behind the rest of the national 

economy. However, the situation is now more severe given the loss of much of the 

manufacturing industry and far less government investment in the Region. 

Broadband was identified as a newly emerging need for rural communities in Appalachia. 

With the introduction of broadband to most areas in the country, the Appalachian Region 

will certainly be left behind without it, though unfortunately, investing in this technology 

will not guarantee growth in the Region either. 

The challenges facing the Region have been relatively consistent over time, and the 

experts tended to agree that these challenges would likely remain and worsen over the 

next decade or two. Experts anticipate a further dwindling of government investment in 

the Region, which will put pressure on the infrastructure and public resources, such as 

schools and hospitals. 

Experts also expect the economic gap between subregions in Appalachia to widen in the 

future. Many of the investment decisions made today, including the location of

broadband infrastructure or roads, will promote growth in some areas and not others. 

Rural areas with better connections to outside markets will be more likely to experience

stronger growth.

The selection of locations for the installation of broadband is closely related to concerns 

about the ability of the Region to adapt to rapidly changing technology. Technology 

improvements are now growing at an exponential rate, and workers will need to learn 

new skills to adapt. Whereas before workers could often train for a position once and 

remain in that job the rest of their career, the workers of the future will need to practice 

continuous learning throughout their careers. Incorporating ever-changing technologies, 

manufacturing jobs are transitioning into more “high-tech” positions, as machines are 

designed to take over certain tasks. There is a stereotype that individuals in the Region 

do not have the skills necessary for high-tech manufacturing. Despite this stereotype, 

people within the region are adapting and working to build these skills in an effort to 

retain these high-paid jobs. 

In addition to the concerns regarding growing technology and worker skill gaps in rural 

environments, the experts also expressed concern about what “development” would look 

like in the future for Appalachia. With increasing urban sprawl in nearby areas, and the 

tendency to over-extract natural resources for quick money, preserving the area’s natural 

endowments will need to be prioritized. One of the experts explained that traditional 

development is generally seen as building more strip malls and other infrastructure. 

Without the right environmental protections in place, too much of this type of 
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

164 | September 2015

development could destroy the natural beauty and way-of-life of the Appalachian Region. 

The natural resources in the area are not only critical to the Region, but to the nation as 

a whole. Appalachia is one of the nation’s largest and relatively undeveloped areas.

Preservation of and national concern for this ecosystem will continue to increase as 

climate change starts to play a bigger role in the future economy. This may present 

economic opportunities to the Region in areas such as the carbon market, where 

revenue could be earned through some form of emissions trading, as is currently done in 

California and other areas around the world.

7.2.2 Target Industries and Sectors for the Appalachian Region

To best leverage the assets provided by rural areas, certain industries and job sectors 

may be more suitable targets of focused efforts to attract opportunity and investments. 

To address the challenges associated with rural communities in the Appalachian Region, 

this portion of the discussions with the various experts focused on identifying key 

industries that provide an advantage to rural communities and how they fit within global 

trends.

Key Industries and Sectors for Rural Communities

Experts explained that identifying key industries or sectors to target for investment is very 

difficult at the broad level of “rural areas,” and depends on the specific characteristics 

and assets of each community. One expert noted that rural areas are likely to have a 

competitive advantage in industries that involve the extraction or consumption of natural 

resources, industries that require low-cost land or low-cost labor, or capital intensive 

industries that involve standardized production. The experts did identify some sectors 

that may be more suitable for rural areas, many of which are already being pursued by 

Appalachian communities, with a few new areas identified. The sectors include: 

• Shale gas fracking While there is certainly money to be made from fracking, 

experts also brought up concerns. Like other types of energy production , 

fracking tends to be very cyclical, leading to a “boom and bust” growth 

situation. Additionally, the majority of jobs created from fracking are 

temporary, and often the higher-paying jobs do not go to local workers122. 

These jobs tend to be held by transient crews from within the oil and gas 

industry; the “indirect” jobs associated with the increased resource 

production, and often held by the locals, tend to be lower-paying service 

related positions. Communities where fracking takes place must also deal 

with the costs often brought on by the process, such as air, water, and noise 

pollution, road congestion and accidents, and possible road damage. 

Fracking is a controversial energy production method that has been banned 

by New York State.

• Other forms of energy production. While some areas in the Appalachian 

Region already mine coal and extract natural gas, there is some potential for 

alternative energy production, including wind and solar. Production of these 

alternative energy sources may help supplement the decline in coal mining.

 

122 http://www.nofrackingway.us/2015/01/30/the-false-promise-of-local-fracking-jobs/
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

September 2015 | 165

• Farming and food supply. The abundance of land resources in the area 

naturally lend themselves to farming as a viable option for some rural 

communities. For those communities close enough to urban areas, there is 

an opportunity to supply agriculture to small-scale farmers markets and food 

co-ops. 

• Tourism and hospitality. Given Appalachia’s abundant natural resources 

and beauty, the Region should provide a great opportunity for tourism 

revenues. However, this will more likely be successful in areas that are better 

connected to more urban areas through the internet and good road systems.

• Manufacturing. This sector has always been important for much of the 

Appalachian Region, but today manufacturing jobs are diminishing. Despite 

the current decrease in manufacturing presence in the area, the industry may 

see a resurgence if more companies choose to locate in Appalachia. Areas 

of Ohio, Pennsylvania, and North Carolina have experienced success in 

attracting and retaining small and skilled manufacturing jobs by adapting to 

the high-skill market needs.

• Services. Key service industry opportunities include back office operational 

jobs such as information technology or accounting related jobs and financial 

services. Service jobs have been growing in Appalachia since the decrease 

in manufacturing jobs. Unfortunately, service jobs tend to pay less than 

manufacturing jobs. One expert highlighted financial services in particular as 

an area of opportunity for Appalachia to develop. Currently there are not 

enough institutions to provide sufficient financing in Appalachia. This gap 

could be filled through internet-based lending platforms such as the Lending 

Club. The lack of familiarity within the Region with these platforms also 

presents an opportunity for intermediary institutions, such as non-profits or 

financial institutions, to connect lenders to borrowers, and spread awareness 

of this new type of service.

• Healthcare administration. As more healthcare facilities open in rural 

Appalachia this sector presents more job opportunities. With targeted 

workforce training, some local residents could be eligible for other nonadministrative jobs as well.

• Retail. Small and medium enterprises (SMEs) can sell products to local 

markets, but one expert pointed out that people in rural communities are not 

big consumers. Increased access to broadband within the Region can aid

SMEs in connecting to the global market. 

• Ecological services. As climate change continues to become a more 

pressing issue, there will likely be more opportunities in trading carbon or 

other greenhouse gas credits. With Appalachia’s vast preserved ecosystem, 

the Region will be well-placed to offer these ecological services within and 

beyond the Region. If emissions limits are set, the Region would be wellpositioned to sell excess emission credits to areas that exceed their allotment 

to generate revenue.
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

166 | September 2015

There was also discussion of how increasing access to broadband in rural communities 

could create jobs. There is a general perception that bringing broadband to an area will 

create great job opportunities for rural areas. However, one expert explained that 

broadband is more likely to create jobs when coupled with business agglomeration, 

which is difficult to do in rural areas. The presence of a broadband network is not a 

guarantee for job creation, but the absence of broadband infrastructure is prohibitive to 

future development. Even if broadband did open up jobs for the rural workforce, they 

would be lower-paid service and retail jobs, not high-tech, higher-paying jobs, like 

developing apps. Much of the rural Appalachian workforce does not have a large 

prevalence of high-level programmers that tech companies seek when choosing where 

to locate. 

Over the past few years, there has been a shift in the types of industries employing 

people in Appalachia. Traditional manufacturing and coal mining jobs in the Region have 

diminished, while jobs in healthcare and other services sectors are increasing. Many of 

the industries and sectors identified by the experts have the potential to increase percapita income in rural areas, though as previously noted the mining and fracking sectors 

are often cyclical, resulting in a sometimes unstable economy. Issues associated with the 

generally cyclical nature of resource extraction are further exacerbated by the decrease 

in coal mining employment in the region. Additionally, one expert pointed out that the 

ability of even higher-paying jobs to increase per-capita income is also highly dependent 

on state and national tax policies.

Key Industries for Rural Areas and Global Trends

The experts had a tendency to compare the industry growth trends in rural Appalachia to 

those in the United States, rather than globally. One expert did remark on the importance 

of the linkage between Appalachian rural markets and global markets, noting that the 

growth of the Region will depend on how well connected it can become to global 

markets.

Most experts agreed that there was a similarity between industry growth trends in rural 

Appalachia and those in the United States. One expert predicts that the decline of 

traditionally rural jobs such as mining and manufacturing will lead industry and sector 

trends in rural areas to become more similar to those in urban areas which are 

connected to the international market. Experts listed some specific sectors that have 

been trending positively both in rural Appalachia and at the national level. These include

fracking, healthcare services, particularly those for the aging population, tourism, 

transportation, public utilities, government and other public sector jobs, and retail. 

One expert pointed out that the general growth of the nation often comes at the expense 

of the Appalachian Region due to the exploitation of natural resource extraction. These 

resources are used to support the growth of the nation, but the Region does not reap the 

benefits as the resources are used elsewhere in the nation and related jobs, as seen in 

the steel industry, are shipped abroad rather than maintained in the Region or nation as 

a whole.
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

September 2015 | 167

7.2.3 Measuring Success of Investments

The third key area of focus for discussion with the experts was the measurement of 

success of investments. The discussion covered criteria for measurement; measurement 

process, metrics and timing; risk and uncertainty in investments; and suggestions for 

best practices in performance measurement and tracking.

Rural Economic Investment Performance Measurement

During the interviews, experts remarked that ARC has done a good job of finding out the 

needs of rural communities and then making investments to try and address those 

needs. When asked about the best way to measure the success of economic 

development investment, experts spoke about two main types of improvements: 

improvements to the process of evaluating the success of investments, and 

improvements to the metrics used to measure this success.

Most experts acknowledged that it is difficult to improve the evaluation process

considering ARC’s constrained resources, though many agreed that ARC could do more 

to establish a direct link between the investments they provide and the successes seen 

in the Appalachian Region. If resources were not an issue, many experts mentioned that 

ARC should perform a type of quasi-experimental control study to identify, somewhat 

more confidently, whether or not ARC’s intervention caused the observed change or if 

this was due to other factors. Alternatively, ARC should at least do a comparison study 

between the region that received the investment and a region that is similar on as many 

levels as possible that did not receive the investment. 

One expert suggested ARC could also hire an external project evaluator to review and 

collect the data on its investments, rather than asking the grant recipients. This approach 

could potentially bring an additional level of consistency and credibility in data reporting 

and measurement. 

Another expert pointed out the importance of considering, for each investment, the 

opportunity cost of that investment. That is, before the investment is made one should 

consider alternative project or program investment options to determine if the proposed 

investment is the best use of those funds. 

Experts agreed that capturing jobs created is an important metric, but that the way it is 

measured could be revised. For instance, the metric could consider the per-capita 

income or median income associated with the job, benefits of the job, whether or not the 

job is full time, and who is getting the job (local residents or others). 

While noting that job creation is an important metric, the experts agreed that simply 

looking at the number of jobs created from an investment does not capture the full 

success of these projects or programs, particularly since not all investments are 

designed to create jobs. For instance, one expert gave the example of one of ARC’s 

smaller investments in an entrepreneur that allowed this young man to participate in a 

business competition. The investment was successful but did not create any jobs as this 

was not its intention. 

There was also consensus that there should not be a standard set of indicators for all 

ARC investments. Instead experts recommended constructing a unique set of indicators 

for each investment that would be agreed on by all involved parties at the beginning of
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

168 | September 2015

the project. Experts did suggest some general metrics or categories to be measured that 

ARC could consider when selecting the metrics for a project, some of which are already 

being tracked. These metrics include: 

• Growth in household income;

• Growth in gross state product;

• Total income generated without jobs (for instance, investing in conserving the 

Appalachia’s natural resources will generate income for the region, though 

not through jobs);

• Amount of small business development;

• Level of entrepreneurial culture;

• Business creation; and

• The link between an ARC investment and a specific strategic growth plan for 

a rural area.

The link between ARC investment and growth plan speaks to ARC’s overarching goal of 

improving local capacity in the Region, indicating the level to which regions are becoming 

more self-sustaining while also encouraging future strategic planning. 

One expert pointed out the need to capture more intermediate input and output metrics. 

For instance, when ARC invests in a workforce training program, the program should 

capture how many people attended the training, how many people completed the 

training, how many jobs were created, and how long the jobs were retained. The 

implementation of a robust system of metrics and frequent data tracking will make 

evaluations easier.

Another expert suggested a focus on the secondary or “follow-on” effects from ARC 

investments, such as additional new manufacturing plants locating in a town due to the 

success of a plant that received funding assistance. In this instance, ARC would count 

the jobs created directly by the first plant, as well as those jobs created by the additional

follow-on plants. This captures how well ARC is able to leverage its funds to create a real 

difference in the region. 

Finally, there was a general consensus that more qualitative measures should be 

tracked. Because ARC engages in so many programs that do not lend themselves to 

quantitative measurements, including building institutional capacity, qualitative data 

should be collected to supplement the quantitative measures. These can be in the form 

of surveys and interviews that capture general opinion of the program or project, and any 

quality of life improvements it may have created for those in the sample.

Timing of Performance Measurement

Experts agreed that the amount of time required to realize a program’s full impact 

depends on the type of investment and thus performance measurement timelines should 

also be project or program specific. Experts generalized that the impacts of investments 

in entrepreneurship and small business development will likely be realized within two to 

five years of funding. Infrastructure investments can take 10 to 20 years before benefits 

can be realized, given the time required for construction, and the time it takes people to 
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

September 2015 | 169

start using the asset, such as a road connecting a previously isolated rural area to a 

more urban area. Finally, investments in education and other social change programs 

can take a generation or more before impacts are fully realized.

Regardless of the time it takes to realize the full impacts of an investment, many experts 

recommended checking in on the progress of an investment every one to two years. 

Experts acknowledged the burden of data collection and ARC’s limited resources, but 

still emphasized the importance of periodically checking in on investments. Experts noted 

that if you wait to check in until the program is near completion, then it is often too late or 

too expensive to make a significant change in the program. Checking in periodically 

allows the program administrator the opportunity to make changes and still have an 

effective program. Experts recommended that ARC implement a standardized check-in 

process where first a preliminary assessment of all the investments is made, and then a 

more in-depth investigation is made only for those investments that stand out as either

not making progress or progressing far above target. Another suggestion was to increase 

the public participation process for implementation and allow a community process to 

adapt to changing needs during the program investment to improve the final product. 

Making changes in response to program feedback could help tailor a more effective 

product and better target future investments.

Program Management and Investment Selection

The likelihood for an investment to succeed depends greatly on the context of that 

particular region, and the needs of its people. Some experts noted that investments in 

infrastructure may be more likely than others to achieve a large “bang for the buck.” One 

expert noted that it is often not wise to only make one type of investment, and noted that 

ARC has done well with spreading investments around to different types of programs 

and areas within the Region. The experts also agreed that ARC has done a good job of 

checking in with the people in rural Appalachian communities to assess their needs 

before investing in a project. 

Experts also found it difficult to assess whether different types of investments are more 

difficult to manage. They noted that difficulty in management depends on the specific 

investment type and context. This same principle applied to determining “risky” 

investments. A few experts agreed that entrepreneurship programs were likely to be 

more risky investments, due to difficulty predicting which companies will or will not 

succeed. One expert posited that it is probably less risky to support entrepreneurs that 

supply to local demand, rather than local firms that serve external markets.

Education programs and workforce training were given as examples of less risky 

investments. One expert noted the importance of having local investors as they tend to 

have an interest in the well-being of the Region and will search for innovative methods to

make an investment work rather than pulling out at the first sign of trouble. Overall, the 

consensus seemed to indicate that ARC would be well served to continue investing in a 

diverse set of projects to minimize risks and maximize impacts.

7.2.4 Role of Funders in Appalachian Economic Development

A subset of the experts were asked questions regarding the role of funders in 

Appalachian economic development. The questions centered on the role of public and 
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

170 | September 2015

private organizations in rural economic well-being and whether these roles have changed 

in recent years. The experts all noted that they did not foresee a time when public 

support would not be necessary in rural regions.

Experts agreed that both public and private entities have an important role to play in 

fostering economic development in these rural communities. After the public sector 

lowers barriers to entry in markets, and builds public assets, the private sector brings 

business to the communities, creating a complementary relationship between the two 

types of entities. One expert specified that it is the role of both types of entities to build 

two key assets: infrastructure and networks. Networks can be anything from links to 

sources of financing, to relationships among local and external institutions. 

One expert highlighted that philanthropic organizations have been disappearing, at least 

in part due to society’s focus on total impact numbers. It is more popular politically to 

support programs and investments that will help more people. The problem is that rural 

areas are intrinsically spread out with small populations. Thus the impact numbers for 

rural investments cannot compete with those in urban areas despite the ability for 

investment in a small rural area to impact an entire community as opposed to a small 

subset.

One expert noted that the roles of funders have changed over time as philanthropic 

entities who once focused on providing funding and networks to rural communities have

shifted to providing top-down planning and problem solving. This expert warned this was 

taking decision making away from the people living in the rural areas, and giving more 

influence to people living removed from the rural issues. This shift has increased the 

importance of ARC as a critical resource for keeping funders informed of the needs of 

rural communities.

7.3 Findings and Recommendations

The findings from the interviews with development experts can be used to help guide 

ARC forward in its future program investments. All development experts commented on 

their respect for ARC and all that it has done for the Region. The expert feedback is 

meant to advise ARC on the types of investments to make going forward, what 

challenges to try and address, and how ARC can better capture and report the success 

of its programs. The following section summarizes some of the main recommendations 

and findings.

One recommendation that came from these expert interviews was that ARC should 

focus on supporting local leadership to promote long term stability and growth of 

the Region. Despite the challenges associated with allocating funding toward programs 

such as the building of civic capacity, that are difficult to measure and often intangible, 

this is a very necessary area to focus in order to implement a long-term development 

plan to move the region forward. There are currently no long term protections for the 

environment, and no way for long-term projects to be coordinated. Considering that 

experts also suggested implementing more qualitative measures for tracking the success 

of ARC investments, this could provide another option for building support for civic 

capacity investments. 

Having strong local leadership could also help address the other two key challenges 

identified: well-paid, stable jobs for the future, and the lack of public funding in rural 
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

September 2015 | 171

areas. Local leaders could champion their needs in an organized manner to secure more 

government funding. And with a development plan in place, there will be tangible projects 

for public funds to go towards.

The findings from the interviews indicate that there is no general roadmap for making 

good investments in rural communities and no easy solution to finding the industry 

investments that provide the biggest “bang for the buck.” Each investment must consider 

the needs and challenges of that particular area, there is no one industry or one type of 

investment that is generally best to help rural communities.

Experts did not identify any revolutionary sectors in which the Region could specialize. 

Industries that have traditionally existed in rural areas – including energy production, 

farming, tourism, manufacturing, services, and retail – will likely continue to persist in the 

Region. Although broadband and shale oil fracking have recently surfaced with the 

potential for new jobs in the Region, neither industry is expected to provide a 

transformative or sustainable improvement for Appalachia. Jobs brought in through 

broadband connections are still likely to be low-wage service positions, and the mere 

presence of broadband does not guarantee an increase in jobs. Shale-drilling tends to 

come loaded with costs as well as benefits, and offers only a “boom and bust” type of 

economic growth, and thus should not be an economy’s primary source of revenue.

With the loss of well-paid manufacturing jobs in many Appalachian communities, ARC 

should focus investment on training the workforce to take on some of the higherskilled jobs within the industries available, including healthcare. To avoid 

Appalachian workers taking the first-available low-skilled positions, ARC could support 

the implementation of targeted training programs to help local Appalachians apply for 

higher-paid positions. Another budding industry for Appalachia is likely to be ecological 

services. ARC should start considering now how to prepare the Appalachian Region to 

trade in the carbon market.

Another point made during the expert interviews was the need to change the way rural 

workers are trained. As technology continues to progress at an exponential rate and 

increasingly jobs are being replaced or enhanced by new technology, workers will no 

longer be able to simply learn one skill and practice that skill for their rest of their career. 

Instead, workers will need to adapt to become life-long learners. Similarly, workforce 

training programs will need to adapt to provide guidance on this continuous learning 

path.

The greatest amount of time during the discussions with experts focused on how ARC 

should measure the success of its investments. There was general consensus that 

job creation should not be the only indicator used to measure the success of an ARC 

investment, with general metrics such as growth in household income and level of 

entrepreneurship in the culture as potential additions. The experts suggested that a 

unique set of indicators should be selected for each investment, due to the wide variation 

in types and categories of investments. More qualitative measures of success are also 

needed to capture much of ARC’s “soft” investments, such as built civic capacity, which 

are not easily quantified. 

Experts also suggested increasing the robustness of the process for measuring project 

impacts, performing quasi-experimental and randomized control studies to isolate ARC’s 

impact on the Appalachian Region. However, many of these changes are quite costly, 
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

172 | September 2015

and thus may be counter-productive. Given the limited resources available to ARC, 

diverting funds to measure the successes rather than funding productive investments 

may actually be detrimental to the Region. Several experts agreed that though the 

results would be very useful, embarking on a quasi-experimental or randomized control 

study would be quite costly given ARC’s limited resources. 

However, there may be some efforts ARC could take to create a more robust data

collection process. For instance, some experts suggested ARC implement a 

standardized process for collecting, measuring, and regularly reporting (annually or 

biennially) the impact data on each investment. This process could be done by an 

outside data expert, or it could be taught to each grant applicant. 

While experts agreed there are some types of investments, such as small business 

development programs, that can achieve impacts within the first five years, there are also 

many others that can easily take 10 to 20 years, such as education or infrastructure 

investments. When the investment is first made and the success metrics are developed, 

the expected impact timeline should be written down. This will also hopefully help avoid 

political pressures for each investment to produce results every two years.

Finally, experts pointed out that government funding to Appalachia has been severely 

diminishing over time. Both government and the private sector have a role to play in 

supporting the Appalachian Region, and neither has been providing adequate support in 

the recent past. Perhaps fostering local leadership and strategic development plans will 

help in this effort to attract more public and private funds to meet the needs of the Region

well into the future.
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

September 2015 | 173

8 Study Findings and Recommendations

The Appalachian Regional Commission has four main goals, as laid out in the 2011-2016 

Strategic Plan:

1. Increase job opportunities and per capita income in Appalachia to reach parity with 

the nation.

2. Strengthen the capacity of the people of Appalachia to compete in the global 

economy.

3. Develop and improve Appalachia’s infrastructure to make the Region economically 

competitive.

4. Build the Appalachian Development Highway System to reduce Appalachia’s 

isolation.

The performance of ARC’s job creation and retention investments is closely related to 

these goals, particularly the first two, and this program evaluation intends to assess the 

performance and economic impacts of these investments between FY 2004 and FY 

2010. In addition, the evaluation offers insight into ways that ARC can enhance its ability 

to document and report program impacts, as well as assess potential industries or 

improvements to target to help improve rural areas in the Region. 

8.1 Key Findings

This job creation and retention program evaluation examined the impact of grant 

investment on income, employment opportunities, job creation and retention, and 

businesses created, retained and served. The evaluation consisted of both quantitative

and qualitative outcomes of the investments to gauge the overall performance of the 

projects. A review of initial ARC grant information, a survey, and interviews with grant 

recipients all inform the analysis.

At the time of grant award, information about the grant amount, its purpose, potential 

impacts, and other factors is collected and documented in the ARC.net database. 

Business/Training grants accounted for the largest number of grants, both in terms of 

actual awards and valuation of funding. This finding is consistent between the ARC.net 

database and the survey responses. Part of this volume is accounted for by the seven 

Enterprise Development Programs, each of which were funded annually during the study 

period. Enterprise Development Programs accounted for nearly 42 percent, $17.8 million 

of the $42.5 million, in grants awarded during this time period.

The survey results show that overall, grant recipients seemed to underestimate some of 

the impacts of their project. This was also true when looking at the outcomes noted in the 

ARC.net database. Particularly, both the responses at the time of project closeout and 

the survey responses indicated that respondents underestimated job creation, 

businesses served and improved, and leveraged private investment. Business/Training 

projects generated the most benefit in each category measured – job creation, job 

retention, businesses served, businesses retained, and leveraged private investment. 

This finding is somewhat unsurprising, as each there were between 19 and 21 responses 

for Business/Training projects, depending on metric, while the number of responses from 
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

174 | September 2015

other categories ranged from four to nine. The survey responses indicated that a large 

portion of the outcomes and outputs were attributable to the Enterprise Development 

Programs. 

In addition to the quantitative outputs and outcomes, some qualitative measures were 

also considered in the survey. Respondents could select multiple beneficiaries when they 

completed the survey, and their responses suggest that most of the projects benefited 

businesses, entrepreneurs and/or non-profit organizations. Sixty-one percent of 

respondents indicated that the primary goal of their project was business assistance. 

Approximately one-quarter of projects had a goal of attracting visitors and eighteen 

percent to promote arts and culture.123 A majority (55 percent) of survey respondents 

indicated that their project outcomes were either entirely (10 percent) or mostly (45 

percent) attributable to ARC’s funding. Eight-six percent of respondents indicated that 

without ARC funding, their project would not have been undertaken (44 percent) or would 

have been done on a smaller scale (42 percent). These results highlight the importance 

of these grants to achieving both the program’s intended local goals and ARC’s larger 

strategic goals.

Beyond to the survey and the case studies, the team also held several discussions with 

rural policy experts in an effort to provide context to the past program investments and 

gain insight for investments going forward. These discussions showed that the 

challenges facing rural Appalachia include a lack of leadership and jobs, and decreasing 

investment. Discussions also included potential target industries to help achieve the goal 

of bringing parity to the region. Among others, identified industries included: energy 

production; farming and food production; tourism and hospitality; manufacturing; and 

services. Many of the grants that ARC awarded during the FY 2004 to FY 2010 period

were in support of these identified target industries. Continued focus on these industries 

could help the region continue to move forward.

Survey respondents, case study participants, and policy experts all indicated that 

measuring success can be costly and difficult. They noted that the process and metrics 

can be improved to provide better information and potentially optimize outputs.

The team identified four key “lessons learned” based on the survey analysis, case 

studies, and policy discussions. These may help the Commission and future grantees 

best leverage available funding to achieve their individual and collective desired 

outcomes. 

• Engage, educate and involve the community

• Change is good – be flexible

• Mind your measures

• Fine-tune the grant process

 

123 Note that respondents could choose more than one primary goal.
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

September 2015 | 175

8.2 Recommendations

The recommendations provided below are based on the survey results, the case studies 

and feedback from the policy interviews. They are intended to both improve the quality of 

the data utilized in ARC program evaluation and the data collection process itself.

Improve the data collection and tracking process. Despite thorough efforts at 

gathering data for these projects, the team was able to collect data for fewer projects 

than desired. In addition to difficulty tracking some grantees, many of those individuals 

contacted did not have data on the measured outputs and outcomes. Over the course of 

nearly 10 years, some of the grant recipients had previously collected this information but 

no longer had the records. Others simply did not track the outputs and outcomes.

Given the high priority of tangible, measured improvements to job opportunities and other 

measures of competitiveness, it is in the best interest of ARC to require tracking for 

reporting purposes. Tracking of the performance metrics that are deemed most important 

to ARC should be required. It is important not to penalize grant recipients should factors 

outside their control result in a lack of expected performance, but tracking and measuring 

these outputs and outcomes will help improve ARC’s project prioritization and best utilize 

limited funds into the future. Informing the Local Development Districts of the importance 

of collecting and maintain performance metrics may be a first-step to improving overall 

data collection.

Clarify and specify measurement metrics and timing. Grant recipients and rural 

policy experts agreed that tracking and measurement of project outcomes was difficult 

and needed improvement. They suggested that ARC implement a standardized process 

for collecting, measuring, and regularly reporting the impact data on each investment. 

This can either be done by an outside resource or by grant applicants educated on the 

tracking and measurement process desired by ARC. At present, there is some confusion 

about the exact nature of each metric and how it should be measured and tracked.

Several additional metrics were suggested to supplement the standard job creation 

measures, including growth in household income and level of entrepreneurship in the 

culture. More qualitative measures of success are also necessary, as there are many 

“soft” outcomes, such as civic capacity, that are very important to communities but are 

not easily quantified. Rural policy experts suggested customizing the indicators for each 

project type, due to the wide variation in types and categories of investment.

The time period for outcomes also varies by project type, with some investments such as 

education programs, taking a longer period to reach their full impact. The tracking and 

measurement process should account for this.

Collect additional information related to funding sources. In many cases, ARC funds 

are not the only source of capital for these projects. Gaining a better understanding of all 

funding required to implement a project, as well as the sources of these funds, could 

provide useful insight to ARC. Understanding differences in funding sources and options 

for various job creation and retention programs may help target and prioritize future 

investments. Additionally, tracking detail on leveraged private investment may be useful 

for other potential project investments where a variety of funding sources are more 

available. This may free some of ARC’s limited available resources for those high-impact 

projects with more limited access to other funding sources may help ARC best utilize the 
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

176 | September 2015

limited available resources. A better understanding of the complete funding picture may 

be informative to ARC as future investments are considered.

Consider targeted investment and focus on key growth industries. Focusing some 

grant resources on potentially high-growth, high-paying industries within the region could 

lead to long-term gains in overall quality of life. The decline of coal mining has led to a 

reduction in high paying jobs in many parts of the Region. Focus on clean energy 

industries and other high-growth potential industries may help provide well-paying jobs 

for area residents while potentially reducing some of the “brain drain” that the region has 

experienced.

Increase focus on workforce training. Increase focus on both general workforce 

training and the means by which workers are trained. A focus on investment in workforce 

training for some higher-skilled jobs within existing and targeted industries, including 

healthcare and ecological services, could help increase per capita income in the Region. 

The way that workers are trained should also be examined. The rapid nature of 

technological improvements requires constant learning. Workforce training programs 

should provide guidance on continuous learning to best prepare area workers for longterm employment.

Support local leadership. One key to continued success in the region is an increased 

focus on leadership within the rural communities. Supporting local leadership can help 

promote long-term stability and growth in the Region. This support can help foster civic 

capacity investment and environmental protections in the longer-term. Strong local 

leadership may also help address two key challenges – access to well-paid, stable jobs 

and a lack of public funding in rural areas. Development plans and local leadership could 

provide rural areas with champions who can organize in an effort to secure additional 

funding.

Consider additional assistance or resources for program administrators. Feedback 

has indicated that the grant writing and tracking process can be burdensome to 

recipients. Several agencies noted that they had forgone potential funding for new 

projects due to the time and effort required in the grant writing process. Though it has the 

potential to be costly and divert resources from actually funding projects and programs, 

additional support or resources both during the grant writing process and during the 

monitoring of grants would be beneficial to grantees. Many of these rural communities 

are strapped for resources, and the time required to plan and monitor a grant is often be 

cumbersome. Providing additional resources may contribute to greater long-term 

success, as well as better data collection and performance monitoring.
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

September 2015 | 177

Appendix 1 - Survey

Appalachian Regional Commission

Program Evaluation of Job Creation and Retention Projects 

Questionnaire

May 2014

This short questionnaire should not take more than 20 minutes to complete. All information will 

be aggregated prior to release. Your responses will be kept confidential. 

The results of this questionnaire will help the Appalachian Regional Commission fulfill its 

mission to promote economic development in the region, and assist current and future grantees 

with improving project performance measurement. If you have any questions, please contact 

Chris Mongelli (617-603-6352 or christopher.mongelli@hdrinc.com) or Julie Marshall (202-884-

7790 or jmarshall@arc.gov). Thank you very much for participating.

Section A: Background Information 

1. In order to track responses and enable us to follow-up with you, we ask that you 

provide your contact information.

• Name of person completing this questionnaire

• Organization

• Address

• City

• State

• Zip

• Phone

• Email

• Website

2. What was the ARC project title?

3. What was the ARC project ID (for example, AL-12345)?

4. This project primarily benefited (select all that apply):

• Households

• New businesses

• Existing small to medium-sized businesses

• Business / industrial park(s)

• Entrepreneurs

• Youth

• Residents

• Government officials

• Non-profit organizations

• Conference attendees

• Downtown district

• Public services

• Researchers

• Students

• Policy makers
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

178 | September 2015

• Healthcare providers

• Future ARC projects

• Other (please specify)

5. The primary goal(s) of the project include (select all that apply):

a. Business assistance

b. Attracting visitors

c. Promoting arts and culture

d. Environmental preservation

e. Developing energy industries

f. Improving energy efficiency

g. Supporting local food production and sales

h. Other (please specify)

6. Based on the following definitions, how would you classify the project?

a. Local Food: A project that aims to start or develop a farm or a business, incubator, etc. 

in the agriculture or culinary fields.

b. Child Care / Education: A project that builds or expands a school, designs a new class 

or curriculum, sets up a day care, or benefits children or students in some other way.

c. Community Building / Tourism: A project that looks to enhance communities through 

building, renovating or marketing tourist attractions or fixing associated infrastructure.

d. Energy / Environmental: A project that looks to build a power plant, improve energy 

efficiency, or protect the environment.

e. Planning / Research: A project that looks to develop a plan, possibly associated with 

one of the other categories, or conduct some form of research.

f. Business / Training: A project that creates a business or incubator for an industry not 

listed in other categories a-e above. This type of project may also involve training 

employees or adults that are not considered students.

g. Other: Please describe.

Business / Training Projects

Additional activities undertaken by this project include (check all that apply):

7. Basic Education / Training / Instruction:

a. Provided general education

b. Provided computer skills training / instruction

c. Provided occupational / job skills training / instruction

d. Provided basic education to adults

e. Provided training in how to use project-purchased equipment

f. Provided instruction in business management

g. Provided pedagogy or teaching skills training to teachers

h. Provided economic development or leadership workshop for locally elected officials

i. Provided mentoring program for high school students

j. Hiring an intern for administrative work

k. Hiring an intern to help with research

l. Giving experience to a student research assistant

m. None of the above
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

September 2015 | 179

n. Other (please specify): 

8. Educational Resources:

a. Installed science lab or other special use classroom equipment

b. Developed/purchased computer-based educational materials

c. Established a community website

d. Developed/purchased paper-based educational materials

e. Developed/purchased teacher training program/materials

f. Developed/purchased instructor or teacher manuals/curriculum

g. Developed/purchased student manuals/materials

h. Developed proficiency standards (e.g. to align with industry standards)

i. Developed education program (e.g. performing arts, mentoring, etc.)

j. Developed resource directory for locally elected officials

k. None of the above

l. Other (please specify):

9. Community Outreach:

a. Provided community outreach activities

b. Established community or business partnerships

c. Distributed funds/mini-grants

d. None of the above

e. Other (please specify):

10. Community Capacity:

a. Development or implementation of a strategic plan

b. Programs that promote citizen empowerment

c. Aid in implementing outcome-based community or economic development activity

d. Community asset mapping

e. Community-based environmental clean-up project

f. Self-help infrastructure project

g. None of the above

h. Other (please specify):

11. Organizational Capacity:

a. Staff and board development for non-profit or community organization

b. Strategic planning for community based organization

c. Fundraising or grant writing training for non-profit employees

d. Establish a community development foundation

e. Technical assistance to establish a regional heritage tourism network

f. None of the above

g. Other (please specify):
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

180 | September 2015

12. Job Assistance and Career Counseling:

a. Provided college counseling

b. Provided career counseling (e.g. discussions, diagnostic/aptitude testing)

c. Provided job search/placement assistance (e.g. job bank, employer outreach)

d. Provided employability skills (e.g. work attitudes/habits)

e. None of the above

f. Other (please specify):

13. Social Support Services:

a. Provided assistance arranging transportation

b. Provided financial assistance 

c. Provided health, therapeutic, or developmental assistance

d. Provided referrals to other agencies for social support services

e. None of the above

f. Other (please specify):

14. Technical Assistance:

a. Assist businesses in developing marketing strategies

b. Implement technology transfer program

c. Facilitate business training course

d. Work with potential entrepreneurs to develop business plan

e. Provide expertise in areas of finance or accounting

f. None of the above

g. Other (please specify):

15. Business Site Development:

a. Extend water/gas/sewer services to business site

b. Extend telecommunications services to business site

c. Reuse of existing brownfield

d. Construction or rehabilitation of building

e. None of the above

f. Other (please specify):

16. Business Incubation:

a. Construction of business incubation building

b. Management and/or operations of incubation building

c. Incubation for manufacturing businesses

d. Incubation for service industries

e. Incubation for food related businesses

f. Incubation for arts related businesses

g. Incubation for a variety of industries

h. None of the above

i. Other (please specify):
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

September 2015 | 181

17. Entrepreneurship Education:

a. K – 12

b. College

c. Adult

d. None of the above

e. Other (please specify):

18. Export Development:

a. Trade show assistance

b. Export readiness training

c. International market research

d. Inbound trade missions

e. Commercial service ‘Gold Key’ service

f. None of the above

g. Other (please specify):

19. Access to Capital:

a. Development venture capital funds

b. Revolving loan funds

c. Micro credit funds

d. None of the above

e. Other (please specify):

20. Please use this box to provide nay additional information, including: data sources; 

estimation methods; or anything else that may be relevant.

Section B: Project Outputs 

What specific results (outputs) were actually achieved by this project? These questions attempt 

to capture the characteristics, capacity and direct outputs of the job creation and retention 

project. Depending on your project timeframe, include CUMULATIVE outputs up to 3 years after 

project closeout.

In the Output column, please provide a numerical estimate. If you did not track information for a 

specific output, please enter “DNT” in the corresponding box in the Output column (e.g., you do 

not have information about the number of households served). If an output is not applicable to 

your project, please enter “NA” in the corresponding box in the Output column (e.g., your project 

did not provide services to households).

Use the definitions provided below to determine how a given output should be expressed.

DEFINITIONS:

Outputs: Measures of a program's activities

Households Served: The cumulative total number of households that are served by your ARC 

project, from project inception until now. 

Businesses Served: The cumulative total number of businesses that are served by your ARC 

project, from project inception until now. 
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

182 | September 2015

Organizations Served: The cumulative total number of organizations that are served by your 

ARC project, from project inception until now. 

Communities Served: The cumulative total number of communities that are served by your 

ARC project, from inception until now.

Visitors Served: The number of new visitors to a destination resulting from an ARC investment.

Students Served: The total number of students served by the ARC project. For example, the 

number of students enrolling in a new community program the year of the project and for three 

years after.

21. Group Outputs: Output

Households Served

Businesses Served

Organizations Served

Communities Served

Visitors Served

Students Served

22. List any other outputs that were measured or evaluated (if no other outputs were measured, 

enter “NA” into the box): ___________________________

23. Please use this box to provide any additional information, including: data sources; 

estimation methods; whether or not outputs vary over time; or anything else that may be 

relevant (if there is no additional information, enter “NA” into the box).

Section C: Project Outcomes

Please provide the following information about final project achievements and results. 

These questions attempt to capture the direct outcomes of the project. Depending on your 

project timeframe, include CUMULATIVE outcomes for 3 years after project closeout.

In the Outcomes column, please provide a numerical estimate. If you did not track information 

for a specific outcome, please enter “DNT” in the corresponding box in the Outcomes column. If 

an outcome measure is not applicable to your project, please enter “NA” in the corresponding 

box in the Outcomes column.

In the Year(s) column, please indicate the number of years for which the outcome measure was 

tracked. 

Use the definitions provided below to determine how a given outcome should be expressed.

DEFINITIONS:

Outcomes: Changes that result from the activities

Households Improved: The cumulative total number of households with a measurable 

improvement due to the ARC project. This number is ALWAYS the same as, or a subset of, the 

“Households Served” output measure.
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

September 2015 | 183

Businesses Improved: The cumulative total number of businesses with a measurable 

improvement due to the ARC project. This number is ALWAYS the same as, or a subset of, the 

“Businesses Served” output measure.

Organizations Improved: The cumulative total number of organizations with a measurable 

improvement due to the ARC project. This number is ALWAYS the same as, or a subset of, the 

“Organizations Served” output measure.

Communities Improved: The cumulative total number of communities with a measurable 

improvement due to the ARC project. This number is ALWAYS the same as, or a subset of, the 

“Communities Served” output measure.

Jobs Created: The total number of (1) direct hires made as a result of the project’s operation 

(e.g. teachers hired, public safety, etc.); and (2) direct private sector jobs created following the 

delivery of ARC-funded services or project completion. This does NOT include construction jobs 

to build the ARC funded project. In the case of part-time jobs, please convert these to full-time 

equivalent and round up to report whole numbers.

Jobs Retained: The total number of jobs that would be lost or relocated without the ARCfunded project. Existing jobs benefiting from an infrastructure upgrade or other ARC-funded 

project should not be counted as “Jobs Retained.”

Businesses Created: The total number of businesses that located in the region as a direct 

result of the ARC-supported project within three years of the project end date..

Businesses Retained: The total number of existing businesses that remained in the region due 

to the ARC-supported project.

Leveraged Private Investment (LPI): The total dollar amount of private sector financial 

commitments and investment that were not part of the project funding, but followed as a result 

of the completion of the ARC project. 

Revenues Increased (non-export): The total dollar amount of any increase in non-export 

(domestic) sales that occurred among businesses as a result of the ARC project.

Revenues Increased (export): The total dollar amount of any increase in export (international) 

sales that occurred among businesses as a result of the ARC project.

Costs Reduced: The total dollar amount of costs reduced as a result of the ARC project 

activities, within one year of project implementation. For example, small business technical 

assistance may help a business streamline and cut costs, or an energy efficiency program may 

help to reduce energy costs.

24. & 25. Group Outcomes: Outcomes Year(s)

Households Improved

Businesses Improved

Organizations Improved

Communities Improved
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

184 | September 2015

26. & 27. Economic Outcomes: Outcomes Year(s)

Jobs Created

Jobs Retained

Businesses Created

Businesses Retained

Leveraged Private Investment ($)

Revenues Increased (non-export, 

$)

Revenues Increased (export, $)

Costs Reduced ($)

28. In what industries were the jobs created or retained? Please add a quantified number of 

jobs for each affected industry. 

• Agriculture

• Mining

• Construction

• Manufacturing

• Transportation Utilities

• Wholesale

• Retail

• Finance, Insurance or Real Estate

• Services

• Other

29. List any other outcomes that were measured or evaluated: _______________________

30. Please provide relevant information about how the outcome measures were calculated

such as data sources used or key assumptions. 

Section D: Other Impacts of ARC Funding

31. To what extent are the estimated outcomes attributable to the ARC intervention and 

funding?

a. Entirely

b. Mostly

c. Somewhat

d. Slightly

e. Too difficult to determine
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

September 2015 | 185

32. What do you think would have happened to the project if ARC funding had not been 

available?

a. Would have been completed with other funds in approximately the same time period

b. Would have been delayed for up to a year

c. Would have been delayed for a year or more

d. Would have been completed on a smaller scale

e. Would not have been undertaken

33. Aside from the direct jobs and investment associated with the project, were there any other 

positive or negative economic impacts on the community? If so, please explain and quantify 

if possible.

34. Did the project help attract any additional government or philanthropic funding? Please 

describe.

35. Please describe any private investment not directly related to the project that occurred as a 

result of the project. 

36. Please provide any additional comments on the project in terms of accomplishments, 

challenges, or other relevant information to help with the program evaluation.

Section E: Long-Range Impacts

Using the measures below, please rate the long-term impact of your project on the economy, 

the environment, and the quality of life in your community. 

DESCRIPTION OF CHOICES:

None: Project had little to no impact on the measure’s current trend

Slight: Project impact was not large enough to reverse or stabilize the measure’s current trend

Moderate: Project impact contributed to the stabilization or reversal of the measure’s current 

trend

High: Project impact was responsible for significant improvement in the measure’s current trend

37. Economic Measures None Slight Moderate High

What impact did this project have on:

Attracting new residents or stabilizing the area’s population

Attracting new jobs or increasing employment at existing businesses

Creating new sources of income or increasing income for local residents

Increasing local business sales or the value of business assets (such as equipment, real 

estate)

Increasing the value of household assets (such as homes, land, farms) for local residents

Increasing the value of community assets (such as community buildings, schools, 

infrastructure, parks)

Other economic measures (please specify):
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

186 | September 2015

38. Competitiveness Measures None Slight Moderate High

What impact did this project have on:

Improving the stability and sustainability of the local economy

Improving the efficiency of business operations or public services

Improving the productivity of students, employees, businesses, land, or other assets

Improving the skill level of the workforce

Increasing the viability of local businesses

Improving access to markets for local products, businesses, artisans, and entrepreneurs

Reducing the cost of doing business

Other competitiveness measures (please specify):

39. Environmental Measures None Slight Moderate High

What impact did this project have on:

Improving air or water quality

Improving quality and/or access to land and natural resources

Waste reduction or improving waste management (reuse, recycling, alternative energy, 

biofuels)

Improving energy security and independence

Increasing energy efficiency or conservation

Other environmental measures (please specify):

40. Quality of Life Measures None Slight Moderate High

What impact did this project have on:

Improving access to culture, arts, historic sites, and other amenities

Improving civic life and governance

Improving public health, safety, or well-being

Improving public services & institutions

Increasing recreational opportunities

Improving the diversity (age, gender, race, ethnicity, economic status) of the population

Increasing knowledge and educational levels of the population

Other quality of life measures (please specify):

Thank you very much for participating in the Appalachian Regional Commission Program 

Evaluation survey. If you have any questions, please contact Chris Mongelli at 

christopher.mongelli@hdrinc.com or 617-603-6352. 
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

September 2015 | 187

Appalachian Regional Commission

Program Evaluation of Job Creation and Retention Projects 

Questionnaire

May 2014

This short questionnaire should not take more than 20 minutes to complete. All information will 

be aggregated prior to release. Your responses will be kept confidential. 

The results of this questionnaire will help the Appalachian Regional Commission fulfill its 

mission to promote economic development in the region, and assist current and future grantees 

with improving project performance measurement. If you have any questions, please contact 

Chris Mongelli (617-603-6352 or christopher.mongelli@hdrinc.com) or Julie Marshall (202-884-

7790 or jmarshall@arc.gov). Thank you very much for participating.

Section A: Background Information 

1. In order to track responses and enable us to follow-up with you, we ask that you 

provide your contact information.

• Name of person completing this questionnaire

• Organization

• Address

• City

• State

• Zip

• Phone

• Email

• Website

2. What was the ARC project title?

3. What was the ARC project ID (for example, AL-12345)?

4. This project primarily benefited (select all that apply):

• Households

• New businesses

• Existing small to medium-sized businesses

• Business / industrial park(s)

• Entrepreneurs

• Youth

• Residents

• Government officials

• Non-profit organizations

• Conference attendees

• Downtown district

• Public services

• Researchers

• Students

• Policy makers

• Healthcare providers

• Future ARC projects

• Other (please specify)
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

188 | September 2015

5. The primary goal(s) of the project include (select all that apply):

a. Business assistance

b. Attracting visitors

c. Promoting arts and culture

d. Environmental preservation

e. Developing energy industries

f. Improving energy efficiency

g. Supporting local food production and sales

h. Other (please specify)

6. Based on the following definitions, how would you classify the project?

a. Local Food: A project that aims to start or develop a farm or a business, incubator, etc. 

in the agriculture or culinary fields.

b. Child Care / Education: A project that builds or expands a school, designs a new class 

or curriculum, sets up a day care, or benefits children or students in some other way.

c. Community Building / Tourism: A project that looks to enhance communities through 

building, renovating or marketing tourist attractions or fixing associated infrastructure.

d. Energy / Environmental: A project that looks to build a power plant, improve energy 

efficiency, or protect the environment.

e. Planning / Research: A project that looks to develop a plan, possibly associated with 

one of the other categories, or conduct some form of research.

f. Business / Training: A project that creates a business or incubator for an industry not 

listed in other categories a-e above. This type of project may also involve training 

employees or adults that are not considered students.

g. Other: Please describe.

Child Care / Education Projects

Additional activities undertaken by this project include (check all that apply):

7. Educational Resources:

a. Installed science lab or other special use classroom equipment

b. Developed/purchased computer-based educational materials

c. Established a community website

d. Developed/purchased paper-based educational materials

e. Developed/purchased teacher training program/materials

f. Developed/purchased instructor or teacher manuals/curriculum

g. Developed/purchased student manuals/materials

h. Developed proficiency standards (e.g. to align with industry standards)

i. Developed education program (e.g. performing arts, mentoring, etc)

j. Developed resource directory for locally elected officials

k. None of the above

l. Other (please specify):
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

September 2015 | 189

8. Social Support Services

a. Provided assistance arranging transportation

b. Provided financial assistance 

c. Provided health, therapeutic, or developmental assistance

d. Provided referrals to other agencies for social support services

e. None of the above

f. Other (please specify):

9. Please use this box to provide any additional information, including: data sources; 

estimation methods; or anything else that may be relevant.

Section B: Project Outputs 

What specific results (outputs) were actually achieved by this project? These questions attempt 

to capture the characteristics, capacity and direct outputs of the job creation and retention 

project. Depending on your project timeframe, include CUMULATIVE outputs up to 3 years after 

project closeout.

In the Output column, please provide a numerical estimate. If you did not track information for a 

specific output, please enter “DNT” in the corresponding box in the Output column (e.g., you do 

not have information about the number of households served). If an output is not applicable to 

your project, please enter “NA” in the corresponding box in the Output column (e.g., your project 

did not provide services to households).

Use the definitions provided below to determine how a given output should be expressed.

DEFINITIONS:

Outputs: Measures of a program's activities

Households Served: The cumulative total number of households that are served by your ARC 

project, from project inception until now. 

Businesses Served: The cumulative total number of businesses that are served by your ARC 

project, from project inception until now. 

Organizations Served: The cumulative total number of organizations that are served by your 

ARC project, from project inception until now. 

Communities Served: The cumulative total number of communities that are served by your 

ARC project, from inception until now.

Visitors Served: The number of new visitors to a destination resulting from an ARC investment.

Students Served: The total number of students served by the ARC project. For example, the 

number of students enrolling in a new community program the year of the project and for three 

years after.
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

190 | September 2015

10. Group Outputs: Output

Households Served

Businesses Served

Organizations Served

Communities Served

Visitors Served

Students Served

11. List any other outputs that were measured or evaluated (if no other outputs were measured, 

enter “NA” into the box): ___________________________

12. Please use this box to provide any additional information, including: data sources; 

estimation methods; whether or not outputs vary over time; or anything else that may be 

relevant (if there is no additional information, enter “NA” into the box.

Section C: Project Outcomes

Please provide the following information about final project achievements and results. 

These questions attempt to capture the direct outcomes of the project. Depending on your 

project timeframe, include CUMULATIVE outcomes for 3 years after project closeout.

In the Outcomes column, please provide a numerical estimate. If you did not track information 

for a specific outcome, please enter “DNT” in the corresponding box in the Outcomes column. If 

an outcome measure is not applicable to your project, please enter “NA” in the corresponding 

box in the Outcomes column.

In the Year(s) column, please indicate the number of years for which the outcome measure was 

tracked. 

Use the definitions provided below to determine how a given outcome should be expressed.

DEFINITIONS:

Outcomes: Changes that result from the activities

Households Improved: The cumulative total number of households with a measurable 

improvement due to the ARC project. This number is ALWAYS the same as, or a subset of, the 

“Households Served” output measure.

Businesses Improved: The cumulative total number of businesses with a measurable 

improvement due to the ARC project. This number is ALWAYS the same as, or a subset of, the 

“Businesses Served” output measure.

Organizations Improved: The cumulative total number of organizations with a measurable 

improvement due to the ARC project. This number is ALWAYS the same as, or a subset of, the 

“Organizations Served” output measure.

Communities Improved: The cumulative total number of communities with a measurable 

improvement due to the ARC project. This number is ALWAYS the same as, or a subset of, the 

“Communities Served” output measure.

Jobs Created: The total number of (1) direct hires made as a result of the project’s operation 

(e.g. teachers hired, public safety, etc.); and (2) direct private sector jobs created following the 
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

September 2015 | 191

delivery of ARC-funded services or project completion. This does NOT include construction jobs 

to build the ARC funded project. In the case of part-time jobs, please convert these to full-time 

equivalent and round up to report whole numbers.

Jobs Retained: The total number of jobs that would be lost or relocated without the ARCfunded project. Existing jobs benefiting from an infrastructure upgrade or other ARC-funded 

project should not be counted as “Jobs Retained.”

Businesses Created: The total number of businesses that located in the region as a direct 

result of the ARC-supported project within three years of the project end date..

Businesses Retained: The total number of existing businesses that remained in the region due 

to the ARC-supported project.

Leveraged Private Investment (LPI): The total dollar amount of private sector financial 

commitments and investment that were not part of the project funding, but followed as a result 

of the completion of the ARC project. 

Revenues Increased (non-export): The total dollar amount of any increase in non-export 

(domestic) sales that occurred among businesses as a result of the ARC project.

Revenues Increased (export): The total dollar amount of any increase in export (international) 

sales that occurred among businesses as a result of the ARC project.

Costs Reduced: The total dollar amount of costs reduced as a result of the ARC project 

activities, within one year of project implementation. For example, small business technical 

assistance may help a business streamline and cut costs, or an energy efficiency program may 

help to reduce energy costs.

13. & 14. Group Outcomes: Outcomes Year(s)

Households Improved

Businesses Improved

Organizations Improved

Communities Improved

15. & 16. Economic Outcomes:

Jobs Created

Jobs Retained

Businesses Created

Businesses Retained

Leveraged Private Investment ($)

Revenues Increased (non-export, 

$)

Revenues Increased (export, $)

Costs Reduced ($)

17. In what industries were the jobs created or retained? Please add a quantified number of 

jobs for each affected industry. 
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

192 | September 2015

• Agriculture

• Mining

• Construction

• Manufacturing

• Transportation Utilities

• Wholesale

• Retail

• Finance, Insurance or Real Estate

• Services

• Other

18. List any other outcomes that were measured or evaluated: __________________________

19. Please provide relevant information about how the outcome measures were calculated

such as data sources used or key assumptions. 

Section D: Other Impacts of ARC Funding

20. To what extent are the estimated outcomes attributable to the ARC intervention and 

funding? 

a. Entirely

b. Mostly

c. Somewhat

d. Slightly

e. Too difficult to determine 

21. What do you think would have happened to the project if ARC funding had not been 

available?

f. Would have been completed with other funds in approximately the same time period

g. Would have been delayed for up to a year

h. Would have been delayed for a year or more

i. Would have been completed on a smaller scale

j. Would not have been undertaken

22. Aside from the direct jobs and investment associated with the project, were there any other 

positive or negative economic impacts on the community? If so, please explain and quantify 

if possible.

23. Did the project help attract any additional government or philanthropic funding? Please 

describe.

24. Please describe any private investment not directly related to the project that occurred as a 

result of the project. 

25. Please provide any additional comments on the project in terms of accomplishments, 

challenges, or other relevant information to help with the program evaluation.
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

September 2015 | 193

Section E: Long-Range Impacts

Using the measures below, please rate the long-term impact of your project on the economy, 

the environment, and the quality of life in your community.

DESCRIPTION OF CHOICES 

None: Project had little to no impact on the measure’s current trend

Slight: Project impact was not large enough to reverse or stabilize the measure’s current

trend

Moderate: Project impact contributed to the stabilization or reversal of the measure’s 

current trend

High: Project impact was responsible for significant improvement in the measure’s current 

trend

26. Economic Measures

What impact did this project have on: None Slight Moderate High

Attracting new residents or stabilizing the area’s population

Attracting new jobs or increasing employment at existing businesses

Creating new sources of income or increasing income for local residents

Increasing local business sales or the value of business assets (such as equipment, real 

estate)

Increasing the value of household assets (such as homes, land, farms) for local residents

Increasing the value of community assets (such as community buildings, schools, 

infrastructure, parks)

Other economic measures (please specify):

27. Competitiveness Measures None Slight Moderate High

What impact did this project have on:

Improving the stability and sustainability of the local economy

Improving the efficiency of business operations or public services

Improving the productivity of students, employees, businesses, land, or other assets

Improving the skill level of the workforce

Increasing the viability of local businesses

Improving access to markets for local products, businesses, artisans, and entrepreneurs

Reducing the cost of doing business

Other competitiveness measures (please specify):

28. Environmental Measures None Slight Moderate High

What impact did this project have on:

Improving air or water quality

Improving quality and/or access to land and natural resources

Waste reduction or improving waste management (reuse, recycling, alternative energy, 

biofuels)

Improving energy security and independence

Increasing energy efficiency or conservation

Other environmental measures (please specify):
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

194 | September 2015

29. Quality of Life Measures None Slight Moderate High

What impact did this project have on:

Improving access to culture, arts, historic sites, and other amenities

Improving civic life and governance

Improving public health, safety, or well-being

Improving public services & institutions

Increasing recreational opportunities

Improving the diversity (age, gender, race, ethnicity, economic status) of the population

Increasing knowledge and educational levels of the population

Other quality of life measures (please specify):

Thank you very much for participating in the Appalachian Regional Commission Program 

Evaluation survey. If you have any questions, please contact Chris Mongelli at 

christopher.mongelli@hdrinc.com or 617-603-6352. 
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

September 2015 | 195

Appalachian Regional Commission

Program Evaluation of Job Creation and Retention Projects 

Questionnaire

May 2014

This short questionnaire should not take more than 20 minutes to complete. All information will 

be aggregated prior to release. Your responses will be kept confidential. 

The results of this questionnaire will help the Appalachian Regional Commission fulfill its 

mission to promote economic development in the region, and assist current and future grantees 

with improving project performance measurement. If you have any questions, please contact 

Chris Mongelli (617-603-6352 or christopher.mongelli@hdrinc.com) or Julie Marshall (202-884-

7790 or jmarshall@arc.gov). Thank you very much for participating.

Section A: Background Information 

1. In order to track responses and enable us to follow-up with you, we ask that you 

provide your contact information.

• Name of person completing this questionnaire

• Organization

• Address

• City

• State

• Zip

• Phone

• Email

• Website

2. What was the ARC project title?

3. What was the ARC project ID (for example, AL-12345)?

4. This project primarily benefited (select all that apply):

• Households

• New businesses

• Existing small to medium-sized businesses

• Business / industrial park(s)

• Entrepreneurs

• Youth

• Residents

• Government officials

• Non-profit organizations

• Conference attendees

• Downtown district

• Public services

• Researchers

• Students

• Policy makers

• Healthcare providers

• Future ARC projects

• Other (please specify)
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

196 | September 2015

5. The primary goal(s) of the project include (select all that apply):

i. Business assistance

j. Attracting visitors

k. Promoting arts and culture

l. Environmental preservation

m. Developing energy industries

n. Improving energy efficiency

o. Supporting local food production and sales

p. Other (please specify)

6. Based on the following definitions, how would you classify the project?

h. Local Food: A project that aims to start or develop a farm or a business, incubator, etc. 

in the agriculture or culinary fields.

i. Child Care / Education: A project that builds or expands a school, designs a new class 

or curriculum, sets up a day care, or benefits children or students in some other way.

j. Community Building / Tourism: A project that looks to enhance communities through 

building, renovating or marketing tourist attractions or fixing associated infrastructure.

k. Energy / Environmental: A project that looks to build a power plant, improve energy 

efficiency, or protect the environment.

l. Planning / Research: A project that looks to develop a plan, possibly associated with 

one of the other categories, or conduct some form of research.

m. Business / Training: A project that creates a business or incubator for an industry not 

listed in other categories a-e above. This type of project may also involve training 

employees or adults that are not considered students.

n. Other: Please describe.

Community Building / Tourism Projects

Additional activities undertaken by this project include (check all that apply):

7. Arts / Culture Tourism:

a. Heritage, craft, or music trails

b. Planning for, designing, or constructing new attractions

c. Training and technical assistance to tourism-based entrepreneurs and businesses

d. None of the above

e. Other (please specify):

8. Sector-Based Strategies:

a. Boosting competitive advantage in value-added agriculture (includes dairy and 

aquaculture)

b. Assistance to a wood product industry

c. Advancement in biotechnology

d. None of the above

e. Other (please specify):
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

September 2015 | 197

9. Energy and the Environment:

a. Development-oriented conservation project

b. Gateway community development

c. Land reclamation

d. Brownfield site preparation

e. Developing “green” infrastructure

f. Water purification/stream clean-up

g. Improving energy efficiency

h. Development of renewable energy sources (wind, solar, etc.)

i. Non-renewable resource industry (coal, gas, etc.)

j. Training or certification program (ex. LEED certification)

k. None of the above

l. Other (please specify):

10. Business Incubation:

a. Construction of business incubation building

b. Management and/or operations of incubation building

c. Incubation for manufacturing businesses

d. Incubation for service industries

e. Incubation for food related businesses

f. Incubation for arts related businesses

g. Incubation for a variety of industries

h. None of the above

i. Other (please specify):

11. Entrepreneurship Education:

a. K – 12

b. College

c. Adult

d. None of the above

e. Other (please specify):

12. Export Development:

a. Trade show assistance

b. Export readiness training

c. International market research

d. Inbound trade missions

e. Commercial service “Gold Key” service

f. None of the above

g. Other (please specify):
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

198 | September 2015

13. Community Infrastructure:

a. Provided first time water/sewer service

b. Upgraded water/sewer service

c. Installed fiber-optic cable to reach unserved area

d. Extended gas line

e. None of the above

f. Other (please specify):

14. Community Facility:

a. Construction of a community center, theater, day care center, or public service building

b. Equipped a public computer lab

c. Technology for a distance learning center

d. None of the above

e. Other (please specify):

15. Community Revitalization:

a. Hired a marketing coordinator

b. Create a Main Street program

c. Streetscaping or infrastructure improvements

d. Erosion control and land stabilization

e. Flood berms

f. Hazard mitigation

g. Flood-recovery coordination

h. Homeland security

i. None of the above

j. Other (please specify):

16. Transportation:

a. Improvement to a railway

b. Widened sidewalks/added bike lanes

c. Constructed a trail

d. None of the above

e. Other (please specify):

17. Housing:

a. Water and sewer to new housing project serving low-income families or special needs 

citizens

b. Homeownership counseling

c. Capacity building for non-profit developers

d. None of the above

e. Other (please specify):

18. Please use this box to provide any additional information, including: data sources; 

estimation methods; or anything else that may be relevant.
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

September 2015 | 199

Section B: Project Outputs 

What specific results (outputs) were actually achieved by this project? These questions attempt 

to capture the characteristics, capacity and direct outputs of the job creation and retention 

project. Depending on your project timeframe, include CUMULATIVE outputs up to 3 years after 

project closeout.

In the Output column, please provide a numerical estimate. If you did not track information for a 

specific output, please enter “DNT” in the corresponding box in the Output column (e.g., you do 

not have information about the number of households served). If an output is not applicable to 

your project, please enter “NA” in the corresponding box in the Output column (e.g., your project 

did not provide services to households).

Use the definitions provided below to determine how a given output should be expressed.

DEFINITIONS:

Outputs: Measures of a program's activities

Households Served: The cumulative total number of households that are served by your ARC 

project, from project inception until now. 

Businesses Served: The cumulative total number of businesses that are served by your ARC 

project, from project inception until now. 

Organizations Served: The cumulative total number of organizations that are served by your 

ARC project, from project inception until now. 

Communities Served: The cumulative total number of communities that are served by your 

ARC project, from inception until now.

Visitors Served: The number of new visitors to a destination resulting from an ARC investment.

Students Served: The total number of students served by the ARC project. For example, the 

number of students enrolling in a new community program the year of the project and for three 

years after.

19. Group Outputs: Output

Households Served

Businesses Served

Organizations Served

Communities Served

Visitors Served

Students Served

20. List any other outputs that were measured or evaluated (if no other outputs were measured 

or evaluated, enter “NA” into the box): ___________________________

21. Please use this box to provide any additional information, including: data sources; 

estimation methods; whether or not outputs vary over time; or anything else that may be 

relevant (if there is no additional information, enter “NA” into the box).
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

200 | September 2015

Section C: Project Outcomes

Please provide the following information about final project achievements and results. 

These questions attempt to capture the direct outcomes of the project. Depending on your 

project timeframe, include CUMULATIVE outcomes for 3 years after project closeout.

In the Outcomes column, please provide a numerical estimate. If you did not track information 

for a specific outcome, please enter “DNT” in the corresponding box in the Outcomes column. If 

an outcome measure is not applicable to your project, please enter “NA” in the corresponding 

box in the Outcomes column.

In the Year(s) column, please indicate the number of years for which the outcome measure was 

tracked. 

Use the definitions provided below to determine how a given outcome should be expressed.

DEFINITIONS:

Outcomes: Changes that result from the activities

Households Improved: The cumulative total number of households with a measurable 

improvement due to the ARC project. This number is ALWAYS the same as, or a subset of, the 

“Households Served” output measure.

Businesses Improved: The cumulative total number of businesses with a measurable 

improvement due to the ARC project. This number is ALWAYS the same as, or a subset of, the 

“Businesses Served” output measure.

Organizations Improved: The cumulative total number of organizations with a measurable 

improvement due to the ARC project. This number is ALWAYS the same as, or a subset of, the 

“Organizations Served” output measure.

Communities Improved: The cumulative total number of communities with a measurable 

improvement due to the ARC project. This number is ALWAYS the same as, or a subset of, the 

“Communities Served” output measure.

Jobs Created: The total number of (1) direct hires made as a result of the project’s operation 

(e.g. teachers hired, public safety, etc.); and (2) direct private sector jobs created following the 

delivery of ARC-funded services or project completion. This does NOT include construction jobs 

to build the ARC funded project. In the case of part-time jobs, please convert these to full-time 

equivalent and round up to report whole numbers.

Jobs Retained: The total number of jobs that would be lost or relocated without the ARCfunded project. Existing jobs benefiting from an infrastructure upgrade or other ARC-funded 

project should not be counted as “Jobs Retained.”

Businesses Created: The total number of businesses that located in the region as a direct 

result of the ARC-supported project within three years of the project end date..

Businesses Retained: The total number of existing businesses that remained in the region due 

to the ARC-supported project.

Leveraged Private Investment (LPI): The total dollar amount of private sector financial 

commitments and investment that were not part of the project funding, but followed as a result 

of the completion of the ARC project. 

Revenues Increased (non-export): The total dollar amount of any increase in non-export 

(domestic) sales that occurred among businesses as a result of the ARC project.
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

September 2015 | 201

Revenues Increased (export): The total dollar amount of any increase in export (international) 

sales that occurred among businesses as a result of the ARC project.

Costs Reduced: The total dollar amount of costs reduced as a result of the ARC project 

activities, within one year of project implementation. For example, small business technical 

assistance may help a business streamline and cut costs, or an energy efficiency program may 

help to reduce energy costs.

22. & 23. Group Outcomes: Outcomes Year(s)

Households Improved

Businesses Improved

Organizations Improved

Communities Improved

24. & 25. Economic Outcomes:

Jobs Created

Jobs Retained

Businesses Created

Businesses Retained

Leveraged Private Investment ($)

Revenues Increased (non-export, 

$)

Revenues Increased (export, $)

Costs Reduced ($)

26. In what industries were the jobs created or retained? Please add a quantified number of 

jobs for each affected industry. 

• Agriculture

• Mining

• Construction

• Manufacturing

• Transportation Utilities

• Wholesale

• Retail

• Finance, Insurance or Real Estate

• Services

• Other

27. List any other outcomes that were measured or evaluated: __________________________

28. Please provide relevant information about how the outcome measures were calculated

such as data sources used or key assumptions. 
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

202 | September 2015

Section D: Other Impacts of ARC Funding

29. To what extent are the estimated outcomes attributable to the ARC intervention and 

funding?

a. Entirely

b. Mostly

c. Somewhat

d. Slightly

e. Too difficult to determine 

30. What do you think would have happened to the project if ARC funding had not been 

available?

a. Would have been completed with other funds in approximately the same time period

b. Would have been delayed for up to a year

c. Would have been delayed for a year or more

d. Would have been completed on a smaller scale

e. Would not have been undertaken

31. Aside from the direct jobs and investment associated with the project, were there any other 

positive or negative economic impacts on the community? If so, please explain and quantify 

if possible.

32. Did the project help attract any additional government or philanthropic funding? Please 

describe.

33. Please describe any private investment not directly related to the project that occurred as a 

result of the project. 

34. Please provide any additional comments on the project in terms of accomplishments, 

challenges, or other relevant information to help with the program evaluation.

Section E: Long-Range Impacts

Using the measures below, please rate the long-term impact of your project on the economy, 

the environment, and the quality of life in your community. 

DESCRIPTION OF CHOICES:

None: Project had little to no impact on the measure’s current trend

Slight: Project impact was not large enough to reverse or stabilize the measure’s current

trend

Moderate: Project impact contributed to the stabilization or reversal of the measure’s 

current trend

High: Project impact was responsible for significant improvement in the measure’s current 

trend
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

September 2015 | 203

35. Economic Measures None Slight Moderate High

What impact did this project have on:

Attracting new residents or stabilizing the area’s population

Attracting new jobs or increasing employment at existing businesses

Creating new sources of income or increasing income for local residents

Increasing local business sales or the value of business assets (such as equipment, real 

estate)

Increasing the value of household assets (such as homes, land, farms) for local residents

Increasing the value of community assets (such as community buildings, schools, 

infrastructure, parks)

Other economic measures (please specify):

36. Competitiveness Measures None Slight Moderate High

What impact did this project have on:

Improving the stability and sustainability of the local economy

Improving the efficiency of business operations or public services

Improving the productivity of students, employees, businesses, land, or other assets

Improving the skill level of the workforce

Increasing the viability of local businesses

Improving access to markets for local products, businesses, artisans, and entrepreneurs

Reducing the cost of doing business

Other competitiveness measures (please specify):

37. Environmental Measures None Slight Moderate High

What impact did this project have on:

Improving air or water quality

Improving quality and/or access to land and natural resources

Waste reduction or improving waste management (reuse, recycling, alternative energy, 

biofuels)

Improving energy security and independence

Increasing energy efficiency or conservation

Other environmental measures (please specify):

38. Quality of Life Measures None Slight Moderate High

What impact did this project have on:

Improving access to culture, arts, historic sites, and other amenities

Improving civic life and governance

Improving public health, safety, or well-being

Improving public services & institutions

Increasing recreational opportunities

Improving the diversity (age, gender, race, ethnicity, economic status) of the population

Increasing knowledge and educational levels of the population

Other quality of life measures (please specify):

Thank you very much for participating in the Appalachian Regional Commission Program 

Evaluation survey. If you have any questions, please contact Chris Mongelli at 

christopher.mongelli@hdrinc.com or 617-603-6352.
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

204 | September 2015

Appalachian Regional Commission

Program Evaluation of Job Creation and Retention Projects 

Questionnaire

May 2014

This short questionnaire should not take more than 20 minutes to complete. All information will 

be aggregated prior to release. Your responses will be kept confidential. 

The results of this questionnaire will help the Appalachian Regional Commission fulfill its 

mission to promote economic development in the region, and assist current and future grantees 

with improving project performance measurement. If you have any questions, please contact 

Chris Mongelli (617-603-6352 or christopher.mongelli@hdrinc.com) or Julie Marshall (202-884-

7790 or jmarshall@arc.gov). Thank you very much for participating.

SECTION A: Background Information 

1. In order to track responses and enable us to follow-up with you, we ask that you 

provide your contact information.

• Name of person completing this questionnaire

• Organization

• Address

• City

• State

• Zip

• Phone

• Email

• Website

2. What was the ARC project title?

3. What was the ARC project ID (for example, AL-12345)?

4. This project primarily benefited (select all that apply):

• Households

• New businesses

• Existing small to medium-sized businesses

• Business / industrial park(s)

• Entrepreneurs

• Youth

• Residents

• Government officials

• Non-profit organizations

• Conference attendees

• Downtown district

• Public services

• Researchers

• Students

• Policy makers

• Healthcare providers

• Future ARC projects

• Other (please specify)
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

September 2015 | 205

5. The primary goal(s) of the project include (select all that apply):

a. Business assistance

b. Attracting visitors

c. Promoting arts and culture

d. Environmental preservation

e. Developing energy industries

f. Improving energy efficiency

g. Supporting local food production and sales

h. Other (please specify):

6. Based on the following definitions, how would you classify the project?

a. Local Food: A project that aims to start or develop a farm or a business, incubator, etc. 

in the agriculture or culinary fields.

b. Child Care / Education: A project that builds or expands a school, designs a new class 

or curriculum, sets up a day care, or benefits children or students in some other way.

c. Community Building / Tourism: A project that looks to enhance communities through 

building, renovating or marketing tourist attractions or fixing associated infrastructure.

d. Energy / Environmental: A project that looks to build a power plant, improve energy 

efficiency, or protect the environment.

e. Planning / Research: A project that looks to develop a plan, possibly associated with 

one of the other categories, or conduct some form of research.

f. Business / Training: A project that creates a business or incubator for an industry not 

listed in other categories a-e above. This type of project may also involve training 

employees or adults that are not considered students.

g. Other: Please describe.

Energy / Environmental Projects

Additional activities undertaken by this project include (check all that apply):

7. Transportation:

a. Improvement to a railway

b. Widened sidewalks/added bike lanes

c. Constructed a trail

d. None of the above

e. Other (please specify):

8. Community Revitalization:

a. Hired a marketing coordinator

b. Create a Main Street program

c. Streetscaping or infrastructure improvements

d. Erosion control and land stabilization

e. Flood berms

f. Hazard mitigation

g. Flood-recovery coordination

h. Homeland security

i. None of the above

j. Other (please specify):
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

206 | September 2015

9. Please use this box to provide any additional information, including: data sources; 

estimation methods; or anything else that may be relevant.

Section B: Project Outputs 

What specific results (outputs) were actually achieved by this project? These questions attempt 

to capture the characteristics, capacity and direct outputs of the job creation and retention 

project. Depending on your project timeframe, include CUMULATIVE outputs up to 3 years after 

project closeout.

In the Output column, please provide a numerical estimate. If you did not track information for a 

specific output, please enter “DNT” in the corresponding box in the Output column (e.g., you do 

not have information about the number of households served). If an output is not applicable to 

your project, please enter “NA” in the corresponding box in the Output column (e.g., your project 

did not provide services to households).

Use the definitions provided below to determine how a given output should be expressed.

DEFINITIONS:

Outputs: Measures of a program's activities

Households Served: The cumulative total number of households that are served by your ARC 

project, from project inception until now. 

Businesses Served: The cumulative total number of businesses that are served by your ARC 

project, from project inception until now. 

Organizations Served: The cumulative total number of organizations that are served by your 

ARC project, from project inception until now. 

Communities Served: The cumulative total number of communities that are served by your 

ARC project, from inception until now.

Visitors Served: The number of new visitors to a destination resulting from an ARC investment.

Students Served: The total number of students served by the ARC project. For example, the 

number of students enrolling in a new community program the year of the project and for three 

years after.

10. Group Outputs: Output

Households Served

Businesses Served

Organizations Served

Communities Served

Visitors Served

Students Served

11. List any other outputs that were measured or evaluated (if no other outputs were measured, 

enter “NA” into the box): ___________________________
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

September 2015 | 207

12. Please use this box to provide any additional information, including: data sources; 

estimation methods; whether or not outputs vary over time; or anything else that may be 

relevant (if there is no additional information, enter “NA” into the box).

Section C: Project Outcomes

Please provide the following information about final project achievements and results. 

These questions attempt to capture the direct outcomes of the project. Depending on your 

project timeframe, include CUMULATIVE outcomes for 3 years after project closeout.

In the Outcomes column, please provide a numerical estimate. If you did not track information 

for a specific outcome, please enter “DNT” in the corresponding box in the Outcomes column. If 

an outcome measure is not applicable to your project, please enter “NA” in the corresponding 

box in the Outcomes column.

In the Year(s) column, please indicate the number of years for which the outcome measure was 

tracked. 

Use the definitions provided below to determine how a given outcome should be expressed.

DEFINITIONS:

Outcomes: Changes that result from the activities

Households Improved: The cumulative total number of households with a measurable 

improvement due to the ARC project. This number is ALWAYS the same as, or a subset of, the 

“Households Served” output measure.

Businesses Improved: The cumulative total number of businesses with a measurable 

improvement due to the ARC project. This number is ALWAYS the same as, or a subset of, the 

“Businesses Served” output measure.

Organizations Improved: The cumulative total number of organizations with a measurable 

improvement due to the ARC project. This number is ALWAYS the same as, or a subset of, the 

“Organizations Served” output measure.

Communities Improved: The cumulative total number of communities with a measurable 

improvement due to the ARC project. This number is ALWAYS the same as, or a subset of, the 

“Communities Served” output measure.

Jobs Created: The total number of (1) direct hires made as a result of the project’s operation 

(e.g. teachers hired, public safety, etc.); and (2) direct private sector jobs created following the 

delivery of ARC-funded services or project completion. This does NOT include construction jobs 

to build the ARC funded project. In the case of part-time jobs, please convert these to full-time 

equivalent and round up to report whole numbers.

Jobs Retained: The total number of jobs that would be lost or relocated without the ARCfunded project. Existing jobs benefiting from an infrastructure upgrade or other ARC-funded 

project should not be counted as “Jobs Retained.”

Businesses Created: The total number of businesses that located in the region as a direct 

result of the ARC-supported project within three years of the project end date..

Businesses Retained: The total number of existing businesses that remained in the region due 

to the ARC-supported project.

Leveraged Private Investment (LPI): The total dollar amount of private sector financial 

commitments and investment that were not part of the project funding, but followed as a result 

of the completion of the ARC project. 
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

208 | September 2015

Revenues Increased (non-export): The total dollar amount of any increase in non-export 

(domestic) sales that occurred among businesses as a result of the ARC project.

Revenues Increased (export): The total dollar amount of any increase in export (international) 

sales that occurred among businesses as a result of the ARC project.

Costs Reduced: The total dollar amount of costs reduced as a result of the ARC project 

activities, within one year of project implementation. For example, small business technical 

assistance may help a business streamline and cut costs, or an energy efficiency program may 

help to reduce energy costs.

13. & 14. Group Outcomes: Outcomes Year(s)

Households Improved

Businesses Improved

Organizations Improved

Communities Improved

15. & 16. Economic Outcomes:

Jobs Created

Jobs Retained

Businesses Created

Businesses Retained

Leveraged Private Investment ($)

Revenues Increased (non-export, 

$)

Revenues Increased (export, $)

Costs Reduced ($)

17. In what industries were the jobs created or retained? Please add a quantified number of 

jobs for each affected industry. 

• Agriculture

• Mining

• Construction

• Manufacturing

• Transportation Utilities

• Wholesale

• Retail

• Finance, Insurance or Real Estate

• Services

• Other

18. List any other outcomes that were measured or evaluated: __________________________
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

September 2015 | 209

19. Please provide relevant information about how the outcome measures were calculated

such as data sources used or key assumptions. 

Section D: Other Impacts of ARC Funding

20. To what extent are the estimated outcomes attributable to the ARC intervention and 

funding?

a. Entirely

b. Mostly

c. Somewhat

d. Slightly

e. Too difficult to determine 

21. What do you think would have happened to the project if ARC funding had not been 

available?

a. Would have been completed with other funds in approximately the same time period

b. Would have been delayed for up to a year

c. Would have been delayed for a year or more

d. Would have been completed on a smaller scale

e. Would not have been undertaken

22. Aside from the direct jobs and investment associated with the project, were there any other 

positive or negative economic impacts on the community? If so, please explain and quantify 

if possible.

23. Did the project help attract any additional government or philanthropic funding? Please 

describe.

24. Please describe any private investment not directly related to the project that occurred as a 

result of the project. 

25. Please provide any additional comments on the project in terms of accomplishments, 

challenges, or other relevant information to help with the program evaluation.

Section E: Long-Range Impacts

Using the measures below, please rate the long-term impact of your project on the economy, 

the environment, and the quality of life in your community. 

DESCRIPTION OF CHOICES:

None: Project had little to no impact on the measure’s current trend

Slight: Project impact was not large enough to reverse or stabilize the measure’s current trend

Moderate: Project impact contributed to the stabilization or reversal of the measure’s current 

trend

High: Project impact was responsible for significant improvement in the measure’s current trend
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

210 | September 2015

26. Economic Measures None Slight Moderate High

What impact did this project have on:

Attracting new residents or stabilizing the area’s population

Attracting new jobs or increasing employment at existing businesses

Creating new sources of income or increasing income for local residents

Increasing local business sales or the value of business assets (such as equipment, real 

estate)

Increasing the value of household assets (such as homes, land, farms) for local residents

Increasing the value of community assets (such as community buildings, schools, 

infrastructure, parks)

Other economic measures (please specify):

27. Competitiveness Measures None Slight Moderate High

What impact did this project have on:

Improving the stability and sustainability of the local economy

Improving the efficiency of business operations or public services

Improving the productivity of students, employees, businesses, land, or other assets

Improving the skill level of the workforce

Increasing the viability of local businesses

Improving access to markets for local products, businesses, artisans, and entrepreneurs

Reducing the cost of doing business

Other competitiveness measures (please specify):

28. Environmental Measures None Slight Moderate High

What impact did this project have on:

Improving air or water quality

Improving quality and/or access to land and natural resources

Waste reduction or improving waste management (reuse, recycling, alternative energy, 

biofuels)

Improving energy security and independence

Increasing energy efficiency or conservation

Other environmental measures (please specify):

29. Quality of Life Measures None Slight Moderate High

What impact did this project have on:

Improving access to culture, arts, historic sites, and other amenities

Improving civic life and governance

Improving public health, safety, or well-being

Improving public services & institutions

Increasing recreational opportunities

Improving the diversity (age, gender, race, ethnicity, economic status) of the population

Increasing knowledge and educational levels of the population

Other quality of life measures (please specify):

Thank you very much for participating in the Appalachian Regional Commission Program 

Evaluation survey. If you have any questions, please contact Chris Mongelli at 

christopher.mongelli@hdrinc.com or 617-603-6352.
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

September 2015 | 211

Appalachian Regional Commission

Program Evaluation of Job Creation and Retention Projects 

Questionnaire

May 2014

This short questionnaire should not take more than 20 minutes to complete. All information will 

be aggregated prior to release. Your responses will be kept confidential. 

The results of this questionnaire will help the Appalachian Regional Commission fulfill its 

mission to promote economic development in the region, and assist current and future grantees 

with improving project performance measurement. If you have any questions, please contact 

Chris Mongelli (617-603-6352 or christopher.mongelli@hdrinc.com) or Julie Marshall (202-884-

7790 or jmarshall@arc.gov). Thank you very much for participating.

Section A: Background Information 

1. In order to track responses and enable us to follow-up with you, we ask that you 

provide your contact information.

• Name of person completing this questionnaire

• Organization

• Address

• City

• State

• Zip

• Phone

• Email

• Website

2. What was the ARC project title?

3. What was the ARC project ID (for example, AL-12345)?

4. This project primarily benefited (select all that apply):

• Households

• New businesses

• Existing small to medium-sized businesses

• Business / industrial park(s)

• Entrepreneurs

• Youth

• Residents

• Government officials

• Non-profit organizations

• Conference attendees

• Downtown district

• Public services

• Researchers

• Students

• Policy makers

• Healthcare providers

• Future ARC projects

• Other (please specify)
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

212 | September 2015

5. The primary goal(s) of the project include (select all that apply):

a. Business assistance

b. Attracting visitors

c. Promoting arts and culture

d. Environmental preservation

e. Developing energy industries

f. Improving energy efficiency

g. Supporting local food production and sales

h. Other (please specify)

6. Based on the following definitions, how would you classify the project?

a. Local Food: A project that aims to start or develop a farm or a business, incubator, etc. 

in the agriculture or culinary fields.

b. Child Care / Education: A project that builds or expands a school, designs a new class 

or curriculum, sets up a day care, or benefits children or students in some other way.

c. Community Building / Tourism: A project that looks to enhance communities through 

building, renovating or marketing tourist attractions or fixing associated infrastructure.

d. Energy / Environmental: A project that looks to build a power plant, improve energy 

efficiency, or protect the environment.

e. Planning / Research: A project that looks to develop a plan, possibly associated with 

one of the other categories, or conduct some form of research.

f. Business / Training: A project that creates a business or incubator for an industry not 

listed in other categories a-e above. This type of project may also involve training 

employees or adults that are not considered students.

g. Other: Please describe.

Local Food Projects

Additional activities undertaken by this project include (check all that apply):

7. Technical Assistance:

a. Assist businesses in developing marketing strategies

b. Implement technology transfer program

c. Facilitate business training course

d. Work with potential entrepreneurs to develop business plan

e. Provide expertise in areas of finance or accounting

f. None of the above

g. Other (please specify):

8. Business Development:

h. Construction or rehabilitation of building

i. Construction of business incubation building

j. Reuse of existing brownfield

k. None of the above

l. Other (please specify):

9. Please use this box to provide any additional information, including: data sources; 

estimation methods; or anything else that may be relevant.
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

September 2015 | 213

Section B: Project Outputs 

What specific results (outputs) were actually achieved by this project? These questions attempt 

to capture the characteristics, capacity and direct outputs of the job creation and retention 

project. Depending on your project timeframe, include CUMULATIVE outputs up to 3 years after 

project closeout.

In the Output column, please provide a numerical estimate. If you did not track information for a 

specific output, please enter “DNT” in the corresponding box in the Output column (e.g., you do 

not have information about the number of households served). If an output is not applicable to 

your project, please enter “NA” in the corresponding box in the Output column (e.g., your project 

did not provide services to households).

Use the definitions provided below to determine how a given output should be expressed.

DEFINITIONS:

Outputs: Measures of a program's activities

Households Served: The cumulative total number of households that are served by your ARC 

project, from project inception until now. 

Businesses Served: The cumulative total number of businesses that are served by your ARC 

project, from project inception until now. 

Organizations Served: The cumulative total number of organizations that are served by your 

ARC project, from project inception until now. 

Communities Served: The cumulative total number of communities that are served by your 

ARC project, from inception until now.

Visitors Served: The number of new visitors to a destination resulting from an ARC investment.

Students Served: The total number of students served by the ARC project. For example, the 

number of students enrolling in a new community program the year of the project and for three 

years after.

10. Group Outputs: Output

Households Served

Businesses Served

Organizations Served

Communities Served

Visitors Served

Students Served

11. List any other outputs that were measured or evaluated (if no other outputs were measured, 

enter “NA” into the box): ___________________________

12. Please use this box to provide any additional information, including: data sources; 

estimation methods; whether or not outputs vary over time; or anything else that may be 

relevant (if there is no additional information, enter “NA” into the box).
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

214 | September 2015

Section C: Project Outcomes

Please provide the following information about final project achievements and results. 

These questions attempt to capture the direct outcomes of the project. Depending on your 

project timeframe, include CUMULATIVE outcomes for 3 years after project closeout.

In the Outcomes column, please provide a numerical estimate. If you did not track information 

for a specific outcome, please enter “DNT” in the corresponding box in the Outcomes column. If 

an outcome measure is not applicable to your project, please enter “NA” in the corresponding 

box in the Outcomes column.

In the Year(s) column, please indicate the number of years for which the outcome measure was 

tracked. 

Use the definitions provided below to determine how a given outcome should be expressed.

DEFINITIONS:

Outcomes: Changes that result from the activities

Households Improved: The cumulative total number of households with a measurable 

improvement due to the ARC project. This number is ALWAYS the same as, or a subset of, the 

“Households Served” output measure.

Businesses Improved: The cumulative total number of businesses with a measurable 

improvement due to the ARC project. This number is ALWAYS the same as, or a subset of, the 

“Businesses Served” output measure.

Organizations Improved: The cumulative total number of organizations with a measurable 

improvement due to the ARC project. This number is ALWAYS the same as, or a subset of, the 

“Organizations Served” output measure.

Communities Improved: The cumulative total number of communities with a measurable 

improvement due to the ARC project. This number is ALWAYS the same as, or a subset of, the 

“Communities Served” output measure.

Jobs Created: The total number of (1) direct hires made as a result of the project’s operation 

(e.g. teachers hired, public safety, etc.); and (2) direct private sector jobs created following the 

delivery of ARC-funded services or project completion. This does NOT include construction jobs 

to build the ARC funded project. In the case of part-time jobs, please convert these to full-time 

equivalent and round up to report whole numbers.

Jobs Retained: The total number of jobs that would be lost or relocated without the ARCfunded project. Existing jobs benefiting from an infrastructure upgrade or other ARC-funded 

project should not be counted as “Jobs Retained.”

Businesses Created: The total number of businesses that located in the region as a direct 

result of the ARC-supported project within three years of the project end date..

Businesses Retained: The total number of existing businesses that remained in the region due 

to the ARC-supported project.

Leveraged Private Investment (LPI): The total dollar amount of private sector financial 

commitments and investment that were not part of the project funding, but followed as a result 

of the completion of the ARC project. 

Revenues Increased (non-export): The total dollar amount of any increase in non-export 

(domestic) sales that occurred among businesses as a result of the ARC project.
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

September 2015 | 215

Revenues Increased (export): The total dollar amount of any increase in export (international) 

sales that occurred among businesses as a result of the ARC project.

Costs Reduced: The total dollar amount of costs reduced as a result of the ARC project 

activities, within one year of project implementation. For example, small business technical 

assistance may help a business streamline and cut costs, or an energy efficiency program may 

help to reduce energy costs.

13. & 14 Group Outcomes: Outcomes Year(s)

Households Improved

Businesses Improved

Organizations Improved

Communities Improved

15. & 16. Economic Outcomes:

Jobs Created

Jobs Retained

Businesses Created

Businesses Retained

Leveraged Private Investment ($)

Revenues Increased (non-export, 

$)

Revenues Increased (export, $)

Costs Reduced ($)

17. In what industries were the jobs created or retained? Please add a quantified number of 

jobs for each affected industry. 

• Agriculture

• Mining

• Construction

• Manufacturing

• Transportation Utilities

• Wholesale

• Retail

• Finance, Insurance or Real Estate

• Services

• Other

18. List any other outcomes that were measured or evaluated: __________________________

19. Please provide relevant information about how the outcome measures were calculated

such as data sources used or key assumptions. 
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

216 | September 2015

Section D: Other Impacts of ARC Funding

20. To what extent are the estimated outcomes attributable to the ARC intervention and 

funding? 

a. Entirely

b. Mostly

c. Somewhat

d. Slightly

e. Too difficult to determine 

21. What do you think would have happened to the project if ARC funding had not been 

available?

a. Would have been completed with other funds in approximately the same time period

b. Would have been delayed for up to a year

c. Would have been delayed for a year or more

d. Would have been completed on a smaller scale

e. Would not have been undertaken

22. Aside from the direct jobs and investment associated with the project, were there any other 

positive or negative economic impacts on the community? If so, please explain and quantify 

if possible.

23. Did the project help attract any additional government or philanthropic funding? Please 

describe.

24. Please describe any private investment not directly related to the project that occurred as a 

result of the project. 

25. Please provide any additional comments on the project in terms of accomplishments, 

challenges, or other relevant information to help with the program evaluation.

Section E: Long-Range Impacts

Using the measures below, please rate the long-term impact of your project on the economy, 

the environment, and the quality of life in your community. 

DESCRIPTION OF CHOICES:

None: Project had little to no impact on the measure’s current trend

Slight: Project impact was not large enough to reverse or stabilize the measure’s current trend

Moderate: Project impact contributed to the stabilization or reversal of the measure’s current 

trend

High: Project impact was responsible for significant improvement in the measure’s current trend
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

September 2015 | 217

26. Economic Measures None Slight Moderate High

What impact did this project have on: 

Attracting new residents or stabilizing the area’s population

Attracting new jobs or increasing employment at existing businesses

Creating new sources of income or increasing income for local residents

Increasing local business sales or the value of business assets (such as equipment, real 

estate)

Increasing the value of household assets (such as homes, land, farms) for local residents

Increasing the value of community assets (such as community buildings, schools, 

infrastructure, parks)

Other economic measures (please specify):

27. Competitiveness Measures None Slight Moderate High

What impact did this project have on:

Improving the stability and sustainability of the local economy

Improving the efficiency of business operations or public services

Improving the productivity of students, employees, businesses, land, or other assets

Improving the skill level of the workforce

Increasing the viability of local businesses

Improving access to markets for local products, businesses, artisans, and entrepreneurs

Reducing the cost of doing business

Other competitiveness measures (please specify):

28. Environmental Measures None Slight Moderate High

What impact did this project have on:

Improving air or water quality

Improving quality and/or access to land and natural resources

Waste reduction or improving waste management (reuse, recycling, alternative energy, 

biofuels)

Improving energy security and independence

Increasing energy efficiency or conservation

Other environmental measures (please specify):

29. Quality of Life Measures None Slight Moderate High

What impact did this project have on:

Improving access to culture, arts, historic sites, and other amenities

Improving civic life and governance

Improving public health, safety, or well-being

Improving public services & institutions

Increasing recreational opportunities

Improving the diversity (age, gender, race, ethnicity, economic status) of the population

Increasing knowledge and educational levels of the population

Other quality of life measures (please specify):

Thank you for participating in the Appalachian Regional Commission Program Evaluation 

survey. If you have any questions, please contact Chris Mongelli at 

christpoher.mongelli@hdrinc.com or 617-603-6352.
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

218 | September 2015

Appalachian Regional Commission

Program Evaluation of Job Creation and Retention Projects 

Questionnaire

May 2014

This short questionnaire should not take more than 20 minutes to complete. All information will 

be aggregated prior to release. Your responses will be kept confidential. 

The results of this questionnaire will help the Appalachian Regional Commission fulfill its 

mission to promote economic development in the region, and assist current and future grantees 

with improving project performance measurement. If you have any questions, please contact 

Chris Mongelli (617-603-6352 or christopher.mongelli@hdrinc.com) or Julie Marshall (202-884-

7790 or jmarshall@arc.gov). Thank you very much for participating.

Section A: Background Information 

1. In order to track responses and enable us to follow-up with you, we ask that you 

provide your contact information.

• Name of person completing this questionnaire

• Organization

• Address

• City

• State

• Zip

• Phone

• Email

• Website

2. What was the ARC project title?

3. What was the ARC project ID (for example, AL-12345)?

4. This project primarily benefited (select all that apply):

• Households

• New businesses

• Existing small to medium-sized businesses

• Business / industrial park(s)

• Entrepreneurs

• Youth

• Residents

• Government officials

• Non-profit organizations

• Conference attendees

• Downtown district

• Public services

• Researchers

• Students

• Policy makers

• Healthcare providers

• Future ARC projects

• Other (please specify)
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

September 2015 | 219

5. The primary goal(s) of the project include (select all that apply):

a. Business assistance

b. Attracting visitors

c. Promoting arts and culture

d. Environmental preservation

e. Developing energy industries

f. Improving energy efficiency

g. Supporting local food production and sales

h. Other (please specify)

6. Based on the following definitions, how would you classify the project?

a. Local Food: A project that aims to start or develop a farm or a business, incubator, etc. 

in the agriculture or culinary fields.

b. Child Care / Education: A project that builds or expands a school, designs a new class 

or curriculum, sets up a day care, or benefits children or students in some other way.

c. Community Building / Tourism: A project that looks to enhance communities through 

building, renovating or marketing tourist attractions or fixing associated infrastructure.

d. Energy / Environmental: A project that looks to build a power plant, improve energy 

efficiency, or protect the environment.

e. Planning / Research: A project that looks to develop a plan, possibly associated with 

one of the other categories, or conduct some form of research.

f. Business / Training: A project that creates a business or incubator for an industry not 

listed in other categories a-e above. This type of project may also involve training 

employees or adults that are not considered students.

g. Other: Please describe.

Planning / Research Projects

Additional activities undertaken by this project include (check all that apply):

7. Research Topics:

a. Performing socioeconomic or demographic analysis

b. Economic impact study 

c. Improving education practices

d. Advancing health technology or awareness

e. Understanding environmental impacts

f. Developing energy efficiency

g. None of the above

h. Other (please specify):

8. Please use this box to provide any additional information, including: data sources; 

estimation methods; or anything else that may be relevant.
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

220 | September 2015

Section B: Project Outputs 

What specific results (outputs) were actually achieved by this project? These questions attempt 

to capture the characteristics, capacity and direct outputs of the job creation and retention 

project. Depending on your project timeframe, include CUMULATIVE outputs up to 3 years after 

project closeout.

In the Output column, please provide a numerical estimate. If you did not track information for a 

specific output, please enter “DNT” in the corresponding box in the Output column (e.g., you do 

not have information about the number of households served). If an output is not applicable to 

your project, please enter “NA” in the corresponding box in the Output column (e.g., your project 

did not provide services to households).

Use the definitions provided below to determine how a given output should be expressed.

DEFINITIONS:

Outputs: Measures of a program's activities

Households Served: The cumulative total number of households that are served by your ARC 

project, from project inception until now. 

Businesses Served: The cumulative total number of businesses that are served by your ARC 

project, from project inception until now. 

Organizations Served: The cumulative total number of organizations that are served by your 

ARC project, from project inception until now. 

Communities Served: The cumulative total number of communities that are served by your 

ARC project, from inception until now.

Visitors Served: The number of new visitors to a destination resulting from an ARC investment.

Students Served: The total number of students served by the ARC project. For example, the 

number of students enrolling in a new community program the year of the project and for three 

years after.

9. Group Outputs: Output

Households Served

Businesses Served

Organizations Served

Communities Served

Visitors Served

Students Served

10. List any other outputs that were measured or evaluated (if no other outputs were measured, 

enter “NA” into the box): ___________________________

11. Please use this box to provide any additional information, including: data sources; 

estimation methods; whether or not outputs vary over time; or anything else that may be 

relevant (if there is no additional information, enter “NA” into the box).
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

September 2015 | 221

Section C: Project Outcomes

Please provide the following information about final project achievements and results. 

These questions attempt to capture the direct outcomes of the project. Depending on your 

project timeframe, include CUMULATIVE outcomes for 3 years after project closeout.

In the Outcomes column, please provide a numerical estimate. If you did not track information 

for a specific outcome, please enter “DNT” in the corresponding box in the Outcomes column. If 

an outcome measure is not applicable to your project, please enter “NA” in the corresponding 

box in the Outcomes column.

In the Year(s) column, please indicate the number of years for which the outcome measure was 

tracked. 

Use the definitions provided below to determine how a given outcome should be expressed.

DEFINITIONS:

Outcomes: Changes that result from the activities

Households Improved: The cumulative total number of households with a measurable 

improvement due to the ARC project. This number is ALWAYS the same as, or a subset of, the 

“Households Served” output measure.

Businesses Improved: The cumulative total number of businesses with a measurable 

improvement due to the ARC project. This number is ALWAYS the same as, or a subset of, the 

“Businesses Served” output measure.

Organizations Improved: The cumulative total number of organizations with a measurable 

improvement due to the ARC project. This number is ALWAYS the same as, or a subset of, the 

“Organizations Served” output measure.

Communities Improved: The cumulative total number of communities with a measurable 

improvement due to the ARC project. This number is ALWAYS the same as, or a subset of, the 

“Communities Served” output measure.

Jobs Created: The total number of (1) direct hires made as a result of the project’s operation 

(e.g. teachers hired, public safety, etc.); and (2) direct private sector jobs created following the 

delivery of ARC-funded services or project completion. This does NOT include construction jobs 

to build the ARC funded project. In the case of part-time jobs, please convert these to full-time 

equivalent and round up to report whole numbers.

Jobs Retained: The total number of jobs that would be lost or relocated without the ARCfunded project. Existing jobs benefiting from an infrastructure upgrade or other ARC-funded 

project should not be counted as “Jobs Retained.”

Businesses Created: The total number of businesses that located in the region as a direct 

result of the ARC-supported project within three years of the project end date..

Businesses Retained: The total number of existing businesses that remained in the region due 

to the ARC-supported project.

Leveraged Private Investment (LPI): The total dollar amount of private sector financial 

commitments and investment that were not part of the project funding, but followed as a result 

of the completion of the ARC project. 

Revenues Increased (non-export): The total dollar amount of any increase in non-export 

(domestic) sales that occurred among businesses as a result of the ARC project.
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

222 | September 2015

Revenues Increased (export): The total dollar amount of any increase in export (international) 

sales that occurred among businesses as a result of the ARC project.

Costs Reduced: The total dollar amount of costs reduced as a result of the ARC project 

activities, within one year of project implementation. For example, small business technical 

assistance may help a business streamline and cut costs, or an energy efficiency program may 

help to reduce energy costs.

12. & 13. Group Outcomes: Outcomes Year(s)

Households Improved

Businesses Improved

Organizations Improved

Communities Improved

14. & 15. Economic Outcomes:

Jobs Created

Jobs Retained

Businesses Created

Businesses Retained

Leveraged Private Investment ($)

Revenues Increased (non-export, 

$)

Revenues Increased (export, $)

Costs Reduced ($)

16. In what industries were the jobs created or retained? Please add a quantified number of 

jobs for each affected industry. 

• Agriculture

• Mining

• Construction

• Manufacturing

• Transportation Utilities

• Wholesale

• Retail

• Finance, Insurance or Real Estate

• Services

• Other

17. List any other outcomes that were measured or evaluated: __________________________

18. Please provide relevant information about how the outcome measures were calculated

such as data sources used or key assumptions. 
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

September 2015 | 223

Section D: Other Impacts of ARC Funding

19. To what extent are the estimated outcomes attributable to the ARC intervention and 

funding? 

a. Entirely

b. Mostly

c. Somewhat

d. Slightly

e. Too difficult to determine 

20. What do you think would have happened to the project if ARC funding had not been 

available?

a. Would have been completed with other funds in approximately the same time period

b. Would have been delayed for up to a year

c. Would have been delayed for a year or more

d. Would have been completed on a smaller scale

e. Would not have been undertaken

21. Aside from the direct jobs and investment associated with the project, were there any other 

positive or negative economic impacts on the community? If so, please explain and quantify 

if possible.

22. Did the project help attract any additional government or philanthropic funding? Please 

describe.

23. Please describe any private investment not directly related to the project that occurred as a 

result of the project. 

24. Please provide any additional comments on the project in terms of accomplishments, 

challenges, or other relevant information to help with the program evaluation.

Section E: Long-Range Impacts

Using the measures below, please rate the long-term impact of your project on the economy, 

the environment, and the quality of life in your community. 

DESCRIPTION OF CHOICES:

None: Project had little to no impact on the measure’s current trend

Slight: Project impact was not large enough to reverse or stabilize the measure’s current trend

Moderate: Project impact contributed to the stabilization or reversal of the measure’s current 

trend

High: Project impact was responsible for significant improvement in the measure’s current trend
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

224 | September 2015

25. Economic Measures None Slight Moderate High

What impact did this project have on: 

Attracting new residents or stabilizing the area’s population

Attracting new jobs or increasing employment at existing businesses

Creating new sources of income or increasing income for local residents

Increasing local business sales or the value of business assets (such as equipment, real 

estate)

Increasing the value of household assets (such as homes, land, farms) for local residents

Increasing the value of community assets (such as community buildings, schools, 

infrastructure, parks)

Other economic measures (please specify):

26. Competitiveness Measures None Slight Moderate High

What impact did this project have on:

Improving the stability and sustainability of the local economy

Improving the efficiency of business operations or public services

Improving the productivity of students, employees, businesses, land, or other assets

Improving the skill level of the workforce

Increasing the viability of local businesses

Improving access to markets for local products, businesses, artisans, and entrepreneurs

Reducing the cost of doing business

Other competitiveness measures (please specify):

27. Environmental Measures None Slight Moderate High

What impact did this project have on:

Improving air or water quality

Improving quality and/or access to land and natural resources

Waste reduction or improving waste management (reuse, recycling, alternative energy, 

biofuels)

Improving energy security and independence

Increasing energy efficiency or conservation

Other environmental measures (please specify):

28. Quality of Life Measures None Slight Moderate High

What impact did this project have on:

Improving access to culture, arts, historic sites, and other amenities

Improving civic life and governance

Improving public health, safety, or well-being

Improving public services & institutions

Increasing recreational opportunities

Improving the diversity (age, gender, race, ethnicity, economic status) of the population

Increasing knowledge and educational levels of the population

Other quality of life measures (please specify):

Thank you very much for participating in the Appalachian Regional Commission Program 

Evaluation survey. If you have any questions, please contact Chris Mongelli at 

christopher.mongelli@hdrinc.com or 617-603-6352.
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

September 2015 | 225

Appendix 2 – Policy Discussion

Interview Questions for Rural Development Experts

Introduction: Hi Mr./Ms. [NAME], thank you for taking the time to speak with us. Before 

we get started, let me take a moment to again quickly run through why we have 

requested this call with you, and then we can go over the topics we would like to discuss.

Background: As we explained in our introductory email, our firm, HDR, has been 

engaged by the Appalachian Regional Commission to evaluate the performance of the 

organization’s Job Creation and Retention investments to evaluate the effectiveness of 

those past investments and provide recommendations for effective use of future funds. 

Our primary goal for this interview is to get a better understanding of major challenges for 

development in rural communities in the US, and what types of solutions are needed. We 

will couple the findings of our discussions with the other results of our evaluation to 

provide insight to ARC on the most effective uses of job creation and retention funds 

going forward.

During the call today we’d like to discuss three main topics: 

• Key challenges for rural communities in Appalachia, and more broadly in the 

US

• Target industries and sectors that the Appalachian region should focus on 

• How to measure the success of investments in rural communities. 

For all the questions, if you can answer specifically for the Appalachian region that would 

be preferable, but your insight on US rural communities in general would also be very 

useful.

Challenges in US rural communities:

1. What are one or two of the major challenges that rural communities face today as 

they try to improve their socio-economic well-being? 

2. Have these challenges changed or evolved over the last two decades? 

3. Do you expect different or additional challenges 10 to 20 years from now? If so, what 

do you anticipate these new challenges will be?

Target industries and sectors for the Appalachians:

1. Do you see an specific industry or sector that presents the greatest opportunities for 

rural communities?

2. Specifically, are there industries or sectors for which rural communities may have a 

competitive advantage? 

3. Have you seen any new, pronounced trends in the types of jobs created in rural 

areas?

4. Do any of these industries or jobs aid in increasing per-capita income in rural areas?

5. How do the trending industries in rural areas fit with global industry and growth 

trends?
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

226 | September 2015

Measuring success of investments:

1. In your opinion, what is the best way to measure the success of an economic 

development investment in US rural communities? 

a. For instance, ARC currently looks at indicators like change in per capita income, 

employment rate, and education rate

2. How much time should pass before these indicators are measured? Does it depend 

on the type of industry/sector?

3. Based on the project analysis portion of this evaluation, preliminary findings indicate 

that most of ARC’s Job Creation and Retention investments are resulting in higher 

impacts than originally estimated. Does this result align with your sense of the overall 

success of job creation and retention investments in general in US rural 

communities, or the Appalachian region specifically?

4. Do you know of any new developments in program evaluation and/or best practices 

in promoting economic development that the ARC should be aware of?

5. Which types of investments give the “biggest bang for the buck” in the long run? 

Which projects have the fastest impact? Is there any overlap between the two? 

6. Are some types of investments harder to manage than others? Do some types of 

investments come with a greater risk of uncertainty in their chances of success or 

the amount of impact? 

7. Do you have any suggestions for best-practices for evaluating investment or grant 

programs?

The role of funders in the economic development of the Appalachians:

1. In your opinion, what role do you think the government (federal and local) and private 

organizations (non-profit and for-profit) should play in fostering greater economic 

well-being in rural communities? For example, are these entities meant to catalyze 

growth and then step away? 

2. Have the roles of funders changed in recent years?

Thank you so much for your time. 

• Would it be okay to contact you again with additional questions if necessary?

• Would it be ok to use your name in our report to ARC? 

• Do you have any suggestions on people or organizations that we should 

speak with? Do you have their contact information?
Program Evaluation of the Appalachian Regional Commission’s Job Creation and Retention Projects

Appalachian Regional Commission

September 2015 | 227

Appendix 3 – Policy Interviewees

• Greg Bishak – Financial Strategies and Research Manager, Community 

Development Financial Institutions Fund, Department of the Treasury

• Rick Cohen – Author of “What Ails Rural Philanthropy and What Must Be 

Done”

• Brian Dabson – Associate Dean, Research Professor and Director, Institute 

of Public Policy, Harry S. Truman School of Public Affairs at the University of 

Missouri

• Edward Feser – Dean at the University of Illinois; Author of “Economic 

Diversity in Appalachia: Statistics, Strategies, and Guides for Action” 

prepared for the Appalachian Regional Commission

• Deborah Markley – Co-Director of Center for Rural Entrepreneurship

• Joe McKinney – Executive Director at the National Association of 

Development Associations

• Scott Miller – Professor at Ohio University

• Michael Oden – Economic development and program evaluation specialist 

from the University of Texas

• Terance Rephann – Regional Economist at the Weldon Cooper Center for 

Public Services, University of Virginia; co-author with Andrew Isserman of 

“The Economic Effects of the Appalachian Regional Commission: An 

Empirical Assessment of 26 Years of Regional Development Planning”

• Robin Stewart – Professor at Ohio University, Voinovich School of 

Leadership and Public Affairs

• Lawrence Wood – Specializes in rural and regional development at Ohio 

University