August 2013

Program Evaluation of the 

Appalachian Regional 

Commission’s Infrastructure & 

Public Works Projects 

FINAL REPORT

DEVELOPED FOR THE

Appalachian Regional Commission 

1666 Connecticut Avenue 

Washington, DC 20009-1068

BY

HDR Decision Economics 

695 Atlantic Avenue, 2nd Floor 

Boston, MA 02111-2626 

with 

Cambridge Systematics 

Economic Development Research 

Group 

and

 Mt. Auburn Associates 
Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

Final Report 1

FINAL REPORT 

TABLE OF CONTENTS 

1 Executive Summary ............................................................................................................................... 5

1.1 Program Evaluation Review .......................................................................................................... 7

1.2 Performance Measure Database .................................................................................................. 8

1.3 Case Studies of Best Practices for Infrastructure .......................................................................10

1.4 Study Findings and Recommendations ......................................................................................12

2 Introduction ..........................................................................................................................................15

3 Definition of Evaluation Criteria and Review of Past Program Evaluations .........................................16

3.1 Program Evaluation Methods ......................................................................................................17

3.2 Comparison to Current Program Evaluation Approaches Used by ARC ....................................24

3.3 Summary and Conclusions .........................................................................................................26

4 Review of Past Reports and Program Evaluations ..............................................................................31

4.1 “Evaluation of the Appalachian Regional Commission’s Infrastructure and Public Works 

Program Projects” ....................................................................................................................................31

4.2 “Program Evaluation of the Appalachian Regional Commission’s Infrastructure and Public 

Works Projects” .......................................................................................................................................32

4.3 “Handbook: Assessing Local Economic Development Opportunities with ARC-LEAP 

(Appalachian Regional Commission Local Economic Assessment Package)” .......................................34

4.4 “Understanding Water and Sewer Reporting Issues” ................................................................. 36

4.5 “Drinking Water and Wastewater Infrastructure in Appalachia: An Analysis of Capital Funding 

and Funding Gaps” ..................................................................................................................................37

4.6 “Government Performance and Results Act of 1993” ................................................................. 38

4.7 OMB Circular A-11, “Preparation, Submission, and Execution of the Budget Applicable to 

Federal Agencies” ...................................................................................................................................39

4.8 “Moving Appalachia Forward: Appalachian Regional Commission Strategic Plan 2011-2016” . 41

4.9 “The Magenta Book: Guidance for Evaluation; United Kingdom Cross-Government Evaluation 

Group” ....................................................................................................................................................43

4.10 “Program Evaluation Methods: Measurement and Attribution of Program Results, Third Edition” 

 ....................................................................................................................................................45

4.11 “Designing Evaluations” ..............................................................................................................48

4.12 “Public Works Program Performance Evaluation” ......................................................................50

5 Performance Measure Database .........................................................................................................52

5.1 Discussion of Performance Measure Database ..........................................................................52

5.2 Analysis and Summary of Project Performance Measure Database ..........................................54

5.3 Performance Measure Conclusions ............................................................................................78

6 Case Studies of Best Practices for Infrastructure ................................................................................80

6.1 Barton Business Park Sewage Treatment Plant – Cumberland, MD .........................................82
2 Final Report

Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

6.2 Big Caney Water Project Phase II – Dickenson County, VA ......................................................85

6.3 Burson Center for Business Development – Carrollton, GA .......................................................89

6.4 Canaseraga Wastewater Collection and Treatment System – Canaseraga, NY .......................94

6.5 Canton Water Improvement Project - Canton Borough, PA .......................................................97

6.6 Clinton I-75 Industrial Park Sewer Line – Clinton, TN...............................................................100

6.7 Dobson I-77 Infrastructure Project – Dobson, NC ....................................................................104

6.8 Muscle Shoals Industrial Infrastructure – Muscle Shoals, AL ...................................................110

6.9 Owingsville/Bath County Industrial Park Wastewater Treatment Plant – Owingsville, KY .......114

6.10 Powdersville Water Storage Tank Project – Powdersville, SC .................................................117

6.11 Racine Water Treatment Plant and Storage – Racine, OH ......................................................120

6.12 Wastewater Treatment Solutions for Small Communities – Canaan Valley Institute, WV .......124

6.13 Water Valley Sewer and Water Improvements – Water Valley, MS .........................................129

6.14 Case Study Summary Findings and Recommendations ..........................................................132

7 Findings and Recommendations .......................................................................................................143

7.1 Key Findings..............................................................................................................................143

7.2 Recommendations ....................................................................................................................144

Appendix A: Appalachian Regional Commission Program Evaluation of Infrastructure and Public Works 

Questionnaire ........................................................................................................................................145

Appendix B: Sample Letter to Grant Recipients ....................................................................................145

Appendix C: Full Database Distribution by Project Type and State ......................................................145

Appendix D: Survey Response Project Types and Responses ............................................................145

Appendix E: Outlier Detection and Removal .........................................................................................145

Appendix F: Regression Analysis Predictive Ability ..............................................................................145
Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

Final Report 3

TABLE OF FIGURES 

Figure ES-1: ARC Region ............................................................................................................................. 5 

Figure ES-2: Number of Projects by State and Infrastructure Type, 2004-2010 Survey Responses........... 8 

Figure 1: Project Distribution by Self-Identified Infrastructure Type, Survey Responses (n=197) .............56

Figure 2: ARC Investment ($) by Year and Self-Identified Infrastructure Type, Survey Responses (n=197)

....................................................................................................................................................................57

Figure 3: ARC Funding by State and Self-Identified Infrastructure Type, Survey Responses (n=197) .....58

Figure 4: Number of Projects by State and Self-Identified Infrastructure Type, Survey Responses (n=197)

....................................................................................................................................................................59

Figure 5: ARC Investments by Economic Status – Dollars of Investment, Survey Responses (n=197) ....60

Figure 6: ARC Investments by Economic Status – Number of Projects Invested, Survey Responses 

(n=197) ........................................................................................................................................................60

Figure 7: Number of Jobs Created or Retained by State, Survey Responses Closed Projects (n=30) .....62

Figure 8: Number of Jobs Created or Retained by Self-Identified Infrastructure Type, Survey Responses 

Closed Projects (n=30) ...............................................................................................................................62

Figure 9: Jobs Created or Retained by Industry, Survey Responses Closed Projects (n=30) ...................63

Figure 10: Number of Businesses Created or Retained by State, Survey Responses Closed Projects 

(n=30) ..........................................................................................................................................................64

Figure 11: Number of Businesses Created/Retained by Self-Identified Infrastructure Type, Survey 

Responses Closed Projects (n=30) ............................................................................................................65

Figure 12: Qualitative Survey Responses Regarding Economic Impacts (n=197) .....................................67

Figure 13: Qualitative Survey Responses Regarding Competitiveness Measures (n=197).......................68

Figure 14: Qualitative Survey Responses Regarding Environmental Measures (n=197) ..........................69

Figure 14: Access Road Projects: Difference between Estimated and Observed Job Creation ..............162

Figure 15: Water/Sewer Projects: Difference between Estimated and Observed Job Creation ..............163

Figure 16: Industrial Site Projects: Difference between Estimated and Observed Job Creation .............163
4 Final Report

Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

TABLE OF TABLES 

Table ES-1: Rating of Approaches and Methods against Criteria......................................................6 

Table ES-2: Number of Projects by State and Infrastructure Type, 2004-2010 Survey Responses...........9 

Table ES-3: Case Study Overview............................................................................................10

Table 1: Summary of Approaches and Methods.........................................................................................26

Table 2: Rating of Approaches and Methods against Criteria ....................................................................28

Table 3: Lessons Learned from Tasks 1 & 2 and Recommendations ........................................................29

Table 4: Number and Share of Grants Received and Survey Responses by State ...................................52

Table 5: Number and Share of Survey Responses by Project Sub-Type ..................................................53

Table 6: Survey Responses by Self-Identified Project Type and State ......................................................54

Table 7: Comparison of Predicted and Actual Household Served and Jobs Created and Retained .........66

Table 8: Number of Projects and Projects with Jobs Data by Type for Full ARCnet and Survey Responses

....................................................................................................................................................................71

Table 9: Comparison of Predicted and Actual Job Creation .......................................................................72

Table 10: Average Job Creation by Project Type for Predicted and Actual Jobs .......................................73

Table 11: Explanatory Variables Tested in Regression Analysis ...............................................................74

Table 12: Regression Results: Predicted Job Creation (ARCnet Database) .............................................75

Table 13: Regression Results: Actual Job Creation (Survey Results) .......................................................76

Table 14: Regression Results: Intermediate Outcomes as a Function of Total Spending ($1 Million) 

(Survey Results) ..........................................................................................................................................77

Table 15: Regression Results: Actual Jobs Created as a Function of Intermediate Outcome (Survey 

Results) .......................................................................................................................................................77

Table 16: Case Study Summary .................................................................................................................80
Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

Final Report 5

1 EXECUTIVE SUMMARY

Highlights 

Methodology 

 The 811 Infrastructure Grant recipients from 2004-2010 provided information on their initial 

application regarding various outcome measures, including job creation and retention potential, 

business creation and retention potential, and households served. 

 Evaluation of program success was conducted using data obtained through a combination of 

surveys and case studies. 

o Survey responses were received from all Appalachian Regional Commission (ARC) 

states and all project types, though not all were equally represented. Grantees from 

Kentucky provided the most responses of any state, water and sewer projects the most 

responses of any type. 

o Thirteen case studies of exemplary projects – one for each state – were conducted for 

projects that met or exceeded expectations. 

Findings 

 Overall, survey response demographics were consistent with the full dataset demographics. The 

largest share of funding went to West Virginia, and among infrastructure types to water and 

sewer projects. Transitional counties had the greatest number of projects funded and the largest 

share of investment was in distressed counties. 

 ARC funding supported projects that created and retained jobs and businesses and showed a 

positive impact on both intermediate measures and long-term outcomes: 

o The 21 projects with sufficient information to facilitate a comparison showed actual job 

creation of 5,051 new jobs, exceeding predictions of 4,181 jobs, despite the recession. 

o Job retention of 1,379 jobs fell short of predictions (2,354 jobs) for the eight projects with 

both predicted and actual values. 

 Survey responses indicated a total of 27,488 households were served by 37 projects, more than 

double the projections provided in the original applications. 

 Predicted job creation rises with project expenditure: $1 million in spending results in 151 access 

road jobs, 52 water/sewer jobs, and 129 industrial site development jobs. 

Lessons Learned 

 Consider the project in the context of broader strategic vision. 

 Think and plan ahead. 

 Tap into outside expertise and resources. 

 Consider how to fully leverage investment and develop strong relationships with funders. 

Recommendations for Future Programs 

 ARC Infrastructure Grant program has shown success in terms of economic development but 

data collection and record keeping for evaluation could use improvement. 

o Require tracking of outputs and outcomes as a stipulation of grant receipt. 

o Clarify project categories in the application. 

o Collect additional information related to funding sources. 

 Consider new types of infrastructure investment to increase competitiveness. 
6 Final Report

Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

ARC investments fund locally developed public works infrastructure projects to assist localities in 

establishing and supporting private-sector businesses. The purpose of these grants is to enhance 

economic development by attracting new industry, encouraging business expansion, diversifying local 

economies, and generating permanent, private-sector jobs. 

The ARC has four main goals, as laid out in the 2011-2016 Strategic Plan: 

1. Increase job opportunities and per capita income in Appalachia to reach parity with the nation. 

2. Strengthen the capacity of the people of Appalachia to compete in the global economy. 

3. Develop and improve Appalachia’s infrastructure to make the region economically competitive. 

4. Build the Appalachian Development Highway System to reduce Appalachia’s isolation.

The performance of the ARC’s infrastructure investment program is closely related to these goals, and the

ARC has commissioned this program evaluation to estimate the effectiveness of regional infrastructure 

and public works investments. The study examines projects from all 13 ARC states. The ARC region is 

outlined in Figure ES-1 below. 

Figure ES-1: ARC Region 

Source: Appalachian Regional Commission 

The purpose of this study is to determine the extent to which ARC allocated funding of infrastructure 

projects contributed to the attainment of the ARC’s economic development objectives set forth in its 

strategic plan. The study searches for correlations between types of projects, levels of funding, and real 

project outcomes to assess the allocation, impact and cost effectiveness of these programs. 

The findings of the study suggest that the grant program is successful in achieving many of ARC’s 

economic development objectives, but could be improved from a data collection and program evaluation 
Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

Final Report 7

perspective. The following provides a more complete overview of the study’s findings and offers some 

recommendations for enhancing the evaluation aspects of the grant program. 

1.1 Program Evaluation Review 

This project commenced with a review of prior ARC Program Evaluations and other agency measures to 

determine the evaluation methods best suited for use in this particular study. The review covered 12 

studies from the ARC and other government agencies and included various evaluation approaches: 

surveys, observation/site visits, case studies, analysis of experimental data, and statistical analysis among 

others. 

The strengths and weaknesses of each evaluation approach, as well as their applicability to the ARC 

infrastructure investment program evaluation, were summarized. The merits of each evaluation method 

were rated against criteria to determine which was best suited for this evaluation. The comparison is 

shown in Table ES-1 below. Based on the study reviews, and giving consideration to the time and 

resource constraints associated with the project, the team recommended using a combination of surveys 

and case studies for the evaluation. 

Table ES-1: Rating of Approaches and Methods against Criteria 

Criteria

Ease of Use 

for ARC

Ease of 

Interpretation

Flexibility

Data 

Availability

Estimation of 

Counterfactual

Transparency

Acceptance & 

Credibility

Approaches & Methods

Surveys     No  

Observations or Site Visits     No  

Analysis of Experimental Data     Yes  

Before & After Comparisons     No  

Theory Based Evaluation & Modeling     Sometime  

Case Studies  







Generally 

Not  

Application of Findings from Other 

Studies     No  

Use of Expert Opinion    

Not 

Explicitly  

Statistical Analysis and Use of Causal 

Models     Sometime  

Input-Output Modeling     No  

Benefit-Cost Analysis     Yes, Needed  

Sensitivity Analysis or Risk Analysis     N/A  
8 Final Report

Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

1.2 Performance Measure Database 

Following the program evaluation review, detailed in the previous section, the HDR Team examined the 

ARCnet database of 811 infrastructure projects funded by the ARC between FY2004 and FY2010. Using 

information provided in the database, correspondence announcing an online-survey was distributed to 

all grantees, based on the database. A survey was then developed to obtain information related to the 

individual projects, as well as the outputs and outcomes generated by the ARC investment. The team 

received 211 responses to the survey, though only 197 of these responses contained enough information to 

analyze. Fourteen of the responses were either blank or duplicates of completed surveys. 

The information gathered in the survey was compiled into a database to analyze the reported outputs 

and outcomes associated with the projects. Responses were received from all 13 states within the 

Appalachian Region. Mississippi had the highest response rate, with 28 responses accounting for 14 

percent of the total. Ohio and Kentucky each had 26 responses. 

When evaluated from a project perspective, water and sewer systems had the largest number of 

responses by infrastructure type, 70 and 52 responses, respectively. This accounts for more than 60 

percent of the total responses received. The share of responses is slightly less than the share of water and 

sewer projects in the full ARCnet database, which is 66 percent. Access roads, which account for 

approximately 15 percent of responses, represent 14 percent of all projects in the ARCnet data. 

From 2004 to 2010, ARC contributed $206.7 million in funding to 811 projects. The survey responses 

accounted for approximately one-quarter of this funding, or $56 million. West Virginia received the 

largest amount of funding relative to both the total ARC contribution and among the survey responses, 

$36 million and $15.9 million, respectively. The largest share of the funding, based on survey responses, 

was spent on water system projects, $28 million. This is very similar to the overall database, where water 

system or water tank projects account for $105 million of the $206.7 million, or approximately 50 

percent, funded by ARC. 

1.2.1 Descriptive Statistics 

The performance measure database included information on 197 projects from all 13 Appalachian states 

across ten different infrastructure types. Approximately 77 percent of the survey responses were from 

water, sewer, or access road projects compared with 80 percent of all projects funded falling in these 

three categories. It should be noted that infrastructure type was identified by the ARC grant recipient. 

Figure ES-2 shows the number of projects funded in each ARC state, based on the survey responses. It 

also presents the types of infrastructure investments that were made. Overall, the bulk of ARC 

investment was in water and sewer infrastructure projects and the Commission sponsored many projects 

in both Kentucky and Mississippi, as presented below. 
Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

Final Report 9

Figure ES-2: Number of Projects by State and Self-Identified Infrastructure Type, 2004-2010 

Survey Responses (n=197) 

In terms of total funding, the largest share of investment was spent in West Virginia. Most of the projects 

were in transitional counties, but the largest share of funding was spent in distressed counties. 

ARC funding contributed to attracting new businesses and jobs to the communities, as well as improving 

the efficiency of business operations. Based on the survey results, water projects generated the most 

benefits in terms of jobs created or retained, businesses served, and households served. Additionally, 

projects in Kentucky seemed to have the highest benefit in terms of households and businesses improved. 

ARC funding of water system, industrial site development, sewer system and access road projects offered 

the highest return on investment, based on survey results. Of the studied cases, water systems projects 

have created or retained 6,005 jobs and 36 businesses. Industrial site development created or retained the 

second highest number of jobs with 1,045 and three businesses. Sewer systems created fewer jobs, 304, 

but the second highest number of businesses, 27. Access road projects created or retained 700 jobs and 

retained one business. 

It is worth noting that expectations or anticipated outcomes of grant recipients were often lower than 

actual outcomes. For example, survey respondents expected to serve approximately 13,000 households 

but actually served nearly 27,500 – more than double the anticipated amount. Job creation was also 

greater than anticipated, with 5,051 jobs compared to 4,181 predicted. Job retention, however, fell short of 

predictions only keeping 1,370 jobs rather than the 2,354 jobs predicted. 

ARC funds accounted for 20 percent of total project costs for those projects that responded to the survey 

and according to the grantees, ARC funding had a significant effect on the outcome of the projects. 

Without this funding, 35 percent of the projects for which responses were gathered would not have 

happened. 

0

5

10

15

20

25

30

AL GA KY MD MS NC NY OH PA SC TN VA WV

Access Road Airport Gas Line

Highway Industrial Site Development Intermodal Facility

Rail Sewer System Telecommunications

Water System
10 Final Report

Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

1.2.2 Econometric Analysis 

As mentioned previously, the data collected through the survey process provided information on 197 

projects funded between 2004 and 2010. The statistical analysis examined both the full universe of 811 

cases in the ARCnet database and these survey responses. Statistical data was observed to be consistent 

between these two datasets; minimum, maximum, and average job creation were similar. The 

econometric analysis provided insight on both the actual job creation generated by the investments and 

the job creation anticipated by grant recipients at the time their application was submitted to ARC. 

Statistical Analysis 

Regression analysis, which is a statistical process for estimating the relationships between variables, 

showed that project size, density, and economic distress all affect job impacts. For example, water/sewer 

projects located in more urbanized areas (as indicated by higher population density) tended to have 

larger job impacts. Access road projects tended to have higher impacts in less dense areas. 

The effect of economic distress was more complicated. In general, job impacts were greater in distressed 

areas, but as spending went up, the effect went down. This could reflect the fact that there is more 

opportunity for growth—or more capacity in the local economy—in distressed areas, but diminishing 

returns from large projects in those areas. 

The study suggests that approximately 144 jobs are created for every mile of access road paved. It is also 

estimated that every 10,000 linear feet of water/sewer construction is associated with 10 new jobs. 

Overall, the findings of both the descriptive and statistical analyses show that the ARC funding had a 

positive impact on both the intermediate measures – the construction of the infrastructure – and the 

long-term outcomes, such as job creation and business retention. 

1.3 Case Studies of Best Practices for Infrastructure 

In addition to the survey, the HDR Team conducted case studies of 13 projects, one in each ARC state. 

The case studies were intended to be exemplary in character in order to provide ARC and communities 

considering similar projects with insights into the ingredients of successful projects. Exemplary projects 

were identified through review of survey data and discussions with ARC program staff and state 

program managers. ARC defines exemplary projects as follows: 

 Met or exceeded outcome projections 

 Unexpected outcomes 

 Consistency with ARC objectives and strategies 

 Projects that achieved multiple objectives/strategies 

 Particularly strong local commitment (e.g., financial contribution; participation of key actors) 

The case studies focused on project planning and implementation, challenges encountered and how they 

were addressed, economic and community impacts, and “lessons learned” for communities undertaking 

similar projects. Information was obtained through review of ARC documents and interviews with key 

project participants, beneficiaries, and observers. Three of the case studies were conducted on-site and 

the remaining ten through telephone interviews. In addition to geographic diversity, the case studies 

were selected to obtain a mix of project types. Basic information on each of the case study projects is 

presented in the following table, with details provided in Section 6. 
Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

Final Report 11

Table ES-3: Case Study Overview 

Grantee Project Title Project Description

Project 

Category & 

Type

ARC 

Funding

Total 

Cost

Primary 

Outcomes

Allegany County, MD Barton Business 

Park Sewage 

Treatment Plant

Construction of a sewage 

treatment plant for 150-

acre Business Park. 

Targeted users include 

advanced manufacturing 

and defense-related 

production facilitates

Business 

Development/ 

Sewer System

$371,000 $1.94 

million

364 jobs 

created; $12 

million 

Leveraged 

Private 

Investment 

(LPI)

Dickenson County, 

VA

Big Caney Water Phase 2 of five-phase 

project to repair water 

infrastructure and extend 

water service to homes 

and businesses in 

unincorporated parts of

the county 

Community 

Development/

Water System 

$300,000 $3.1 

million

453 

residences 

and 10+ 

businesses 

served

Carroll Tomorrow, 

Carroll County (GA)

Burson Center 

for Business 

Development

Development of small 

business incubator 

providing affordable space 

for small, early-stage 

businesses for periods 

ranging from one to five 

years.

Business 

Development/ 

Business 

Incubation

$216,882 $1.9 

million

362 jobs 

created; $42 

million LPI 

Village of Canaseraga, 

NY

Canaseraga 

Wastewater 

Collection and 

Treatment 

System

Construction of new 

wastewater treatment 

system to replace failing 

residential septic systems

Community 

Development/ 

Sewer System

$150,000 $4.59 

million

215 

residences 

and 

businesses 

served; 155 

jobs 

retained

Borough of Canton, 

PA

Canton Water 

System 

Improvements

Replacement of 75-year 

old water mains and the 

construction of water 

storage tank

Community 

Development/ 

Water System

$285,000 $3.7 

million

750 

households 

and 70 

businesses 

served; 200 

jobs 

retained

City of Clinton, TN Clinton I-75 

Industrial Park 

Sewer Line

Improvements to 

wastewater collection and 

transmission system to 

support construction of 

auto parts production 

plant 

Business 

Development/ 

Sewer System

$450,000 $1.26 

million

560 jobs 

created; 

$65+ million 

LPI

Town of Dobson, NC Dobson I-77 

Infrastructure

Extension of municipal 

water and wastewater 

service to a site of planned 

commercial development 

at interstate highway exit

Business 

Development/ 

Water and 

Sewer Systems

$200,000 $2.2 

million

77 jobs 

created; 

$18.25 

million LPI

City of Muscle 

Shoals, AL

Muscle Shoals 

Industrial 

Infrastructure

Utilities and road 

improvements in a new 

industrial park to support 

construction of auto parts 

production facility 

Business 

Development/ 

Industrial Site 

Development

$500,000 $2.73 

million

800 jobs 

created; $60 

million LPI 
12 Final Report

Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

Grantee Project Title Project Description

Project 

Category & 

Type

ARC 

Funding

Total 

Cost

Primary 

Outcomes

City of Owingsville, 

KY

Owingsville/Bath 

County 

Industrial Park 

Wastewater 

Treatment Plant

Installation of wastewater 

treatment plant to meet 

the expansion needs of 

food processing plant

Business 

Development/ 

Sewer System 

$500,000 $3.04 

million

222 jobs 

created and 

retained; 

$26.2 

million LPI 

SC Budget & Control 

Board/Powdersville 

Water District

Powdersville 

Water District 

Water Storage 

Tank

Construction of water 

storage tank

Community 

Development/ 

Water System

$500,000 $1.69 

million

410 jobs 

created 

Village of Racine, OH Racine Water 

Treatment Plant 

and Storage 

Facility

Replacement of 

deteriorating water 

facilities with new water 

treatment plant and 

ground water storage tank

Community 

Development/ 

Water System

$457,000 $1.96 

million

315 

households 

served; 55 

jobs created 

and retained 

Canaan Valley 

Institute, WV

Wastewater 

Solutions for 

Small 

Communities 

Support for development 

of decentralized 

wastewater treatment 

facilities in small, 

unincorporated 

communities in southern 

WV’s coal region through 

technical assistance to 

nonprofit organizations 

Community 

Development/ 

Sewer System

$96,811 $176,865 77 

residences 

and 1 

business 

served; 

additional 

900 

residences 

projected 

City of Water Valley, 

MS

Water Valley 

Sewer & Water 

Improvements

Repair of deteriorated 

sewer lines in residential 

area and a part of 

downtown, and extension 

of water and sewer service 

to another residential area

Community 

Development/ 

Sewer and 

Water 

Systems

$152,547 $602,547 354 

residences 

and 15 

businesses 

served; 100 

jobs 

retained

1.4 Study Findings and Recommendations 

The HDR Team’s infrastructure program evaluation work examined the impact of a particular 

investment on income, employment opportunities, job creation and retention, as well as the number of 

households and businesses served by the project. The evaluation considered both quantitative and 

qualitative outcomes of the investments in order to gauge the overall performance of the projects. 

Water and sewer projects were the most frequent uses of grants, as well as the most heavily funded 

project types, based on survey results. These projects also generated the most benefit in terms of jobs 

created or retained, businesses served, and households served. 

Based on the questionnaire responses and case study findings, grant recipients seemed to underestimate 

the job and business impacts that the investment would have on their area. In many cases, the 

anticipated outcome per dollar spent was actually less than the actual outcome per dollar spent – a 

benefit to the Commission. Qualitatively, survey and case study respondents thought that the funding 

they received contributed to attracting new businesses and jobs to the community, as well as improving 

the efficiency of business operations. According to respondents, approximately 35 percent of the projects 

would not have occurred without the contributions from the ARC, indicating the importance of the 

program toward achieving the Commission’s goals.
Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

Final Report 13

Overall, the evaluation found mixed results in terms of outcomes and outputs, though there are many 

factors contributing to these results. The varied feedback and success of the projects, as identified 

through the survey results and case study evaluations, has led the team to create a series of 

recommendations for future program funding prioritization. This list of recommendations is not meant 

to be a total assessment of the program or its priorities, but rather is intended to support future success 

while achieving the goals of the Commission. 

The HDR team observed several important “lessons learned” through the questionnaire and case study 

process that may help the Commission and future grant recipients best leverage ARC funding to achieve 

maximum economic development outcomes: 

 Consider the project in the context of a broader strategic vision. 

 Think and plan ahead. 

 Right-size your project, but know that sometimes a larger initial investment may save money in 

the long run. 

 Consider how to fully leverage your investment and develop strong working relationships with 

funders. 

 Develop strong collaborative relationships and effective communications among all key local 

actors. 

 Tap into outside expertise and resources. 

 Be prepared to meet business timelines and take a hands-on approach during the construction 

phase. 

 Take a long-term view. 

The recommendations provided below are intended to both improve the quality of the data utilized in 

ARC program evaluation and also the data collection process itself. 

Require tracking of outputs and outcomes as a stipulation of receiving the grant. Several 

respondents noted that they were only required to keep records for five years and thus did not have any 

information on the project in question. Others simply did not have the information because they did not 

track the outputs or outcomes. This led to a lower than desirable response rate to the questionnaire and 

the possibility of skewed results. Given the high priority of tangible improvements to job opportunities 

and other competitiveness measures, it would be in the best interest of the Commission to require 

tracking simply for reporting and analysis purposes. This does not mean that a lack of the anticipated 

success in a particular case would be penalized, but would simply allow for better future prioritization of 

funding investments. One possible option would be to develop a letter directed to the local development 

districts highlighting the importance of collecting this information.

Consider new types of infrastructure investment to increase competitiveness. Water and sewer 

infrastructure projects accounted for the largest share of ARC spending during the evaluation period, and 

these projects are essential in providing basic services to underserved areas. As the region’s most basic 

needs are met, it is important to invest in additional infrastructure. For example, a strong fiber optic and 

broadband infrastructure could help increase Appalachia’s competiveness with other regions of the 

country.

Clarify Project Categories. There may be some disconnect in the classification of projects for funding 

purposes and the project type as viewed by the grant recipient. When comparing the project type from 

the ARCnet database to the project type as identified by the survey respondent, there were multiple 

cases of discrepancies between the two. For example, in one case, a project was classified as an access 

road in the ARCnet database, though it is actually an airport runway. Clarity and consistency on project 

type will aid in tracking performance metrics in the future.
14 Final Report

Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

Collect Additional Information Related to Funding Sources. It is not always the case that ARC funds 

are the only sources of capital for specific infrastructure projects. Better understanding of all funding 

required to bring a project to fruition, as well as the sources of these funds, would provide useful insight 

to ARC. It would be interesting to know, for example, whether greater success in terms of job retention 

and creation occurs when there are many parties involved in funding the project. A better understanding 

of the leveraging power of ARC investments, as well as the non-ARC incentives that are being provided 

to support specific projects, may be informative to ARC as they consider future investment.
Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

Final Report 15

2 INTRODUCTION 

Appalachian Regional Commission investments fund locally developed public works infrastructure 

projects to assist localities in establishing and supporting private-sector businesses. The purpose of these 

grants is to enhance economic development by attracting new industry, encouraging business expansion, 

diversifying local economies, and generating permanent, private-sector jobs. Business location decisions 

are heavily influenced by a number of factors in a geographic area. In addition to a skilled labor force and 

access to suppliers, adequate public facilities in the form of roads, water/sewer, telecommunications and 

utilities, and appropriately developable land in the form of industrial and commercial parks, are key 

factors to attract businesses to an area. 

Performance management is a policy-directed, data-driven, performance-based business practice that 

links organizational goals and objectives to resources and results. The outcomes of performance-based 

management include a more efficient distribution of limited resources and a focus on accountability of 

decision-making. Performance management has been in use for over a decade in various public works 

fields, and over that time it has evolved from a focus on identifying measures and tracking performance to 

using performance measures in decision-making. This increased focus on managing with performance 

measures has enhanced the role of performance evaluation to address two important questions: 1) How 

are we doing? And 2) How can we do better? 

This study aims to evaluate ARC funded infrastructure and public works projects to assess whether or 

not the projects have achieved their stated goals as well as these projects’ contribution to the strategic 

objectives of the Commission. This study covers projects funded from 2004 to 2010 throughout the 

Appalachian Region, focusing on economic impacts including job creation and retention, business 

service, economic diversification and income growth. 

The first section of this report provides detail on the various methods employed by the ARC and other 

organizations to evaluate program performance. This information is based on a review of ARC and other 

studies that focus on program evaluation reports, and guidance and related program evaluation studies 

from other government agencies. The second section of this report provides information on the 

development of the performance measure database as well as an analysis of the findings. The final section 

of the report details case studies of funded projects. 
16 Final Report

Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

3 DEFINITION OF EVALUATION CRITERIA AND REVIEW OF PAST 

PROGRAM EVALUATIONS 

Traditional evaluation methods and other approaches are described in the report, and studies that have 

relied on these methods to measure program performance are highlighted throughout. The report also 

compares ARC’s existing approaches to measuring project performance with the methods used by other 

organizations or recommended by guidance documents that have been reviewed. The major evaluation 

approaches discussed in the report include: 

 Surveys 

 Observation or site visits 

 Analysis of experimental data 

 Before and after comparisons 

 Theory-based evaluation and modeling 

 Case studies 

 Application of other studies 

 Use of expert opinion 

 Statistical analysis and causal models 

 Input-output modeling 

 Benefit-cost analysis 

 Sensitivity analysis and risk analysis 

A complete annotated bibliography that provides the name of the study, primary purpose, overall 

conclusions and other information is also provided in this report. Studies from both the Appalachian 

Regional Commission and other Government agencies were reviewed for depth of analysis of 

performance measures. The studies that were reviewed include: 

 Program Evaluation of the Appalachian Regional Commission's Infrastructure and Public Works 

Projects (2007 and 2000) 

 Assessing Local Economic Development Opportunities with ARC-LEAP (Appalachian Regional 

Commission Local Economic Assessment Package) (2004) 

 Understanding Water and Sewer Reporting Issues: A Study of the Economic Development 

Infrastructure and Resources Survey Methodology (1999) 

 Drinking Water and Wastewater Infrastructure in Appalachia, An Analysis of Capital Funding 

and Funding Gaps, UNC Environmental Finance Center (July 2005) 

 Government Performance and Results Act of 1993, passed by the 103rd U.S. Congress 

 Office of Management and Budget (OMB) Circular A-11, Preparation, Submission, and Execution 

of the Budget applicable to federal agencies (August 2011) 

 Moving Appalachia Forward: Appalachian Regional Commission Strategic Plan 2011-2016 

 The Magenta Book: Guidance for Evaluation, United Kingdom Cross-Government Evaluation 

Group (April 2011) 

 Deputy Comptroller General, Treasury Board of Canada’s Program Evaluation Methods: 

Measurement and Attribution of Program Results, Third Edition 
Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

Final Report 17

 United States General Accounting Office’s (GAO's) Designing Evaluations report (March 1991) 

 Economic Development Administration’s (EDA’s) Public Works Program Performance 

Evaluation (May 1997) 

The remainder of this section covers an overview description of different methods to conduct program 

evaluation, keeping in mind that it is common to apply two or more techniques to strengthen evaluation 

findings and examine causal relationships between a program and its desired outcomes. Section 3.2 

provides a more detailed literature review of four studies specifically related to the ARC, while Section 

3.3 details program evaluation guidance and reports from other government agencies in the U.S., Canada, 

and Europe. 

3.1 Program Evaluation Methods 

As discussed in the studies reviewed, there are a number of different approaches to measuring the 

performance of projects. A description of the method, its pros and cons, and its applicability to ARC 

public infrastructure project evaluation is provided. Where relevant, specific guidance or studies that 

discuss or utilize the approach are referenced. 

3.1.1 Surveys 

A very common approach to assessing project performance by private and public entities is the 

implementation of a survey. There are three major survey types discussed in the literature: cross 

sectional, panel, and criteria referenced. 

1. Cross Sectional: Measurements are made for multiple units at a single point in time. 

2. Panel: Similar to a cross-sectional survey but has the added feature that the information is 

acquired from a given sample unit at two or more points in time. 

3. Criteria Referenced: Enables evaluators to answer normative questions, which compare actual 

performance to an external standard of performance. 

Surveys are versatile and, when properly done, produce reliable and valid information. They require 

expertise in their design, conduct and interpretation, however, and the results can be easily misused. The 

guidance offered in several of the studies reviewed, and in particular the Treasury Board of Canada’s 

“Evaluation Methods: Measurement and Attribution of Program Results,” recommends several basic 

steps before implementing a survey. 

1. Other evaluation options: Evaluator should check to see if required data are available elsewhere 

and determine whether the survey is the most cost effective and efficient way to collect the data. 

2. Good front end planning: 

a. Identify what specific information will address a given evaluation issue; 

b. Determine sources of information (e.g., types of respondents); 

c. Choose methods to be used for sampling sources (e.g., random sampling) and collecting 

information (e.g., structured interviews and self-administered questionnaires); 

d. Determine timing and frequency of information collection; 

e. Decide on basis for comparing outcomes with and without a program; and 

f. Develop the analysis plan. 

3. Survey design: Avoid “nice to know” data collection and keep survey succinct and clear. 
18 Final Report

Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

4. Survey pretest: Pretest on a representative sample of the survey population. Provides information 

on clarity of questions, response rate, time and length to complete, and the appropriateness of 

survey method. It also allows for adjustments to the survey approach as necessary. 

Applicability to ARC Program Evaluation 

ARC utilizes cross-sectional surveys to collect data for program evaluation in a number of different areas 

and then uses this information to estimate the full impacts of a program. Previous ARC program 

evaluations have employed this approach, which is consistent with the guidance. The ARCnet database 

of projects includes a significant amount of data collected through the use of surveys (most recently 

conducted online). Additionally, the Local Development Districts (LDDs) conduct surveys every two 

years in an effort to obtain information related to water and sewer use in the Appalachian region. 

Historically, this has resulted in some information being collected that is useful in assessing the need for 

facilities, but some LDDs provide outdated or incomplete information because they do not have data 

available to them that would facilitate a more accurate completion of the survey. While surveys are 

applicable to ARC evaluations, the LDD water/sewer experience highlights the importance of pre-testing 

the survey instrument. 

3.1.2 Observation or Site Visits 

This program evaluation technique involves on-site visits by an evaluator to locations where a program is 

operating to observe activities and to take notes. Program participants and leaders may or may not be 

aware that the program is being evaluated. When considering this evaluation approach, it is important to 

be aware that program staff may alter their behavior if they know they are being observed. The following 

recommendations were made in the reviewed guidance: 

a. U.S. General Accounting Office (GAO)1 encourages an evaluator to consider social/cultural 

mores of the area where the evaluation is to be conducted. If the evaluation is in a rural area, for 

example, the evaluator must be sensitive to the difference in culture between a rural and urban 

environment. 

b. Consider whether the population being studied is stable. Are residents transient or do people 

live in the study area for decades? A neighborhood where residence is transient may require a 

different implementation strategy than one in which people do not move. 

c. Determine whether an on-site visit to evaluate a program is justifiable on a cost basis. On-site 

visits can be very useful but also very costly. 

d. Be aware that an evaluator may input some subjectivity in his or her assessment of a program. 

e. Keep in perspective that observation offers only anecdotal evidence unless it is combined with a 

planned program of data collection. By itself, it is not grounds for generalization about a 

program. 

While this type of evaluation approach may be costly and may be subject to the perception of the 

evaluator, it does offer the opportunity to collect important qualitative information that may not be 

collected through other evaluation methods. The guidance offered by the Treasury Board of Canada 

points out that this approach may allow an evaluator to see things that may escape staff members or 

identify issues they are reluctant to raise in an interview. This is an important benefit of this evaluation 

approach. 

 

1

 United States General Accounting Office, Program Evaluation and Methodology Division, Designing Evaluations, 

March 1991. 
Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

Final Report 19

Applicability to ARC Program Evaluation 

Site visits have been a part of the previous infrastructure program evaluations conducted by ARC. In the 

past, these visits have been one of several approaches used in these evaluations. As another example, the 

EDA’s “Public Works Program Performance Evaluation,” conducted in May 1997, involved site visits. 

Specifically, EDA invited grantees to seminars at 13 locations across the country. At these seminars, they 

were instructed on technicalities of the team’s information requests and the specific information that 

would be required as part of the program evaluation. One-day seminars were held in Atlanta, Austin, 

Chicago, Cincinnati, Denver, Hartford, Little Rock, Los Angeles, Myrtle Beach, Orlando, Philadelphia, 

San Francisco, and St. Louis. Research team members also physically visited 25 percent of the grantee 

locations for site inspections. Information from these visits, combined with survey data, was used in the 

evaluation performance metrics. Like ARC, EDA’s performance measures include permanent jobs created 

and private and public funds leveraged. Information related to the diversification of the local economy 

and tax base added to the community was also assembled by EDA program evaluators. 

3.1.3 Analysis of Experimental Data 

Based on the guidance, program evaluation through experimental designs is the most rigorous approach 

available for establishing a causal relationship between programs and their outcomes. Unfortunately, this 

approach is impossible to implement for many programs, particularly when the program has been 

running for some time. 

Experimental designs are characterized by a random assignment of participants to a “treatment” or 

separate “control” group. This process ensures that the groups being evaluated are equivalent (other than 

treatment), and the randomized assignment of participants to different groups prior to a program being 

implemented helps create an even playing field from which to draw comparisons, once the program is 

operational. Other related designs include: 

1. Quasi-experimental design 

a. Do not use randomization to create treatment and control groups. The treatment group 

is usually already given and one or more comparison groups are selected to match the 

treatment group as closely as possible. 

b. Implementation takes creativity and skill to design. 

c. Can provide highly accurate findings. 

2. Implicit design 

a. Typical evaluation design, but least rigorous. 

b. Post-program evaluation with no control group – make a measurement after exposure to 

the program and then make assumptions about conditions prior to the program. 

c. Flexible, versatile and practical to implement. 

d. Offers little objective evidence of the results caused by the program. 

Applicability to ARC Program Evaluation 

ARC empirically analyzed the extent to which it has stimulated the Appalachian economy in a quasiexperimental study conducted by Andrew Isserman & Terance Rephann. The study measures the effects 

of ARC programs on 391 counties within the region using this evaluation approach.2

 This approach could 

 

2

“The Economic Effects of the Appalachian Regional Commission: An Empirical Assessment of 26 Years of Regional 

Development Planning,” prepared by Andrew Isserman and Terance Rephann, Journal of the American Planning 

Association, Volume 61, Issue 3, 1995. 
20 Final Report

Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

be useful to ARC in its infrastructure program evaluations as well, but it would be challenging because of 

the number and range of projects being evaluated, the cost to conduct this type of evaluation, and the 

need to determine a statistically valid “control” group for comparison purposes. 

3.1.4 Before and After Comparisons 

Before and after assessments3 compare outcomes for the units of study before the units were exposed to a 

program to outcomes measured after they began to participate in it. Unlike experiments, there is no 

separate comparison group, which is a weakness of this approach. This can be mitigated somewhat by 

evaluating the program at many different points in time. With a sufficient number of evaluation points, 

an interrupted time series analysis can be applied to this design to help draw causal inferences. A rule-ofthumb is a minimum of 50 before/after observations, based on the evaluation guidance reviewed. Data 

consistency is also an important element to before and after comparisons. The time series developed 

needs to be free of definitional and measurement changes, because these can be mistaken for program 

effects. 

Applicability to ARC Program Evaluation 

This comparison approach has been utilized by ARC in its prior program evaluations, including past 

studies to estimate the impacts of ADHS highway investments. Specifically, ARC developed profiles of 

counties prior to infrastructure program implementation and compared this baseline to the counties after 

the program was put in place. The study was strengthened by comparing the results to counties with no 

investment. 

3.1.5 Theory Based Evaluation and Modeling 

According to The HM Treasury’s “The Magenta Book, Guidance for Evaluation,” theory-based evaluation 

approaches can offer an overarching conceptual framework within which specific evaluation studies can 

be designed. This approach is complementary, and not an alternative, to other evaluation approaches. 

The use of a theory-based evaluation can provide new data and evidence that can be incorporated into 

the evaluation framework as appropriate. One practical way to do this is through a simulation model. 

Simulation models can be used to combine existing and new evidence to answer evaluation questions. 

These models can be subject to some uncertainty, however, due to the need to make assumptions about 

how different pieces of evidence are related. Simulation models do allow an evaluator to estimate the 

incremental effects of a program in complex and uncertain situations, which is not the case with some 

other evaluation approaches. For very complex programs, evaluation may only be possible through 

theory-based evaluation or simulation modeling. 

Generally, a simulation has three main components based on evaluation guidance: 

1. Input data 

2. Mathematical model 

a. Stochastic – incorporates a random data generator 

b. Deterministic – does not incorporate a random data generator 

3. Output data 

 

3

 Implicit Designs and Before/After Comparisons are similar evaluation approaches, but the former makes 

assumptions about conditions before the program was put in place, and the latter uses observations both before and 

after the program was implemented. 
Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

Final Report 21

Simulation resembles other statistical techniques, such as regression analysis, and these techniques may 

actually be used to build a simulation model. Once the model is constructed, however, the inputs 

provided are actually used by the model to generate outputs. The output generated by the simulation 

model can be checked against actual real world outcomes. 

The guidance suggests that some form of simulation modeling is likely to play a role in a large proportion 

of program evaluations. Where outcomes are expected to occur over a number of years, some simulation 

of these effects might be necessary to ensure that evaluation evidence is obtained in a timely fashion. The 

primary benefit of simulation modeling is that an evaluator can estimate incremental effects of a program 

in a complex and uncertain situation. The primary con to this approach is that a sophisticated 

understanding of the dynamics of the program is required, as is some skill in building quantitative 

models (or the cost of obtaining models, such as the Regional Economic Models, Inc. [REMI] dynamic, 

time-series based economic forecasting and simulation model). 

Applicability to ARC Program Evaluation 

ARC has utilized simulation models when it conducts its economic impact assessments of programs, 

including IMPLAN, ARC-LEAP, REMI, and TREDIS. These models are primarily used to estimate 

indirect and induced effects based on direct effects elicited through surveys. EDA also included the use of 

simulation models in some of its program evaluations. For example, EDA conducted input-output 

analysis to estimate indirect employment and investment impacts (multiplier effects) of EDA 

investments. They also conducted regression analysis to estimate the impact of the EDA investments on 

countywide employment and wages. 

3.1.6 Case Studies 

Case studies usually involve a variety of data collection methods, including surveys and interviews. 

While it is tempting to do more case studies than fewer, guidance suggests that the study of a critical 

case may be more defensible than the case study of a representative sample. Unfortunately, using one 

case study in an evaluation does not enable the evaluator to make generalizations about the overall 

program. Case studies: 

1. Enable an in-depth analysis that would not be possible with more general approaches; 

2. Are typically expensive and time consuming; 

3. Usually lack a statistical basis from which to generalize conclusions – difficult to do enough case 

studies to provide an adequate sample from which to draw conclusions; and 

4. Provide broad insights on the program and the real world context of impacts. 

The guidance recommends that case studies be carried out before, or in tandem with, other data 

collection efforts. According to studies offering guidance on program evaluation, there are generally three 

types of case studies: 

1. Single case study 

a. Information is acquired about a single individual project, process, or entity. 

b. Qualitative information that describes events and conditions from various points of view 

can be collected. 

c. Selection of a single case and the data collection may be challenging. 

d. Analyzing and reporting qualitative data can be difficult, and the evaluation design must 

have explicit plans for these tasks. There are several different types of single case studies 

based on the literature: 

 Illustrative – Describes event or condition. 
22 Final Report

Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

 Exploratory – May serve as a precursor to a larger evaluation or provide 

preliminary information. 

 Critical instance – Investigates one problem or event. 

2. Multiple case studies 

a. Likely to produce stronger conclusions than a single case study. 

b. May be appropriate for evaluating either program operations or results. 

c. Selecting variety among cases is important to avoid bias, as is uniform information 

collection. 

3. Criteria-referenced 

a. Can be adapted to answer normative questions about how well program operations or 

outcomes meet certain performance criteria. 

b. Carefully choosing the sample of cases to evaluate is critical. 

Applicability to ARC Program Evaluation 

A case study can help develop a clear, well-reasoned and comprehensive understanding of the situation, 

project or people affected by a program. Case studies often are used to supplement or complement a more 

quantitative evaluation approach, helping evaluators better understand the causal logic. 

The ARC’s “Drinking Water and Wastewater Infrastructure in Appalachia: An Analysis of Capital 

Funding and Funding Gaps” utilized case studies, as well as a survey and information obtained from 

other reports, in its evaluation. Specifically, ARC examined particular towns, regions, and states to 

ascertain specific needs for drinking and wastewater and compared these needs to the available data and 

funds. For the infrastructure and public works program evaluation, case studies of 13 projects will 

highlight best practices and provide context for the data analysis components of the evaluation. 

3.1.7 Application of Other Studies 

A literature review of other studies, or a file review of data and information, enables an evaluator to make 

the best use of previous work in the field under investigation. Application of other studies can help: a) to 

determine the best analytical methods and performance measures to use; and b) use assumptions and 

data findings to help compare and estimate program impacts. In addition, the evaluator may learn from 

the experiences, findings, and mistakes of other entities that have conducted program evaluations. Past 

research can suggest hypotheses to be tested or evaluation issues that should be examined in the current 

study, and specific methodological difficulties can be identified and mitigated. In some cases, evaluation 

questions may be directly answered on the basis of past work or redundant data collection. 

Another advantage to reviewing other studies is that sources of secondary data, such as previous surveys 

or organizational records, may be uncovered, which may lessen the need to collect primary data. While 

this evaluation approach is economical, it is not always appropriate for a particular program evaluation. 

Another limitation to using other studies to evaluate a particular program is that it can sometimes be 

difficult to determine the accuracy of the data gathered through a previous study. 

Many project evaluations utilize a review of other studies as a first step in a larger evaluation of a 

program. ARC has employed this approach in its work, and other organizations have incorporated the 

same sort of preliminary review of studies prior to embarking upon a larger program evaluation. 
Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

Final Report 23

Applicability to ARC Program Evaluation 

Reviews of other studies are primarily applicable in terms of determining methodology and performance 

measures for the infrastructure and public works evaluation. The team cross-checked results from the 

new program evaluation with the results from previous efforts that attempted to measure the same kinds 

of ARC infrastructure and public works programs. 

3.1.8 Other Potentially Applicable Evaluation Approaches 

Other applicable approaches to program evaluation were discussed in the studies reviewed by the team. 

These methods are generally sub-sets of or related to the over-arching methods described above and are 

described below. 

Use of Expert Opinion 

Collecting data from experts is a method best suited to supplementing other measures of program 

outcomes. An advantage to this approach is that data can be collected and summarized systematically. In 

addition, the opinions of experts may vary and experts may not be equally knowledgeable about a 

program or issue. While this approach should not be used as a sole source of data for an evaluation, it can 

help fill in the gaps or offer detail on a particular program. For ARC program evaluations, this approach 

could be part of a case study or survey effort. 

Statistical Analysis and Causal Models 

Statistical analysis and development of a causal model requires expertise, and not all programs can be 

analyzed using these approaches. A causal model is based on an equation that describes the marginal 

impact of a set of selected independent variables on a dependent variable. This approach focuses on 

variables to be included in the model (endogenous and exogenous) and their postulated relationships. 

Regression analysis is a technique employed to measure the impact of one variable or variables on 

another, the causal relationship. With respect to program evaluation, the program is only one of several 

independent variables that are expected to affect the dependent variable. Determining when it is 

appropriate and how to apply statistical analysis to measure causality is the focus of “Impact Analysis for 

Program Evaluation.”

4 The author also discusses the use of experiments and quasi-experiments as part of 

an overall evaluation that includes statistical analysis. 

Input-Output (I-O) Modeling 

A variation of theory-based evaluation and modeling, this type of approach consists of using a model 

designed to depict the mutual interdependence among different parts of an economy. It describes how 

one industry uses the outputs of other industries as inputs, and how its own outputs are used by other -

industries as inputs. ARC uses I-O models to measure the total economic impacts (including multiplier 

effects on indirect and induced demand) of direct project impacts, which are often obtained through 

interviews with grantees. While useful for better understanding the impact of a program, these types of 

models and their results can be misused in evaluations. For example, offsetting negative effects generated 

by taxes and borrowing necessary to support a program need to be factored into an I-O model. 

Otherwise, the impacts of the program on the economy may be overstated. Other challenges include 

 

4

 Impact Analysis for Program Evaluation, Second Edition, by Lawrence B. Mohr, 1995. 
24 Final Report

Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

measuring the direct impacts of a program in terms of factors such as additional spending or

employment, or controlling for potential displacement of activity from other sectors of the economy (e.g., 

new retail activity that displaces or impacts existing retail establishments). 

The Economic Development Administration and ARC have utilized I-O models in their program 

evaluations to estimate total job impacts, business sales, and other economic variables. 

Benefit-Cost Analysis (BCA) 

Benefit-cost analysis looks at a program’s net worth and assesses and compares program alternatives. The 

issue of attribution and incrementality of effects over a “base case” must be addressed prior to doing this 

type of analysis. Benefit-cost analysis is more a comprehensive approach to assessing the effects of a 

program or project than an approach to measuring its performance. It uses the results of interviews, 

experiments, case studies, and so forth, as inputs and produces summary measures of monetary value. 

Most typically, benefit-cost analysis is used to assess the future impacts of proposed investments, 

although it can be used for program evaluation if benefits can be measured and isolated. Regardless of the 

timing of this analysis, the guidance recommends that a sensitivity analysis of key assumptions be 

conducted if this analytical approach is employed. Details related to sensitivity analyses are provided 

below. 

Sensitivity Analysis and Risk Analysis 

Analytical work is often directed toward a single “expected outcome,” supplemented with alternative 

scenarios. The limitation of this approach is clear – while it may provide the single best statistical 

estimate, it offers no information about the range of other possible outcomes and their associated 

probabilities. 

There are two ways to mitigate this issue. One common approach to providing added perspective on 

reality is “sensitivity analysis.” Key assumptions are varied one at a time in order to assess their relative 

impact on the expected outcome. The problem here is that the assumptions are often varied by arbitrary 

amounts. A more serious concern with this approach is that, in the real world, assumptions do not veer 

from actual outcomes one at a time. It is the impact of simultaneous differences between assumptions 

and actual outcomes that is needed to provide a realistic perspective on the riskiness of a forecast. 

Risk Analysis avoids the problems outlined above by measuring the probability or “odds” that an 

outcome will actually materialize. This is accomplished by attaching ranges (probability distributions) 

to the forecasts of each input variable. The approach allows all inputs to be varied simultaneously within 

their distributions, thus avoiding the problems inherent in conventional sensitivity analysis. The 

approach also recognizes interrelationships between variables and their associated probability 

distributions. Sensitivity and risk analyses can be useful as part of an overall program evaluation.5

3.2 Comparison to Current Program Evaluation Approaches Used by ARC 

Generally, the ARC has evaluated its infrastructure investments using a combination of approaches 

applied to a sample of ARC projects. The sample is selected to be representative of the mix of project 

types and area characteristics within the region. The first step in the evaluation has been a review of ARC 

records and classification of the programs into a database. A phone survey of several local or regional 

 

5

 As an example, see the Federal Highway Administration (FHWA) Economic Analysis Primer, August 2003. 
Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

Final Report 25

development staff, local government and civic leadership, and/or private sector representatives was then 

conducted. Information collected through the survey and other secondary data sources were used to 

construct project profiles. Site visits were then made to validate project results and to develop more 

detailed case studies. Site selection has historically been based on project type, geography, and area 

demographics. 

For past studies, the projects were evaluated by comparing the anticipated and actual project outcomes 

in terms of key performance measures. In the initial project applications for ARC funding, local 

applicants are required to estimate the number of jobs to be created or retained, the number of businesses 

to be served or retained, and the number of new or existing households to be served directly by the 

project. This information is considered the “anticipated” outcomes. For the ARC program evaluation of 

infrastructure conducted in both 2000 and 2007, surveys conducted by phone and site visits were 

performed to gather information and validate project results. In 2000, eight site visits were made to 

validate project results and develop case studies. Information related to several different categories was 

collected and examined during these visits and interviews: 

 Project area distress based on the poverty rate, three-year average unemployment rate, and per 

capita market income as compared to the national averages 

 Project data and budget information (cost, sources of funding) 

 Project fiscal and economic impact analyses 

 Economic trend and vitality analyses of primary impact counties to understand the economic 

context of impacts 

 Interviewee information 

 Qualitative project objectives and outcomes 

 Impact comments and discussion 

ARC interviews and site visits helped to determine the total number of jobs created and retained 

whenever possible. Other information collected included estimates of retained wages and new income. 

ARC supplemented this primary data with average wage data from the U.S. Bureau of Labor Statistics. 

The interviews also helped determine whether ARC’s investments broadened the type of jobs available 

and the desirability of available jobs, as well as the importance of ARC investment and the role that the 

projects played. 

For the ARC evaluations, estimated economic development impacts were compared with the actual 

results for each project in which an investment was made. This before and after approach enabled a 

variety of ratios to be constructed, including an investment ratio that compares ARC and total public 

funding with private investment results. Other ratios constructed in these studies included taxes per 

dollar of public investment, total public investment per job, and the private sector leverage that resulted 

from the program investment by ARC. Private sector leverage was based on the total private investment 

that was spurred (and reported) as a result of the ARC program investment. Economic impacts were also 

estimated using IMPLAN and the ARC-LEAP model. 

The evaluation guidance that was reviewed suggests that utilizing multiple evaluation approaches is 

useful in that each approach may help fill in the gaps that may occur when using only a single approach. 

ARC’s decision to utilize surveys, on-site visits and case studies, as well as before and after comparisons 

aided by an input-output model, is consistent with the program evaluation guidance reviewed by the 

team and similar to how some other agencies assess their program’s impacts.

Ideally, ARC would utilize an experimental design approach to measure its program outcomes. The 

guidance indicates that this evaluation approach is the most rigorous. As is the case with many program 

investments, however, it would be very difficult and very costly to implement this type of approach. The 
26 Final Report

Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

ARC funds numerous programs across the Appalachian Region and establishing a control and treatment 

group for each program would be time- and cost-prohibitive. 

3.3 Summary and Conclusions 

The approaches and methods described in this section are summarized in Table 1 below. The table 

includes a brief description of the methods, identifies the specific questions being addressed, and assesses 

the extent to which the methods have been used in the evaluation of ARC programs. 

Table 1: Summary of Approaches and Methods 

Approaches 

& Methods Brief Description Principal Strengths & Weaknesses

Use in ARC 

Evaluations

Surveys Collecting data or information 

directly from those affected by a 

program or action, typically 

through a mail or online 

questionnaire, or through phone 

interviews. May be cross-sectional, 

panel, or criteria referenced.

Allows gathering large amounts of 

performance data at a relatively low 

cost. Accuracy of data depends on 

survey and questionnaire design, as 

well as quality of information 

available to interviewees. Can only 

be used to evaluate the direct effects 

of a program or action.

Yes, extensive

Observations

or Site Visits

Visits by an evaluator to locations 

where a program is operating, to 

observe activities, assess changes, 

and take notes.

Allow gathering detailed, 

qualitative information on the 

effects of a program at a specific 

location, or set of locations. Is often

costly to implement and findings 

may be biased by evaluators’ 

perception.

Yes

Analysis of 

Experimental Data

Comparison of performance 

between a treatment group and a 

control group. Design may be 

experimental (with random 

assignment of participants), quasiexperimental (without 

randomization), or implicit 

(without control group).

Considered most rigorous approach 

to establishing cause-and-effect 

relationships between programs 

and observed changes. But is 

generally difficult and costly to 

implement, and may not be suited 

to all programs and actions.

Limited

Before & After 

Comparisons

Comparison of performance before 

and after implementation of a 

program, for a single group of units 

or participants.

Effective in measuring changes over 

time, but causal relationships 

between programs and outcomes 

may not be formally established.

Yes

Theory Based 

Evaluation 

& Modeling

Use of a conceptual framework

and/or model to develop 

assumptions about, or simulate the 

impacts of a program.

Must be used in conjunction with 

other approaches and methods. 

Particularly effective in estimating 

the impacts of complex programs. If 

not properly documented, 

simulation models may be viewed 

as “black boxes,” limiting the 

applicability of their output.

Yes, in the 

estimation of 

indirect and 

induced 

economic 

impacts
Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

Final Report 27

Approaches 

& Methods Brief Description Principal Strengths & Weaknesses

Use in ARC 

Evaluations

Case Studies Detailed analysis of a critical case 

or a limited set of cases, involving a 

variety of information gathering 

techniques (including interviews, 

site visits, etc.).

Provide in-depth information and 

analysis of effects, but can only be 

applied to a limited number of units 

or locations. Results can rarely be 

generalized. Particularly useful in 

validating or augmenting the 

findings of a broader data collection 

effort (e.g., survey). 

Yes

Application of 

Findings from Other 

Studies

Use of methods, data sources, 

assumptions and/ or estimates 

from existing studies.

Useful in methodology development 

stages. “Transfer” of impact 

estimates and other quantitative or 

qualitative findings are generally 

low-cost, but not considered bestpractice.

Only to inform 

selection of 

methods and 

performance 

measures

Use of Expert 

Opinion

Eliciting information directly from 

Subject Matter Experts, to 

supplement other data sources and 

collection efforts.

Particularly useful when 

observation and traditional data 

collection techniques fail to 

produce adequate results. May also 

be used for review and validation. 

Requires use of formal elicitation 

techniques to limit potential for 

bias.

No

Statistical Analysis 

and Use of Causal 

Models

Use of statistical methods, 

including regression analysis, to 

estimate “causal” relationships 

between program attributes and 

performance. Often used for the 

analysis of experimental or quasiexperimental data.

Allows formal hypothesis testing, 

but robustness of results depends 

on quality of data and survey design 

(see Analysis of Experimental 

Data). Is not applicable to all types 

of programs.

Limited

Input-Output 

Modeling

Use of accounting tables tracing 

linkages of inter-industry 

purchases and sales to estimate the 

indirect and induced economic 

impacts of a program or action.

A sub-set of Theory Based 

Evaluation & Modeling. Economic 

impacts estimated with off-theshelf I/O models are typically gross 

impacts, ignoring many further 

possible interactions and responses.

Yes, in the 

estimation of 

total economic 

impacts

Benefit-Cost 

Analysis

Estimation of the costs and 

benefits of a program, defined in 

terms of well-being (welfare) with 

a view to providing an overall 

assessment of value-for-money. Is 

not a performance-measurement 

approach per se.

Primarily used for program 

assessment, ex-ante; but may be 

used ex-post to estimate the return 

on investment of existing programs 

(based on performance measures 

developed through other means).

No

Sensitivity Analysis 

or Risk Analysis

General approaches to 

incorporating uncertainty into 

performance measurement and 

program evaluations. Can be 

applied in conjunction with any of 

the above quantitative methods.

Assists decision-makers in 

assessing the degree of confidence 

they should place in the outcomes 

of an evaluation.

No
28 Final Report

Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

Table 2 evaluates each approach or method against a set of criteria, defined as follows: 

 Ease of use in ARC evaluations: whether, once developed, the method and associated tools can be 

used by local, regional and ARC staff with a range of technical expertise; 

 Ease of interpretation: whether the resulting performance measures can be easily understood and 

interpreted; 

 Flexibility: whether the method can be applied to a wide range of conditions and projects; 

 Data availability: whether the data required to apply the method are readily available and/or 

accessible at a reasonable cost; 

 Ability to “attribute” impacts: whether the contribution of the program to observed outcomes can 

be formally tested; 

 Estimation of counterfactual: whether the method requires use of a control group and/or estimation 

of a counterfactual; 

 Transparency: whether all steps of the method can be described clearly, including where applicable 

the “logic” used to derive outcomes from output measures; and

 Acceptance and credibility: whether the method is widely accepted and/or conforms to industry 

standards. 

The ratings shown in the table were developed, somewhat subjectively, by the Project Team. The criteria 

are rated on a scale of one to four (stars), four representing the easiest or most flexible method, and one 

the least.

Table 2: Rating of Approaches and Methods against Criteria 

Criteria

Ease of Use 

for ARC

Ease of 

Interpretation

Flexibility

Data 

Availability

Estimation of 

Counterfactual

Transparency

Acceptance & 

Credibility

Approaches & Methods

Surveys     No  

Observations

or Site Visits     No  

Analysis of Experimental 

Data     Yes  

Before & After 

Comparisons     No  

Theory Based Evaluation 

& Modeling     Sometimes  

Case Studies    

Generally 

Not  

Application of Findings 

from Other Studies     No  

Use of Expert Opinion    

Not 

Explicitly  

Statistical Analysis and 

Use of Causal Models     Sometimes  
Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

Final Report 29

Criteria

Ease of Use 

for ARC

Ease of 

Interpretation

Flexibility

Data 

Availability

Estimation of 

Counterfactual

Transparency

Acceptance & 

Credibility

Approaches & Methods

Input-Output Modeling     No  

Benefit-Cost Analysis    

Yes, 

Needed  

Sensitivity Analysis 

or Risk Analysis     N/A  

Lessons learned from the literature review and selected recommendations for subsequent tasks and 

future ARC evaluation efforts are summarized in Table 3 below. In drafting these recommendations, the 

Project Team recognizes ARC’s time and resource constraints, and considers the specific focus of its 

ongoing evaluation and research efforts. 

Table 3: Lessons Learned from Tasks 1 & 2 and Recommendations 

Approach 

or Method Lessons Learned Selected Recommendations Main Sources

Surveys  Response rates may be lower 

than expected. 

 The information collected 

through surveys may be 

incomplete, imprecise or 

outdated.

 Survey respondents may have 

difficulties thinking in terms 

of “counterfactual” and 

assessing the true 

contribution of a program.

 Avoid “nice to know” data 

collection, and keep survey succinct 

and clear.

 Pretest the survey instrument on a 

representative sample of the 

population. Provides information on 

clarity of questions, response rate, 

time and length to complete. 

 Review the appropriateness of the 

survey method for collecting the 

information needed for an 

evaluation.

 Focus on direct effects and provide 

guidance with respect to assessing 

attribution and cause-and-effect.

 Offer interviewees the opportunity 

to rate the strength of their claims 

of new jobs and other effects.

 Treasury 

Board 

of Canada

 UK Cross 

Government 

Evaluation 

Group

Observation 

or Site Visits

 Approach may allow 

evaluators to see things that 

may escape staff members or 

identify issues they are 

reluctant to raise in an 

interview.

 Findings may be biased by 

evaluators’ subjectivity and/or 

changes in the behavior of 

those being observed.

 To the extent possible, 

systematically include site visits in 

program evaluations.

 Use observation results to augment 

and validate survey findings, or (in 

survey design stage) to assist with 

questionnaire development.

 Instruct evaluators to conduct 

extensive background research on 

site or unit prior to visit.

 Treasury 

Board 

of Canada
30 Final Report

Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

Approach 

or Method Lessons Learned Selected Recommendations Main Sources

Analysis of 

Experimental 

Data

 Use of experimental or quasiexperimental data is 

considered the most rigorous 

approach to assessing the 

effects of a program.

 Consider supplementing ongoing 

evaluation efforts with occasional 

statistical analyses, in the spirit of 

Isserman & Rephann (1995).

 Explore possibility of hypotheses 

testing using information from the 

ARCnet database.

 Treasury 

Board 

of Canada

 Isserman & 

Rephann 

(1995)

Before & 

After 

Comparisons

 Approach is commonly used 

in program evaluation, but 

must be carefully designed 

and/or supplemented with 

other methods to establish 

causality.

 Comparisons across many 

different time periods, as 

recommended in the 

literature, may be costly or 

impractical – if the program 

was recently implemented.

 Results may be tainted by 

definitional and measurement 

changes.

 To the extent possible, before and 

after comparisons should be 

supplemented with cross-sectional 

analyses and comparisons to a 

control group (e.g., counties 

without investment).

 Through desktop research and 

interviews, evaluators should 

identify all potential factors, other 

than the program, that may have 

impacted the performance of the 

unit.

 ARC

 Treasury 

Board 

of Canada

Theory Based 

Evaluation 

& Modeling

 Generally used as a 

complement to other methods.

 Often the only way to 

estimate certain effects (e.g., 

indirect and induced 

economic impacts).

 Results may be difficult to 

interpret or may be misused 

(e.g., estimation of gross vs. 

net job creation).

 Provide detailed guidance for the 

interpretation of model output.

 Highlight the limitations of off-theshelf I/O models with respect to 

estimating the net effects of 

programs.

 In reporting evaluation results, 

clearly separate what is measured 

from what is estimated with the 

model.

 ARC

 Treasury 

Board 

of Canada

 UK Cross 

Government 

Evaluation 

Group

Case Studies  Often used in combination 

with a more quantitative 

approach, helping evaluators 

better understand causality.

 Selection of cases is critical to 

generating informative value.

 Encourage evaluators to justify and 

document the selection of case 

studies, ex-ante.

 Make sure that the content of the 

case studies is aligned with the 

purpose of the evaluation and other 

data collection efforts.

 Use case study results to illustrate 

the impacts of a program, and 

possibly comment on the validity of 

findings developed through other 

means.

 Treasury 

Board 

of Canada

 UK Cross 

Government 

Evaluation 

Group

Application 

of Findings 

from Other 

Studies

 Generally not recommended, 

unless it is used in the 

development of methods and 

metrics, or the identification 

of data sources.

 N/A  Treasury 

Board 

of Canada
Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

Final Report 31

4 REVIEW OF PAST REPORTS AND PROGRAM EVALUATIONS 

4.1 “Evaluation of the Appalachian Regional Commission’s Infrastructure and Public 

Works Program Projects” 

This report was prepared for the Appalachian Regional Commission by The Brandow Company and 

Economic Development Research Group in 2000. 

4.1.1 Purpose and Goals 

The purpose of the evaluation was to assess past program performance, identify opportunities for future 

improvement, and facilitate the public’s understanding of the benefits of ARC’s infrastructure 

investments. The study focused primarily on economic development impacts, with the exception of 

quality-of-life impacts due to water/sewer improvements.

4.1.2 Key Issues Identified 

1. The report does not cover all of ARC public works investments through the program. Projects 

included in the evaluation sample were limited to those funded from 1990 to 1997. Programs funded 

earlier were often lacking detail and more recently funded projects were in general not mature 

enough to be evaluated. 

2. The report is primarily concerned with economic development impacts. Job creation, business 

service, income growth, economic diversification, tax revenues, and changes in total business output 

that can be attributed to ARC investments were the study’s focus. While residential water and sewer 

projects receive some analysis and discussion of quality-of-life impacts, changes in public health that 

may have resulted from various projects are not quantified. 

3. Some indicators provide a context for local and project analysis but do not provide a basis for 

inferring project cause and effect. The report provides a variety of traditional and innovative 

economic indicators for project impacts (e.g., growth trends, business retention, job replenishment, 

business vitality, industrial diversification). These indicators can provide a context for project 

analysis, facilitate a better understanding of project area economies/needs, and inform qualitative 

discussions of how some projects affected land use/development patterns or entrepreneurial vitality 

in the primary impact areas. The economic diversification and job growth analysis provides specific 

measures of how projects affected existing economic conditions. 

4.1.3 Key Performance Measures 

The study used a variety of performance measures to gauge the extent to which projects achieved their 

projected outcomes and aligned with the strategic goals of the ARC: 

1. Projected jobs and jobs created, jobs attributable to ARC investment 

2. Leveraging rates for other project related funds 

3. ARC’s relative funding contribution

4. Diversification effects of investment on local economic base using location quotients at the national 

and county level 

5. Indirect economic impacts 

6. Local tax base impacts 
32 Final Report

Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

7. Impact/cost analysis of projects 

8. Quality-of-life improvements to residences (water/sewer projects) 

4.1.4 Analytical Approach 

A selection of 99 projects initiated between 1990 and 1997 were analyzed: 2 industrial park projects; 11 

business incubator projects; 15 access roads projects; 51 water/sewer projects. The sample includes 

projects in all states in ARC; reflects metro/non-metro mix in region; includes projects of all eligible 

economic designations; includes very small, very large and average scaled projects. 

1. Surveys: One to four interviews with local/regional development staff, local government/civic 

leadership, private sector representatives for each project. Project profile developed: 

a. Project area distress data 

b. Project data and budget information 

c. Project fiscal and economic analyses 

d. Economic trend analyses of primary impact counties 

e. Economic vitality analyses of primary impact counties: 

i. Comparison of start-up rates nationally and in project area 

ii. Percentage of surviving young firms against US 

iii. Firm retention rate 

iv. Number of jobs lost by failed firms in the firm retention analysis 

f. Interviewee information 

g. Qualitative project objectives and outcomes 

h. Impact comments and discussion 

2. Observation/site visits: Eight site visits to validate project results and develop case studies. 

3. Before/after comparisons and Input-Output Modeling: 

a. Anticipated versus actual project outcomes, in terms of performance measures. Share 

attributable to ARC based on ARC share of total investment 

b. Jobs, personal income generated from business attraction/expansion/retention. Local tax 

revenue impacts 

c. Impacts per dollar of total public investment 

d. Impacts per dollar of ARC investment 

e. Total public investment per job 

f. Private sector leverage = (total dollars private investment/total dollars public investment) 

g. County-level economic profiles developed as baseline. If applicable, local economic 

changes corresponding to project impacts measured and compared to the baseline. 

h. Performance of project counties compared to national trends (i.e., economic 

diversification, entrepreneurial vitality, business growth). 

i. Multiplier impacts using IMPLAN 

4.2 “Program Evaluation of the Appalachian Regional Commission’s Infrastructure and 

Public Works Projects” 

This report was prepared for the Appalachian Regional Commission by BizMiner/Brandow Company, 

Inc. and Economic Development Research Group in October of 2007. 
Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

Final Report 33

4.2.1 Purpose and Goals 

The study sets out to examine infrastructure and public works projects from an economic development 

perspective, to evaluate whether or not the projects achieved their original objectives and how these 

project investments contributed to attaining the Appalachian Regional Commission’s strategic goals. 

The evaluation looked at a representative sample of all infrastructure investments in six different 

categories – industrial parks and sites, water and sewer projects, access roads, business incubators, 

telecommunications and housing – from 1999 through 2005. 

4.2.2 Key Issues Identified 

The study analyzed about 25 percent of ARC infrastructure and public works project investments from 

1999 to 2005. 

4.2.3 Key Performance Measures 

The evaluation considered a variety of performance measures to determine how closely the projects met 

the goals of the ARC strategic plan. These measures include the following: 

1. Number of jobs projected and actually created or retained upon project completion 

a. Anticipated impacts – applicants for funding are required to project impacts 

b. Gathered data on the outcomes and showed the results 

2. Leveraging rates for other project-related funds, including state, local, other federal, and private 

investments 

3. Determination of the agency’s relative funding contribution 

4. Calculation of the job creation rate attributable to ARC’s investment once the impact on other funds 

is considered 

5. Diversification effects of the projects on the local economic base 

6. Indirect and induced economic effects attributable to the project 

7. Impacts on the local tax base resulting from the projects 

8. An impact/cost analysis of the projects 

9. Qualitative quality-of-life improvements provided to residential households served by the housing 

and water and sewer projects 

10. Economic vitality measures such as diversification, mature firm growth, and entrepreneurial vitality 

(start-up growth) 

4.2.4 Analytical Approach 

The evaluation reviewed 104 of the more than 400 investment projects between 1999 and 2005, ensuring 

the selection of a representative sample of all project types. The projects were categorized into three 

broad categories – economic development, community development, and housing development. 

Economic development projects promote business development by attracting new jobs and saving 

existing jobs that are in danger of being lost. Community development projects improve basic health 

and/or quality of life in a community; water/sewer and telecommunications projects generally fall into 

this category. Housing development projects involve the construction or rehabilitation of housing for 

low- and moderate-income residents; these projects do not generate jobs. 

1. Surveys: ARC used interviews to assess the total number of jobs created and retained whenever 

possible and to gather information on estimates of retained wages and new income, supplemented 

with average wage data from the Bureau of Labor Statistics. They also helped determine whether the 
34 Final Report

Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

investments broadened the type of jobs available and the desirability of available jobs. Finally, the 

interviews provided information related to the importance of ARC investment and the role that they 

played. For each project, a profile was developed similar to the 2000 study. 

2. Observation/site visits: Used to validate project results. 

3. Before/after comparisons and Input-Output Model: Measures of economic conditions in each of 

the project areas using indexes for comparison purposes. The indexes are designed to focus on 

strategic ARC objectives including area economic diversity, development of robust patterns of 

growth among area firms, and incubation of a strong entrepreneurial culture. 

a. Economic development impacts were estimated and compared with the actual results for 

the entire project in which the investment was made using a ratio. This is accurate only if 

all of the project results depended exclusively on the ARC funding and none would have 

occurred without it. 

b. Investment ratios were developed to compare the total public funding with actual results 

(private investment value compared to public or ARC funding). 

4.3 “Handbook: Assessing Local Economic Development Opportunities with ARCLEAP (Appalachian Regional Commission Local Economic Assessment 

Package)” 

This report was prepared for the Appalachian Regional Commission by Economic Development Research 

Group, Inc. in 2004. 

4.3.1 Purpose and Goals 

The ARC-LEAP model is set to define key concepts involved in evaluating an area’s economic 

performance, identifying targets for further economic development, and defining priorities for improving 

local economic competitiveness. ARC-LEAP is a tool developed specifically for the ARC and its Local 

Development Districts. The goal is to help practitioners identify target industries for economic 

development in three specific areas: economic assessment, targeting diagnostics, and policy analysis. 

1. Economic Assessment: Evaluation tool for local practitioners to assess current economic conditions 

and likely future trends 

2. Targeting Diagnostics: Diagnostic tool to provide data-based analysis for economic development 

strategies and industries to target for growth

3. Policy Analysis: An analysis tool for assessing potential economic development consequences of 

future policy initiatives, which can affect local costs, labor force skills and/or infrastructure 

characteristics 

The tool is applicable to situations that include high unemployment and low wages, seasonal 

fluctuations in employment, isolation and lack of local opportunities, or over-dependence on a particular 

industry or a few large employers. 

4.3.2 Key Issues Identified 

The analysis is limited by the fact that it is only a tool for analysis, and knowledge of the area is required 

to make correct decisions based on model output. It is important that the tool not be used blindly. It is 

necessary to use a proper basis for comparison, correct business classifications by industry, proper detail 

on area competitiveness, identify differences in business sales and employment trends, and to interpret 

industry targets correctly. The tool and its outputs require careful interpretation. 
Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

Final Report 35

4.3.3 Key Performance Measures 

The study does a base analysis of current conditions, measures performance gaps, and provides insight on 

the advantages and disadvantages of the study area in terms of costs, size and characteristics of the 

workforce and sensitivity of the area each of the various factors based on input-output tables, access to 

transportation, and quality of infrastructure to support growth in the region. 

Targeting diagnostics evaluate sensitivity to a variety of economic factors, strength of area, production 

and labor costs, taxes, labor availability, workers, and transportation infrastructure. 

Policy analysis allows users to analyze the effects of alternative future policies and investments on the 

future business attraction potential of a local area. Estimates include new jobs associated with improved 

business attraction potential. 

4.3.4 Analytical Approach 

This study differs from the other studies reviewed as part of this work effort. Rather than focusing on 

program performance, the model helps policy makers understand existing economic conditions, potential 

industries to target, and prospective economic development impacts associated with future policy 

initiatives. Because it is less a program evaluation tool and more a policy direction tool, the analytical 

approach described below does not follow the format of the other studies reviewed. 

The first step to using the tool is to define the study areas. This requires identification of the study target 

area and comparison areas. The study area is generally a county or multi-county region. Identification of 

the study area should consider four factors: 

1. The area of agency focus 

2. The economic market area 

3. The area of most appropriate data availability 

4. The project policy impact area 

Identification of comparison areas involves identifying one of two kinds of appropriate comparison areas 

based on their characteristics: 

 Local comparison areas are generally adjacent and competing with the target area, and are used 

when looking at business mix and competitiveness factors. These should be of similar urban/rural 

density and share similar natural resources. 

 Industry trend comparison areas are usually larger areas, such as the entire United States, and are 

used to compare local business trends. 

Local Economic Performance Analysis evaluates the mix and performance of industries in the study area 

by comparing the target area to local areas and national averages. This is done using US Department of 

Census County Business Patterns and IMPLAN. The handbook identifies methods to interpret the 

economic performance analysis. 

 Business Mix analysis to contrast the performance of current and historic data in the target area 

with that of the comparison area to estimate an “expected number” of jobs that would occur in the 

study area if it had an industry mix identical to the comparison area. 

 Business Trend Comparison uses shift/share analysis to identify the types of business that are 

particularly thriving or declining in the area and compares their performance to the national average. 

This calculates the percent change in the number of employees in each sector for the study area and 

the US over the past five years using a ratio of the changes. 
36 Final Report

Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

Quantitative measurement of area characteristics identifies key factors affecting the location of 

businesses including the cost of doing business, size of labor and customer markets, and access to 

transportation facilities and broadband connectivity. 

The next step is to rate local facilities and resources at the local level including the conditions of 

industrial sites and buildings, the availability of business support programs, and the level of local labor 

force skills and education training. This is done using a series of six worksheets to analyze business 

facilities. The worksheets identify business support programs, land inventory, building inventory, labor 

force characteristics, and tourism. 

Finally, the tool evaluates the results at an industry level using either business mix or business trends. 

This process has three steps: 

 Analyze differences in business mix and trends against regional patterns and national trends to 

classify local business performance in terms of mix and growth trends; 

 Identify industries that appear to be locally under-represented in terms of mix and/or locally underperforming in terms of trend, and identify them as potential candidates for improvement based on a 

measure of potential for additional job growth over a ten-year period; and 

 Isolate the candidates for improvement that are in industries expected to be growing in the future 

and identify them as “strong candidates for future growth.”

The diagnostic analysis identifies business advantages and disadvantages for each industry in which 

there is a potential for growth and business attraction as identified in the assessment. This is done based 

on a variety of factors including: total production costs; labor costs; land/office costs; energy costs; taxes; 

availability of labor; availability of skilled workers; water transportation; air transportation; rail 

transportation; highway transportation; and broadband access. In this analysis, factors are identified as 

“critical” or “important” local disadvantages based on the level of importance to the competitiveness of 

specific industries. 

Based on the results of this process, a strategic plan should be developed. The strategic plan consists of 

three basic steps: Strengths, Weaknesses, Opportunities and Threats (SWOT) Review, Mission 

Statement, and Strategic Plan Development. 

4.4 “Understanding Water and Sewer Reporting Issues” 

This report was prepared for the Local Development Districts (LDDs) by Amy McAbee-Cummings in 

1999. 

4.4.1 Purpose and Goals 

The Economic Development Infrastructure and Resources Survey (EDIRS) is distributed to the LDDs 

every two years to monitor changes in household access to public water and sanitary sewer systems. The 

most recent survey (1998) revealed that many of the LDDs had difficulty collecting information. This 

report attempts to determine the reasons that collecting this data is problematic and to devise a way to 

obtain information that reflects the true situation of households in Appalachia regarding access to water 

and sewer facilities. 
Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

Final Report 37

4.4.2 Key Issues Identified 

LDDs indicate that a lack of man power and time are the most significant obstacles to obtaining accurate 

data. Some data had to be compiled from scratch, and the information that existed was not always in the 

format requested in the EDIRS. Public sewer systems are not regulated by federal law and there is no 

compiled data, making collection of data related to these systems time intensive. The US Environmental 

Protection Agency compiles information about public water systems, which is available through the 

internet, making this element of the data collection effort more straightforward. Another issue was the 

lack of current household population data; only 1990 county-level US Census data were available. 

4.4.3 Key Performance Measures 

The report attempted to better understand why water and sewer systems data collected by the LDDs 

was incomplete. The report was not intended to measure the performance of the water and sewer 

systems as much as the performance of the LDDs in collecting data for the 1998 survey. The report found 

that a lack of consolidated public sewer data and inconsistent measurement of the number of households 

contributed to the inaccuracy of the survey.

4.4.4 Analytical Approach 

Ten LDDs were selected for phone interviews, in which they were asked about their experiences in 

completing the survey. 

4.5 “Drinking Water and Wastewater Infrastructure in Appalachia: An Analysis of 

Capital Funding and Funding Gaps” 

This report was prepared for the Appalachian Regional Commission by Jeff Hughes, Richard Whisnant, 

Lynn Weller, Shadi Eskaf, Matthew Richardson, Scott Morrissey, and Ben Altz-Stamm of the UNC 

Environmental Finance Center School of Government in July 2005. 

4.5.1 Purpose and Goals 

The purpose of this report was to help policy makers, including the ARC, understand how water and 

wastewater services are provided and funded, as well as what may be done to more effectively meet the 

needs of the region. The study was set forth to answer six basic questions about the current state, need 

for investment, funding sources, funding gaps, financial management strategies and policy 

implementations that can meet the water and wastewater infrastructure needs in the region. 

4.5.2 Key Issues Identified 

Types and sizes of water systems, methods of disposing of wastewater, funding programs, and 

institutional models for providing services vary widely across the states and sub-regions in the area. 

Significantly fewer households in Appalachia have access to centralized drinking water and wastewater 

services than in the rest of the country and those with access to systems pay a much higher percentage of 

their income for these services than elsewhere in the country. Each state has its own approach to 

administering federally supported programs and allocating funding from state-supported programs. 

Extracting county-level data from the existing data sets was difficult due to the manner in which the 

data were collected or compiled. Most of the information used in the analysis was from existing data sets, 
38 Final Report

Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

which found a lack of reliable data to answer fundamental questions such as the percentage of 

households with onsite systems. 

4.5.3 Key Performance Measures 

The study did not identify many performance measures, but rather identified an approach to evaluating 

the level of service for water and wastewater infrastructure to individuals and businesses in the region as 

a whole. 

4.5.4 Analytical Approach 

This study primarily uses public sources and case studies to determine infrastructure needs, and the level 

of funding necessary to meet these needs. Public sources include the Environmental Protection Agency 

(EPA), the US Geological Survey (USGS) and the US Census Bureau, as well as private credit-rating 

agencies. 

1. Surveys and Site Visits:

a. Looked at several different surveys, and there are a large number, but none of them 

provides all of the information necessary to generate needs estimates for Appalachia as a 

whole. 

b. Collected data about actual documented needs and conducted analysis to estimate the 

undocumented needs. 

c. Looked at information on particular towns, regions, and states to examine specific needs 

for drinking and wastewater and compared that to the available data and funds. Visited 

the offices of the towns under study to collect data. 

2. Application of Other Studies:

a. Looked at both top-down and bottom-up studies. Information collected locally is 

different than that collected regionally and at the state level. 

i. Estimates vary in accuracy and the amount of missing data. 

ii. Reasons for missing data can be linked to the manner in which the survey is 

implemented and the perceived incentives or disincentives that systems have for 

providing information, as well as the capacity of a particular system to provide 

information. 

b. Needs data are often pulled together for policy purposes and do not necessarily consider 

the inclusion of the types of infrastructure improvements and costs necessary to improve 

conditions, which presents difficulties in accurately assessing and comparing needs 

across regions. 

4.6 “Government Performance and Results Act of 1993” 

This Act was passed by the 103rd Congress of the United States of America. 

4.6.1 Purpose and Goals 

The goal of this Act is to provide for the establishment of strategic planning and performance 

measurement. The stated purpose and goals are as follows: 

1. Improve the confidence of the American people in the capability of the federal government, by 

systematically holding federal agencies accountable for achieving program results. 
Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

Final Report 39

2. Initiate program performance reform with a series of pilot projects in setting program goals, 

measuring program performance against those goals, and reporting publicly on their progress. 

3. Improve federal program effectiveness and public accountability by promoting a new focus on 

results, service quality, and customer satisfaction. 

4. Help federal managers improve service delivery, by requiring that they plan for meeting program 

objectives and by providing them with information about program results and service quality. 

5. Improve congressional decision making by providing more objective information on achieving 

statutory objectives, and on the relative effectiveness and efficiency of federal programs and 

spending. 

6. Improve internal management of the federal government. 

4.6.2 Key Issues Identified 

There were no key issues identified. 

4.6.3 Key Performance Measures 

Rather than providing specific performance measures, this report provided evaluation guidelines. 

Specifically, agencies are required to prepare annual performance plans and reports that cover each 

program activity set forth in the budget that shall: 

 Establish performance goals to define the level of performance to be achieved by a program activity; 

 Express the goals in an objective, quantifiable, and measurable form unless authorized to be in an 

alternative form; 

 Describe the operational processes, skills and technology, human, capital, information and other 

resources required to meet the performance goals; 

 Establish performance indicators to be used in measuring or assessing the relevant outputs, service 

levels, and outcomes of each program activity; 

 Provide a basis for comparing actual program results with the established performance goals; and 

 Describe means used to verify and validate measured values. 

Each annual report will review the success of achieving the fiscal year’s performance goals and evaluate 

the performance plan for the current fiscal year relative to actual performance. It will also explain and 

describe where a performance goal has not been met, including reasons why the goal was not met, the 

plans and schedules for achieving the established goals and, if the goal is infeasible, reasons why and 

what can be done about it. Finally, the report will describe the use and assess the effectiveness of 

achieving performance goals, including the summary findings of program evaluations that have been 

completed during the fiscal year. 

4.6.4 Analytical Approach 

No specific analytical approach was recommended or highlighted. 

4.7 OMB Circular A-11, “Preparation, Submission, and Execution of the Budget 

Applicable to Federal Agencies” 

This report outlines the guidelines for federal agencies to prepare, submit, and execute their annual 

budgets. It is from August 2011. 
40 Final Report

Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

4.7.1 Purpose and Goals 

The OMB Circular includes the following: 

1. An overview of the budget process. 

2. Development of the President’s Budget and how to prepare and submit materials required for OMB 

and presidential review of agency requests and for formulation of the FY 2013 budget. 

3. Supplementals and amendments, deferrals and presidential proposals to rescind or cancel funds, and 

investments. 

4. Instructions on budget execution, including guidance on the apportionment and reapportionment 

process, a report on budget execution and budgetary resources, and a checklist for fund control 

regulations. 

5. Federal credit programs, including requirements related to the preparation of budget estimates and 

to budget execution. 

6. New requirements under GPRA Modernization Act; the Administration's approach to performance 

management; requirements for strategic plans and annual program performance reports; and an 

overview of the performance plan/performance budget. 

4.7.2 Key Issues Identified 

The OMB Circular covers a great deal of budget-related information. Of relevance to ARC’s program 

evaluation, however, is the Circular’s emphasis on performance goals and measurement as a way to 

promote an agency’s mission. Guidance on how to ensure that the data on which performance is 

measured is provided in the Circular and of relevance to ARC program evaluation. Data quality issues 

outlined in the Circular include: 

 Data Limitations: Performance data must be appropriately accurate and reliable for intended use. 

Significant or known data limitations should be identified and evaluators should include a 

description of the limitations, the impact they have on goal achievement, and the actions that will be 

taken to correct the limitations. Examples of data limitations include imprecise measurement and 

recordings, incomplete data, and inconsistencies in data collection procedures. 

 Verification and Validation: The OMB circular encourages agencies to consider the verification and 

validation procedures outlined below. Agencies should have in place verification and validation 

(V&V) techniques that will ensure the completeness and reliability of all performance measurement 

data contained in their Annual Performance Plans and reports. Agencies are encouraged to consider 

the following verification and validation factors: 

1. Standards and procedures 

a. Source data are well defined/documented; definitions are available and used. 

b. Collection standards are documented/available/used. 

c. Data reporting schedules are documented/distributed/followed. 

d. Supporting documentation is maintained and readily available. 

e. Collection staffs are skilled/trained in proper procedures. 

2. Data entry and transfer 

a. Data entry methodology is documented and followed. 

b. Data are verified. 

c. Procedures for making changes to previously entered data are documented and followed. 

d. Data are available when needed for reporting and other critical decision making cycles. 

e. Data entry staff are skilled and trained in proper procedures. 
Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

Final Report 41

3. Data integrity 

a. Equipment and program reliability cannot compromise data accuracy. 

b. Accountability for data integrity clearly rests with the person entering the data, and the 

responsible program specialist and manager. 

4. Data quality and limitations 

a. Accuracy limits of all data are defined in the context of intended use. 

b. Any other data limitations are explained and documented. 

c. Method for handling anomalous data is established and used. 

d. Third party evaluations are conducted. 

e. Use of externally controlled data is documented. 

5. Oversight and certifications 

a. Accountability for data accuracy exists in responsible employee performance standards. 

b. Responsible officials certify that procedures were followed each reporting period. 

c. Responsible officials certify that data accuracy has been checked each reporting period. 

 External Assessments: Evaluations, peer reviews, and performance audits can mitigate the risk of 

bias in performance reporting. 

 Scope: Because most agencies process a large amount of performance measurement data, agencies 

should apply judgment when deciding which performance measures will be verified and validated. 

 Frequency of Validation and Verification: Data quality should be known to users and the 

sufficiency of its accuracy for answering key questions. Validation and verification should be carried 

out on an appropriately periodic basis (i.e., annually or biennially). 

 Agency Head Responsibility: Agency heads are officially accountable for the accuracy and reliability 

of performance data. The agency head shall include in the transmittal letter of the agency’s annual 

performance review a brief statement on the completeness and reliability of the performance data, 

and on what data limitations exist. 

4.7.3 Key Performance Measures 

Not relevant, as this Circular was not an actual program evaluation, nor did it provide specific guidance, 

other than data related, on measuring performance. 

4.7.4 Analytical Approach 

No specific analytical approach was recommended or highlighted. 

4.8 “Moving Appalachia Forward: Appalachian Regional Commission Strategic Plan 

2011-2016” 

4.8.1 Purpose and Goals 

To ensure that Appalachia’s leaders and citizens have the capacity, capability, and resources needed to 

build and strengthen local economies. The plan is a guide to take targeted and measurable actions toward 

its vision of bringing Appalachia into economic parity with the nation as a whole. ARC is uniquely suited 

to help guide and foster these efforts for change. The plan calls for alignment of resources to maximize 

results and performance measurement that ensures that the partnership is effective and accountable. It is 

intended to support general goals and six-year performance goals. 
42 Final Report

Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

 General Goals 

1. Increase job opportunities and per capita income in Appalachia to reach parity with the nation. 

2. Strengthen the capacity of the people of Appalachia to compete in the global economy. 

3. Develop and improve Appalachia’s infrastructure to make the Region economically 

competitive. 

4. Build the Appalachian Development Highway System to reduce Appalachia’s isolation.

 Six-Year Performance Goals (assuming that ARC’s annual funding remains at the current 

levels) 

1. 120,000 jobs created or retained. 

2. 120,000 households served with new or improved water and sewer infrastructure. 

3. 120,000 citizens of the Region will benefit from enhanced education and job-related skills. 

4. 150 miles of the Appalachian Development Highway System will be opened to traffic (based on 

the current level of transportation funding from the US Congress). 

The ARC Strategic Plan identifies a number of guiding principles for program evaluation and selection, as 

follows: 

 Promote homegrown solutions 

 Create sustainable economic development 

 Address persistent economic distress 

 Seed innovation 

 Stimulate investments by federal, state, and local agencies; philanthropies; and the private sector 

 Support inclusive local decision making 

 Act as a clearinghouse for ideas and expertise 

 Capitalize on existing assets 

 Encourage lifelong learning 

 Seek regional solutions 

 Strengthen global competitiveness 

4.8.2 Key Issues Identified 

There were no specific issues identified. 

4.8.3 Key Performance Measures 

 Leverage: Additional public and private financial and technical support attracted by ARC 

investments 

 Jobs: Gauge involvement in job-generating programs by measuring jobs created and jobs retained 

 Employability: Measure improvements in high school graduation rates, increases in college 

attendance and graduation rates, completion of workforce training programs and the number of 

children in early childhood education programs 

 Infrastructure development and connectivity: Number of citizens served by new or improved 

infrastructure, connectivity improvements particularly between rail and highway, and new highway 

miles 

4.8.4 Analytical Approach 

No specific analytical approach was recommended or highlighted. 
Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

Final Report 43

4.9 “The Magenta Book: Guidance for Evaluation; United Kingdom Cross-Government 

Evaluation Group” 

“The Magenta Book” was developed by the United Kingdom Cross-Government Evaluation Group in 

April 2011. 

4.9.1 Purpose and Goals 

This report aims to meet the needs of policy makers and analysts working in public policy. It explains the 

important issues and questions in designing and managing evaluations; evaluation options that are 

available; why evaluation improves policy making; how to interpret evaluation results and evidence; and 

why thinking about evaluation before and during the policy design phase can improve the quality of the 

evaluation results without hindering the policy process. It is complementary guidance to the HM 

Treasury Green Book, furthering guidance on the evaluation stage of the policy process, and it provides 

consistency if government departments and agencies follow the manual. 

4.9.2 Key Issues Identified 

This report delves into approaches for evaluation of process and improvements by different 

methodologies. 

4.9.3 Key Performance Measures 

The performance measures vary by the type of evaluation being conducted. 

4.9.4 Analytical Approach 

This report outlines approaches for individual interviews, group interviews, observation/participation, 

surveys, consultative and deliberative methods, statistical analysis of quantitative data, document 

analysis, and ethnography, as well as considerations for undertaking an evaluation and the stages and 

development of evaluations. 

The book identifies practical issues to take into account when designing evaluations including laying out 

the main steps of the evaluation process and considering the timing and resources necessary to conduct 

the evaluation. The main steps in the evaluation process are as follows: 

 Define policy objectives and intended outcomes 

 Consider implications of policy design for evaluation or feasibility 

 Define the audience for the evaluation 

 Identify the evaluation objectives and research questions 

 Select evaluation approach 

 Identify data requirements 

 Identify necessary resources and governance arrangements 

 Conduct evaluation 

 Use and disseminate evaluation findings 
44 Final Report

Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

The guidebook then highlights the stages of an evaluation as follows: 

 Recognize there are a number of stages in an evaluation, including determining which questions to 

ask, which type of evaluation is appropriate, and when and how to carry out the evaluation. 

 Develop a logic model, define audience for the evaluation, identify evaluation objectives and research 

questions, select evaluation approach, identify data requirements, identify necessary resources and 

governance arrangements, and conduct the evaluation. 

 Plan by specifying objectives, timeframes, resource requirements, governance arrangements and 

terms of reference. Consider who will use the findings and what for, to determine how to undertake 

the process. 

 Consider that evaluation objectives and research questions should guide a review of existing 

evidence relevant to the research question. 

 Make certain that evaluation planned to answer questions of immediate interest is also capable of 

having a longer-term strategic influence. 

Key program evaluation methods discussed included: 

Case Studies: Case studies, along with process evaluation and action research, can be used to evaluate 

the implementation and delivery of a policy to provide feedback on a wide range of issues. This can 

include whether policy is being implemented as planned, what is working and what is not, and whether 

the expected outputs and outcomes are being delivered. It is important to consider information 

requirements at the planning stage for any economic evaluation. These evaluation techniques use a range 

of qualitative and quantitative research methods including one-on-one interviews, group interviews, 

surveys, and observations. Multiple methods are often used to provide triangulation of data and 

corroborate findings. 

Causal and Statistical Analysis: The guidebook sets out steps for empirical impact evaluation using 

statistical techniques. It is important to note that this is not feasible for every policy and there may be 

data constraints. The goal of the empirical impact evaluations is to determine whether a policy caused a 

particular outcome to occur by measuring the outcome compared to an estimate of what would have 

happened without the policy. These impact evaluations are not a guarantee of producing the “right 

answer” and can sometimes produce false results; risk of false results can be mitigated through careful 

design of the research, and sufficient investment in data collection. When using this technique it is best 

to control the comparison group to take into account the possibility of selection bias. Reporting the 

results of an evaluation should distinguish between descriptive statistics on outcomes and true impact 

evaluation that takes potential non-policy causes for observation into account. 

There are several different types of comparison groups that can be used. Overall, the design of the 

empirical study is incredibly important, and the types of data analysis should be considered in the 

planning process. The guidebook suggests the possibility of using time trends through a difference-indifferences approach, or instrumental variables and before and after studies if there are no actual 

comparison groups. 

Finally, after the evaluation it is important to draw together and report the evidence of the evaluation. 

The use and dissemination of the findings and the strategy for synthesizing the results should be decided 

upon during the planning stage to avoid any possible accusations of selecting the results that best 

support a particular viewpoint. 
Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

Final Report 45

4.10 “Program Evaluation Methods: Measurement and Attribution of Program Results, 

Third Edition” 

This study was prepared by and for the Deputy Comptroller General Branch, Treasury Board of Canada, 

Secretariat. 

4.10.1 Purpose and Goals 

The report discusses the appropriate methods for achieving timely, relevant, credible, and objective 

findings/conclusions on program performance, based on valid and reliable data collection and analysis. 

Its focus is on the methodological considerations present in the development of a study to assess program 

outcomes. 

4.10.2 Key Issues Identified 

There are three phases to an evaluation: 

1. Evaluation assessment or framework: Planning phase for the evaluation

2. Evaluation issues 

a. Continued relevance: Program continues to be relevant to government priorities and 

citizen needs 

b. Results: Program meets its objectives, stays within budget, and does not cause unwanted 

results 

c. Cost effectiveness: Program involves the most appropriate, cost effective and efficient 

method to meet objectives 

3. Evaluation study 

a. Decision-making based on findings/recommendations 

The study recommended that the evaluator establish the incremental effects of the program and be wary 

of making generalizations based on the evaluation-determined results of the program. It is only 

appropriate to generalize in planning of a future program if the program can be replicated and is not 

specific to a particular time, place or other set of circumstances. To lend greater credibility to the 

evaluation findings, evaluators should address evaluation issues from a number of different perspectives 

using multiple lines of evidence. 

4.10.3 Key Performance Measures 

This study focused on analytical approaches to program evaluation, but not on specific performance 

measures. 

4.10.4 Analytical Approach 

The study discussed the following evaluation strategies: 

1. Surveys: Approach is versatile and, when properly done, produces reliable and valid 

information. They require expertise in their design, conduct and interpretation and can be 

easily misused. Several basic steps should be followed before implementing a survey: 

a. Identify what specific information will address a given evaluation issue. 
46 Final Report

Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

b. Pre-test the surveys on a representative sample of the survey population. Pretesting 

provides information on clarity of questions, response rate, time and length to complete, 

appropriateness of survey method. 

c. Ensure that required data are not available elsewhere and cannot be collected more 

efficiently and appropriately by other data collection methods. 

d. Keep in mind that collecting “nice to know” data adds to the cost of the program 

evaluation. 

2. Observation/Site Visits: Technique involves on-site visits to locations where the program is 

operating to observe activities and to take notes. Notes from observations should be written up 

immediately after the visit and be detailed. Reliability/validity of observations depends on the 

skill of the observer and the observer’s awareness of any bias he/she may bring to the task. 

Program staff may alter their behavior if they know they are being observed. 

3. Case Studies: Usually involve a variety of data collection methods. To measure program 

results, a case study of a critical case may be more defensible than the case study of a 

representative sample. They allow the evaluator to perform an in-depth analysis that would 

not be possible with more general approaches. These are typically expensive and time 

consuming. As a result, the set of case studies chosen will usually lack a statistical basis from 

which to generalize conclusions. 

4. Analysis of Experimental Data: This approach creates groups through the random 

assignment of participants to a “treatment” or separate “other than treatment” (i.e., control) 

group. This ensures groups to be compared are equivalent. It is the most rigorous approach 

available for establishing causal relations between programs and results. Its main drawback is 

that experiments are often difficult to implement. 

a. Quasi-experimental designs are similar but do not use randomization to create treatment 

and control groups. Treatment group is usually already given and one or more comparison 

groups are selected to match the treatment group as closely as possible. These take 

creativity and skill to design but can give highly accurate findings. 

b. Implicit designs make a measurement after exposure to the program and make 

assumptions about conditions prior to the program. It is the most typically used but least 

rigorous. They are typically weak in terms of measuring changes and attributing them to 

the program. It is basically a post-program design with no control group. They are flexible, 

versatile and practical to implement. They offer little objective evidence of the results 

caused by the program. 

5. Simulation Modeling: Simulation models allow evaluator to estimate the incremental effects 

in complex and uncertain situations. Some skill in building quantitative models is required and 

the dynamics of the program must be well understood. A simulation has three main 

components: 

a. Input data 

b. Mathematical model 

i. Stochastic – incorporates a random data generator 

ii. Deterministic – does not incorporate a random data generator 

c. Output data 

6. Application of Other Studies: A literature or file review enables an evaluator to make the best 

use of previous work in the field under investigation and learn from their experiences, findings, 

and mistakes. Past research may suggest hypotheses to be tested or evaluation issues that 

should be examined in the current study. Specific methodological difficulties can be identified 

and mitigated. Evaluation questions may be directly answered on the basis of past 

work/redundant data collection in some cases. Sources of secondary data may be uncovered 
Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

Final Report 47

and lessen the need to collect primary data. It is an economical approach but may not be 

entirely relevant and it may be difficult to determine the accuracy of the data gathered through 

a previous study. 

7. Other: 

a. Expert opinion – Can be collected and summarized systematically. It is a method best 

suited to supplementing other measures of program outcomes. Identifying a large enough 

group of experts may be difficult and opinions may differ. Experts may also not be equally 

knowledgeable. Should not be used as a sole source of data for an evaluation. 

b. Statistical analysis – Statistical tabulations, graphical displays and statistics (e.g., mean, 

variance) can depict key characteristics of the data. Drawing conclusions about the 

relationships among variables and to generalize these conclusions to other situations is 

also a use of statistical analysis. In this case, all that is being established is a relationship. 

To conclude that the program caused the result requires a more rigorous analysis. This 

approach requires expertise. Not all program results can be analyzed statistically and the 

way data are categorized can distort as well as reveal important differences. Nonstatistical data analysis allows the evaluator to use all available information, even that 

information that is not easily quantified. The validity and accuracy of conclusions of nonstatistical analysis depend on the skill and judgment of the evaluator and logic of 

arguments presented. 

c. Causal model – An equation that describes the marginal impact of a set of selected 

independent variables on a dependent variable. This approach focuses on variables to be 

included in the model (endogenous and exogenous) and their postulated relationships. 

The program is only one of several independent variables that are expected to affect the 

dependent variable. 

d. Input-output model – Static economic model designed to depict the mutual 

interdependence among different parts of an economy. It describes how one industry uses 

the outputs of other industries as inputs, and how its own outputs are used by other 

companies as inputs. They can be misused in evaluations. Offsetting negative effects 

generated by taxes and borrowing necessary to support a program, for example, may not 

have been reflected in the I-O model. 

e. Micro-economic analysis – This model describes the economic behavior of an economic 

unit (e.g., individual, household, company). They are typically represented by equations 

that depict demand and supply functions for a good or service. These equations describe 

the relationship between price and output. Usually this model type requires use of an 

economist. 

f. Macro-economic analysis – These models deal mainly with inflation, unemployment and 

large aggregates such as gross national product. The model can identify critical links 

between aggregate broad variables, but they can generate erroneous results if they omit 

key factors. Many have poor predictive capability, particularly in the short run. If the 

program is large, relative to the economy, and derived impacts are long term, they can be 

appropriately used. 

g. Cost- benefit analysis – This analysis looks at the program’s net worth and can judge and 

compare program alternatives. The issue of attribution or incremental effect of a program 

must be addressed before doing this type of analysis. They can often help an evaluator 

identify the full range of costs associated with a program but they do not, in and of 

themselves, explain particular outcomes and results. A sensitivity analysis of key 

assumptions is important. 
48 Final Report

Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

4.11 “Designing Evaluations” 

This report was prepared by the United States General Accounting Office, Program Evaluation 

Methodology Division for use by Congressional decision makers, March 1991. 

4.11.1 Purpose and Goals 

The report provides evaluation design ideas aimed at producing answers to descriptive, normative and 

impact (cause-and-effect) questions. 

4.11.2 Key Issues Identified 

1. Good front-end planning of evaluation design is critical. There are several key elements of a 

design that should be specified before information is collected. The choices made with respect 

to the following elements can significantly impact the quality of information that can be 

acquired, the strength of the conclusion, and the cost, timeliness and usefulness of the 

evaluation. Elements for consideration: 

a. Kind of information to be acquired; 

b. Sources of information (for example, types of respondents); 

c. Methods to be used for sampling sources (for example, random sampling); 

d. Methods of collecting information (for example, structured interviews and selfadministered questionnaires); 

e. Timing and frequency of information collection; 

f. Basis for comparing outcomes with and without a program (for impact or cause-andeffect questions); and 

g. Analysis plan. 

2. Evaluator should determine whether a question is descriptive, normative or impact (causeand-effect) as part of the evaluation design. 

3. Design should reflect the size of the program being evaluated and likely use of the evaluation 

results. 

4. Questions should be posed clearly, addressed appropriately, and inferences should be drawn 

“commensurate with the power of the design and the availability, validity, and reliability of the 

data.”

5. Consider utilizing a “pilot case” to test the initial assumptions about the program, data, and 

evaluation methodology is tested in the field. Choosing a site for the pilot case that represents 

the average is important. 

6. Time, cost and staff expertise should be considered when designing an evaluation. 

7. Secondary constraints include location and facilities. Make certain that facilities include 

equipment required to conduct surveys/observations. 

8. Be sure that evaluator is aware of previous work completed. 

4.11.3 Key Performance Measures 

Key performance measures were not addressed in this report. 

4.11.4 Analytical Approach 

The study discussed the following four evaluation strategies: 
Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

Final Report 49

1. Surveys: When considering a survey, it is important to define whether the evaluation is 

descriptive or normative. In addition, the survey should be structured so that conclusions 

about the sample and out-of-sample population can be drawn. Typically, information is 

collected in closed form where respondent chooses from responses provided by the evaluator. 

Probability sampling is recommended and it is important to choose neutral turf for survey 

implementation. Several survey types were mentioned in the study: 

a. Cross sectional – Measurements are made at a single point in time. Proper sampling, 

pretesting survey instrument, following up on non-respondents are critical. A broad 

literature review and thorough understanding of events/conditions in question is 

important for causal inference. 

b. Panel survey – Similar to cross-sectional survey but has the added feature that the 

information is acquired from a given sample unit at two or more points in time. Sampling, 

pretesting of survey instruments, non-respondent follow-up and causal inference must be 

paid attention to in designing panel surveys for evaluation. 

c. Criteria referenced – Evaluators can use this survey instrument to answer normative 

questions, which compare actual performance to external standard of performance. 

Causal inference is not possible because the evaluation does not produce an estimate of 

what the outcomes would have been in the absence of the program. 

2. Observation/Site Visits: Facilitates drawing inferences about programs to answer impact 

questions. Field experiments take place in less contrived settings. It is generally an impact 

(cause-and-effect) approach to evaluation. Some recommendations highlighted in the study 

include: 

a. Considering social and cultural mores of the area where the evaluation is to be conducted. 

b. Considering whether the population being studied is stable – transient residence vs. same 

home for 40 years. 

c. Assessing whether the travel cost to the site is justifiable. 

3. Case Studies: An analytical description of an event, process, institution or program. One 

reason for conducting a case study is that the data required is beyond what could be collected 

through a simple survey. This type of evaluation may be either descriptive or normative. The 

study indicated that a case study is weaker than a survey in terms of external validity and it 

emphasized the importance of good design of the case study. There were several case study 

types discussed in the report: 

a. Single case – Information acquired about a single individual, process, or entity. Allows 

collection of qualitative information that describes events and conditions from various 

points of view. Selection of a case and data collection may be challenging. Analyzing and 

reporting qualitative data can be difficult and the evaluation design must have explicit 

plans for these tasks. Case studies can be illustrative of an event or condition, exploratory 

as providing preliminary information, or investigative of a problem or best practice. 

b. Multiple case – This study may produce stronger conclusions than a single case study. 

May be appropriate for evaluating either program operations or results. Selecting variety 

among cases is important to avoid bias, as is uniform information collection. 

c. Criteria-referenced – Case studies can be adapted to answer normative questions about 

how well program operations or outcomes meet certain criteria. How to reach consensus 

on the criteria and how to measure performance against a criterion should be 

considerations. Carefully choosing the sample of cases to evaluate is also critical. 

4. Analysis of Experimental Data: Some units of study are randomly assigned to a treatment 

group and some are assigned to one or more comparison groups. The program’s effects are 

estimated by comparing outcomes for the treatment group with outcomes for each comparison 
50 Final Report

Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

group. This is usually the strongest design for causal inference and it provides a way to assess 

weaknesses or potential weaknesses in a cause-and-effect design. It typically permits very 

persuasive statements about the cause of observed outcomes. In evaluating a program to 

determine whether it causes a particular outcome, this approach would expose one group to 

the program and the other would not be exposed. The difference in outcomes would be 

attributed to the program, with some qualifications. One con to this approach is that if drop 

out rates between the groups vary, then the evaluator’s estimate of effects may be distorted.

a. Non-equivalent comparison group – The main purpose is to answer impact (cause-andeffect) questions. A treatment group and one or more comparison groups participate in 

this approach. Unlike a true experiment, membership is not randomly assigned. As a 

result, causal statements about treatment effects may be weakened. This issue can be 

somewhat mitigated by choosing comparison groups that are as similar as possible. This 

approach is administratively easier to implement than true experiments. It is important 

that data on how the groups differ is collected early on. 

5. Before/After Comparisons: This approach compares outcomes for the units of study before 

the units were exposed to a program to outcomes measured one or more times after they began 

to participate in it. There is no separate comparison group. Multiple evaluation points in time 

strengthen this approach. With a sufficient number of evaluation points, an interrupted time 

series analysis can be applied to this design to help draw causal inferences. A rule-of-thumb is a 

minimum of 50 observations. Data consistency is also an important element to before/after 

comparisons. Time series need to be free of definitional and measurement changes, because 

these can be mistaken for program effects. 

6. Use of Existing Data: Because primary data collection can be expensive, it is wise to see if 

existing data will suffice. Even if it is determined that new data is needed, the analysis of 

already available data may provide quick questions that will be more completely addressed 

with new data at a later time. 

a. Secondary data analysis – Secondary data analysis might answer an evaluation question 

utilizing other data, such as US Census. Most likely answers in secondary data analysis 

are descriptive, but normative and impact (cause-and-effect) questions can be considered. 

Some potential issues with this application is that data cannot be accessed, it is difficult 

to document data and collection, and the data collected for another purpose may not 

exactly meet the need of the current evaluation. 

b. Evaluation synthesis – Evaluator combines a number of previous evaluations that 

essentially address the current question. The goal is to provide a conclusion that is more 

credible than that of any one study. This approach often reveals informational gaps, which 

may be useful to know when clarifying a debate or moving forward in the gathering of 

new data. 

4.12 “Public Works Program Performance Evaluation” 

This May 1997 report was prepared by the Rutgers University, New Jersey Institute of Technology, 

Columbia University, Princeton University, National Association of Regional Councils, and University of 

Cincinnati for the Economic Development Administration (EDA), U.S. Department of Commerce. 

4.12.1 Purpose and Goals 

The study evaluated impacts of 205 EDA Public Works Program projects, including industrial parks, 

water and sewer systems, industrial access roads, vocational/technical education facilities, harbors and 
Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

Final Report 51

ports, and business incubators. The purpose of these projects is to attract new industry, encourage 

business expansion, diversify local economies, and generate long-term, private-sector jobs. 

4.12.2 Key Issues Identified 

A small percentage of grantees cited EDA program management issues. No other program issues 

identified. With respect to data collection, difficulty of obtaining indirect job data noted. 

4.12.3 Key Performance Measures 

 Private sector dollars invested in the project 

 Additional public dollars invested in the project 

 Additional dollars invested directly related to, but not part of, the project 

 Other dollars invested indirectly related to the project 

 Jobs created or retained 

 Increases in local tax base 

 Intended beneficiaries located, retained or expanded in the community 

 Diversification of local economy (fulfillment of strategic plan) 

4.12.4 Analytical Approach 

A methodology was developed to attribute jobs and private sector investments to EDA investments. 

EDA’s method relied heavily on the judgment of the grantees and regional office directors, which could be 

subject to positive bias. While difficult to develop objective measures of attribution, respondents might 

have been asked a more rigorous set of questions. Alternatively, measures of local fiscal capacity might 

have been developed to assess the community’s level of fiscal distress and need for federal support.

EDA’s investments were classified as either critical (i.e., without funding, project would not have been 

undertaken), resulting in attribution of all jobs and private investment to EDA’s investment; or essential 

(i.e., without funding the project would have been seriously compromised) resulting in attribution of 

jobs and private investment to EDA’s investment proportionate to the investment of other public 

funders. Based on classifications made by grantees and regional office directors, the research team 

classified EDA investment as essential in all cases when the EDA share was less than 25 percent. 

Information was collected through the following means: 

1. Surveys: Mail and telephone solicitation to obtain project statistics and to quantify project 

outcomes, involving a mail survey and callbacks to obtain and verify project information. 

2. Observation/Site Visits: Seminars with grantees held in 13 locations to instruct them on 

technicalities of information requests and the specific information that would be required as part of 

the evaluation. In addition, half-day site visits to 25 percent of grant locations for field verification of 

scale and health of the project, and number of direct and indirect jobs. 

3. Case Studies: A brief one-page case study write-up for the projects visited was also compiled. 

4. Input-Output and Statistical Analysis: In a related study, EDA conducted input-output analysis to 

estimate indirect employment and investment impacts of the EDA investments, and conducted 

regression analysis to estimate the impact of the EDA investments on countywide employment and 

wages. 
52 Final Report

Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

5 PERFORMANCE MEASURE DATABASE 

5.1 Discussion of Performance Measure Database 

A database of performance measures was developed based on information collected through a survey sent 

to all grantees that had received infrastructure funding from ARC between 2004 and 2010. The survey 

requested information about the outcomes and long-term impacts of these projects. 

The team started with the ARCnet infrastructure database of all ARC funded infrastructure projects 

from 2004 to 2010. This database of 811 projects contains information from the Grant application, 

including project number, open/closed status, amount of ARC funding, location, project type, and 

anticipated outcomes and outputs. 

Based on the information in the database and an attempt to be consistent with other program 

evaluations, such as those for education and health services, the team created a survey to distribute 

electronically to grantees. The goal of the survey was to collect as much information as possible about 

actual project outputs and outcomes as compared to the anticipated outputs and outcomes that were 

provided on the grant applications. A copy of the electronic questionnaire is available in Appendix A. 

An electronic correspondence (see Appendix B) was sent to all grantees with information about the 

survey purpose and background, accessing the survey on-line and requesting participation. For those emails that were undeliverable and those grantees that did not provide e-mail addresses, paper letters 

were mailed to the provided addresses containing the same information as the electronic correspondence. 

Of the 811 grantees that were contacted, the team received 197 responses that contained enough 

information to analyze. There were 211 survey responses, but 14 were either blank or duplicate and thus 

not counted. There were respondents from every state, with Mississippi, Kentucky and Ohio accounting 

for the highest response rates with 28 responses (14 percent) from Mississippi and 26 (13 percent) each 

from Kentucky and Ohio. 

A breakdown of projects funded and survey responses received by state is presented in Table 4 below. As 

mentioned previously and shown below, Mississippi received the largest number of grants during the 

study period, followed by Ohio and Kentucky. This is consistent with the response rate; these three 

states provided the most survey responses. Alabama received 10 percent of grants but only accounted for 

seven percent of responses. Looking at response rates relative to the share of projects, New York, West 

Virginia, Maryland and Kentucky had the highest response rates. 

Table 4: Number and Share of Grants Received and Survey Responses by State 

State MS KY OH PA TN WV MD AL NY NC SC GA VA Total

Database Projects 120 82 92 74 65 44 44 79 35 51 32 67 26 811

Share of Projects 15% 10% 11% 9% 8% 5% 5% 10% 4% 6% 4% 8% 3% 100%

Survey Response 

Projects 28 26 26 16 16 16 14 13 13 9 9 8 3 197

Share of Responses 14% 13% 13% 8% 8% 8% 7% 7% 7% 5% 5% 4% 2% 100%

Response Rate 23% 32% 28% 22% 25% 36% 32% 16% 37% 18% 28% 12% 12%
Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

Final Report 53

The majority of responses were for either water or sewer systems,6 followed by access roads, 

telecommunications projects and industrial site developments. These five categories of projects 

accounted for more than 95 percent of responses. A breakdown of responses by project type is shown in 

Table 5 below. The distributions of survey responses are very similar to the distribution of all projects 

funded. 

Table 5: Number and Share of Survey Responses by Project Sub-Type 

Projects by Sub-Type

Sub-Type

All 

Projects

% of 

Total

Survey 

Responses

% of 

Total

Water System 315 38.8% 70 35.5%

Sewer System 217 26.8% 52 26.4%

Access Road 116 14.3% 30 15.2%

Telecommunications 65 8.0% 19 9.6%

Industrial Site Development 86 10.6% 17 8.6%

Highway 1 0.1% 3 1.5%

Intermodal Facility 3 0.4% 2 1.0%

Rail 3 0.4% 2 1.0%

Airport 1 0.1% 1 0.5%

Gas Line 4 0.5% 1 0.5%

Total 811 100% 197 100%

The largest number of responses was water systems in Kentucky, with 17 survey responses received by 

the team. Additionally, Ohio, Tennessee, and West Virginia each had nine responses relating to water 

projects and Mississippi had nine sewer system responses. The breakdown of responses by state is 

presented in Table 6 below, with the blue shaded cells indicating the highest response rates. A table 

showing the distribution of the full database by project type and state can be found in Appendix C. The 

full database and the survey responses are relatively consistent in terms of distribution. 

Table 6: Survey Responses by Self-Identified Project Type and State 

AL CO GA KY MD MS NC NY OH PA SC TN VA WV Total

Access Road 0 0 3 0 8 6 1 1 4 6 1 0 0 0 30

Airport 0 0 0 0 0 0 0 0 0 1 0 0 0 0 1

Gas Line 0 0 0 0 0 0 0 0 0 0 0 1 0 0 1

Highway 0 0 0 0 0 0 0 1 1 0 0 0 1 0 3

Industrial Site Development 2 0 0 0 4 4 0 0 1 3 1 2 0 0 17

Intermodal Facility 0 0 0 1 0 0 0 0 0 1 0 0 0 0 2

 

6

 Note that the responses from the ARCnet database for all projects are based on the project sub-type as categorized 

by ARC. The categories for the survey responses are based on information provided by the respondent. While 

generally consistent, there were some cases where the infrastructure type in the response was different than the 

database. 
54 Final Report

Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

Rail 1 0 0 0 0 0 0 0 1 0 0 0 0 0 2

Sewer System 4 0 1 8 1 9 2 2 9 2 3 4 0 7 52

Telecommunications 2 0 0 0 2 2 1 7 1 1 1 0 2 0 19

Water System 4 0 4 17 0 6 5 2 9 2 3 9 0 9 70

Total 13 0 8 26 15 27 9 13 26 16 9 16 3 16 197

Note: Project Type is based on the type identified by survey respondent and not ARCnet project type. 

As mentioned above, the team considered 197 of the responses complete enough to analyze; however, not 

all of these responses provided complete information. The survey contained an option to enter a value of 

“Did Not Track” or “Not Applicable” for some questions, which meant that a numerical response was not 

provided for every question. In other cases, respondents left questions that were not mandatory blank. 

Again, no numerical value was provided. Thus, many of the performance metrics that the team attempted 

to measure, including the size of the capacity improvement (miles, linear feet, millions of gallons per day, 

etc.) and the outputs, including concepts such as telecommunications speed, previous water system 

capacity, number of water system failures, access road traffic, etc., were not tracked with numeric values. 

This reduced the number of results that could be quantitatively analyzed. Complete details of the survey 

results are available in Appendix D. 

5.2 Analysis and Summary of Project Performance Measure Database 

The project team examined more than 50 performance measures to assist ARC staff, managers, and grant 

applicants in the process of evaluating the benefits of the program as clearly and transparently as 

possible. 

The analysis focuses on four key areas: 

 How is the program funding spent? These measures describe how ARC has made investments 

across several dimensions, including by state, by year, and by the economic status of regions. 

 How do recipients perceive program benefits? These measures describe how grant recipients 

perceive the long-range impacts of the projects on the economy, the environment, and the quality 

of life in the communities. 

 What are the real impacts of the program? These measures evaluate economic development 

impacts, such as jobs created/retained, businesses created/retained, households/businesses 

improved, communities served, etc. In addition, this tier includes project outcome measurements, 

such as linear feet of pipes installed, miles of road built, telecommunication sites developed, and 

miles of rail tracks built. Real impact performance measurements were evaluated for projects 

that are currently closed and for which real measurements of impacts can be made. Projected 

outcomes for projects still under development are also captured. 

 How cost-effective is the program? These measures compare the relative costs and outcomes of 

the projects by infrastructure type. 

In order to conduct the analysis, the team reviewed the ARCnet database and 2012 survey responses and 

combined the information received from the survey with the information in the ARCnet database. Several 

challenges were encountered sorting the data since many respondents left answers blank or, in many 

instances, did not know the outcome or benefits of the project. To accurately depict the benefits of the 

program, assumptions were made in the process of cleaning the data. The full ARCnet database was used 

to obtain information about the funds allocated by ARC to individual projects per fiscal year. Though all 
Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

Final Report 55

projects considered were funded between 2004 and 2010, not all of the projects are considered “closed.” 

Also, when a respondent provided a range of values – for example “between 10-15 jobs created” – the 

median was assumed. 

The following sections provide results from the performance measure analysis and the econometric 

analysis conducted as part of this study. 

5.2.1 Descriptive Statistics 

From 2004 to 2010, the ARC contributed nearly $206.7 million toward funding 811 various infrastructure 

projects that totaled more than $1 billion in value. 

Based on the survey response, a total of 197 projects funded in part by ARC between 2004 and 2010 were 

evaluated in detail. The sample projects are distributed among all 13 Appalachian states and include a 

wide range of infrastructure programs, including highway corridors, water and sewer infrastructure 

projects, telecommunications projects, rail, gas line, and airports. Of the 197 projects analyzed, 123 (62 

percent) are related to water and sewer infrastructure. Multiple access road projects, 

telecommunications projects and industrial site development projects were also funded. Figure 1 below 

shows the distribution of these 197 projects by infrastructure type.
56 Final Report

Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

Figure 1: Project Distribution by Self-Identified Infrastructure Type, Survey Responses (n=197) 

It is important to note that this information and the following detailed analysis are strictly based 

on the self-identified responses to the survey, and as highlighted in the previous section, the 

characteristics of the projects for which surveys were completed may vary from the characteristics of the 

overall investments in infrastructure funded during the relevant time period by ARC. 

ARC Investments Overview 

Business location decisions are made based on a number of factors, including the labor force, access to 

suppliers, appropriately developable land, and adequate public infrastructure. ARC investments are 

intended to enhance economic development in the Region by attracting new industry, encouraging 

business expansion, diversifying local economies, and generating permanent, private-sector jobs. This 

section of the report aims to identify how ARC has made investments across several dimensions, 

including by state, by year, and by the economic status of regions. 

Expenditures 

The survey responses accounted for a total of slightly more than $56 million of the $206.7 million that 

ARC invested during the time period. Of this, nearly $46.5 million was spent on water and sewer system 

investments, $28 million on water (50 percent) and $18.4 million on sewer systems (33 percent) 

accounting for 83 percent of the total spending from 2004 to 2010, according to the survey responses. 

The ARCnet database includes all projects funded by ARC from 2004 to 2010. The share of water and 

sewer projects actually funded is slightly less than the share of these project types in the survey 

responses; 78 percent of all projects in the ARCnet database were water or sewer systems. Forty-five 

percent, or $93 million, of all ARC grant funds during the 2004-2010 period were spent on water system 

projects and 33 percent, or $67.4 million, were spent on sewer system projects. As seen in Figure 2, water 

15%

1%

1% 2%

9%

1%

1%

10% 26%

36%

Access Road

Airport

Gas Line

Highway

Industrial Site Development

Intermodal Facility

Rail

Sewer System

Telecommunications

Water System
Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

Final Report 57

and sewer system projects account for the largest segment of spending in each year of the study period as 

well. While the distribution of projects by type in the survey responses is not identical to the 

distribution of projects by type in the ARCnet database, the relative shares by project type are similar. 

Figure 2: ARC Investment ($) by Year and Self-Identified Infrastructure Type, Survey Responses 

(n=197) 

The survey responses show an average of $8 million spent annually, with expenditures in 2008 and 2010 

exceeding $10 million and expenditures in 2006 the lowest at approximately $5.7 million. The total 

project database also demonstrates relatively even spending with an average of $29.5 million spent per 

year during the study period, with the highest spending in 2009 – $33.2 million – and the lowest 

spending in 2006 – $27.0 million. 

Consistent with the full ARCnet database of 811 projects, West Virginia and Kentucky received the 

highest levels of funding among survey respondents. Based on the survey responses, funds were primarily 

spent on water and sewer infrastructure projects. West Virginia received $10.6 million for water systems 

projects and $5.3 million for sewer system projects, while Kentucky received $6.2 million and $3.5 

million, respectively. 

This can be compared to the $25 million in water system improvements and $10.5 million in sewer 

improvement in West Virginia from the full database and the $18.4 million in water and $10 million in 

sewer improvements in Kentucky. Figure 3 presents the number of projects by type and by state as 

reflected in the survey responses. 

$0

$2,000

$4,000

$6,000

$8,000

$10,000

$12,000

2004 2005 2006 2007 2008 2009 2010

Thousands

Water System

Telecommunications

Sewer System

Rail

Intermodal Facility

Industrial Site Development

Highway

Gas Line

Airport

Access Road
58 Final Report

Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

Figure 3: ARC Funding by State and Self-Identified Infrastructure Type, Survey Responses 

(n=197) 

Projects by State 

Based on the survey responses, the ARC contributed to the greatest number of infrastructure projects in 

Mississippi (27), Kentucky (26), and Ohio (26) between 2004 and 2010. Of the projects in Mississippi, 

nine were sewer systems, six each were water systems and access roads, four were related to industrial 

site development and two were telecommunications related. Based on survey responses, ARC 

contributed funding to 17 water systems, eight sewer systems, and one intermodal facility project in 

Kentucky. The mix of projects in Ohio was nine water and nine sewer projects, four access roads, and one 

each of highway, industrial site, rail, and telecommunications projects. The full distribution of projects 

by state, based on survey responses, is presented in Figure 4 below. 

$0

$2,000

$4,000

$6,000

$8,000

$10,000

$12,000

$14,000

$16,000

AL GA KY MD MS NC NY OH PA SC TN VA WV

Thousands

Access Road Airport Gas Line

Highway Industrial Site Development Intermodal Facility

Rail Sewer System Telecommunications

Water System
Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

Final Report 59

Figure 4: Number of Projects by State and Self-Identified Infrastructure Type, Survey Responses 

(n=197) 

County Economic Status 

The ARC uses an index-based county economic classification system to identify and monitor the 

economic status of Appalachian counties. The system involves the creation of a national index of county 

economic status by comparing each county's averages for three economic indicators—three-year average 

unemployment rate, per capita market income, and poverty rate—to national averages. Based on this 

information and their relative position nationally, the Appalachian counties are classified into one of five 

economic status designations: distressed, at-risk, transitional, competitive, or attainment. 

Distressed counties rank in the worst 10 percent of economically depressed counties in the nation, and 

attainment counties are the strongest 10 percent of the nation’s counties. The largest share of the nation’s 

counties – between the worst and best 25 percent – are transitional. The following figure shows ARC 

investments by economic status in the year the project was approved for funding. The Appalachian 

Region has a higher proportion of distressed and at-risk counties than the nation as a whole. Thirtyseven percent of the ARC funds spent on the 197 projects that responded to the survey (approximately 

$20.8 million for 100 projects) were invested in transitional counties and 50 percent (approximately 

$28.2 million for 69 projects) were invested in distressed counties. The full breakdown of number of 

projects and funding by county status can be seen in the following two figures. 

0

5

10

15

20

25

30

AL GA KY MD MS NC NY OH PA SC TN VA WV

Access Road Airport Gas Line

Highway Industrial Site Development Intermodal Facility

Rail Sewer System Telecommunications

Water System
60 Final Report

Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

Figure 5: ARC Investments by Economic Status – Dollars of Investment, Survey Responses (n=197) 

Figure 6: ARC Investments by Economic Status – Number of Projects Invested, Survey Responses 

(n=197) 

$4,065 

$737 

$500 

$28,220 

$200 

$129 

$20,836 

$613 

$750 

 $- $5,000 $10,000 $15,000 $20,000 $25,000 $30,000

At-Risk

Competitive

Competitive/Transitional

Distressed

Distressed/At-Risk

Distressed/Transitional

Transitional

Transitional/At-Risk

Transitional/Distressed

Thousands

15

5

1

69

1

1

100

2

3

0 20 40 60 80 100 120

At-Risk

Competitive

Competitive/Transitional

Distressed

Distressed/At-Risk

Distressed/Transitional

Transitional

Transitional/At-Risk

Transitional/Distressed
Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

Final Report 61

The 100 projects located in transitional areas were across a mix of infrastructure types. Approximately 

one-quarter of projects were for access roads, sewer systems and water systems (22, 23 and 26, 

respectively). There were a total of 69 projects funded in distressed counties; shown in Figure 6, 85 

percent of these (a total of 59 projects) were for water or sewer systems; specifically, 35 water systems 

and 24 sewer systems. 

Economic Development Impacts 

Another aspect of the evaluation was to look at the quantitative impacts of the project funding. These 

measures examine real, tangible economic development impacts, such as jobs created/retained, 

businesses created/retained by state and infrastructure type. The limited set of completed surveys leaves 

some investment categories and states without sufficient impact data to adequately analyze the effects of 

ARC projects. In addition, it is important to note that not all of the respondents to the survey tracked or 

provided answers to every measure. For instance, of the 197 survey responses received, only 84 (43 

percent) provided a value for jobs created, with 38 of those values being zero and only 54 (27 percent) 

provided a value for jobs retained, with 22 of those being zero. 

Based on the responses for which sufficient data were available (i.e., 34 out of the 92 “closed” projects), 

ARC infrastructure investments created or retained a total of more than 8,000 jobs as indicated by the 

respondents. This total is driven by a few large projects in a few states. For example, South Carolina 

projects resulted in the largest number of jobs created/retained among the ARC states, 4,331. Of this, 

3,000 jobs were created from a single project, the Greer Water Treatment Plant in Greenville. Tennessee, 

Tennessee and Maryland projects also generated a relatively high number of jobs, as shown in Figure 7. 

Most of these jobs are due to the 1,600 jobs retained by the Hawkins County Water Storage Tank project 

in Tennessee, the 850 jobs retained by the Donaldson Industrial Hangar Upgrades in South Carolina, and 

the 450 jobs created by the Barton Business Park project in Maryland. 

As illustrated in Figure 8 below, the majority of jobs created or retained were attributable to water 

systems and industrial site development projects. These two categories of projects account for nearly 87 

percent of all jobs created or retained. Access road and sewer projects also supported the creation or 

retention of more than 1,000 jobs. According to survey respondents, the majority of these jobs, 

approximately 55 percent, are in manufacturing. 
62 Final Report

Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

Figure 7: Number of Jobs Created or Retained by State, Survey Responses Closed Projects (n=34) 

Figure 8: Number of Jobs Created or Retained by Self-Identified Infrastructure Type, Survey 

Responses Closed Projects (n=34) 

- 

160 

672 

700 

2 

190 

7 

101 

65 

4,331 

1,890 

- 

10 

 - 1,000 2,000 3,000 4,000 5,000

AL

GA

KY

MD

MS

NC

NY

OH

PA

SC

TN

VA

WV

700 

65 

- 

1,045 

- 

- 

304 

9 

6,005 

 - 1,000 2,000 3,000 4,000 5,000 6,000 7,000

Access Road

Airport

Highway

Industrial Site Development

Intermodal Facility

Rail

Sewer System

Telecommunications

Water System
Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

Final Report 63

Approximately 39 percent of jobs created or retained were unable to be classified to a specific industry. 

The largest number of jobs created or retained was in manufacturing industries – approximately 4,500 

jobs or more than 55 percent. Aside from “other,” services then accounted for the next largest share with 

2.1 percent, or 170 jobs created or retained, as shown in Figure 9 below. 

Figure 9: Jobs Created or Retained by Industry,7 Survey Responses Closed Projects (n=30) 

 

7 Note that these are only for closed projects, and “other” may include some jobs in categories shown in the graphic 

when respondents were uncertain of the industry classification. 

0.4% 0.0% 0.7%

55.3%

0.0%

2.1%

0.8%

0.4%

39.4%

1.0% Agriculture

Mining

Construction

Manufacturing

Transportation Utilities

Services

Retail

Finance, Insurance or Real Estate

Other

Wholesale
64 Final Report

Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

Examining only the 92 projects that are “closed” from the survey responses, 30 of which provided a 

numerical response for businesses, funding from ARC between 2004 and 2010 contributed to the 

creation or retention of 74 business establishments, as shown in Figure 10 below. The ARC investments 

contributed to the largest number of establishments in Georgia with a total of 23 new or retained 

businesses. This is due to the creation of four businesses and the retention of 19 business sites because of 

the Jefferson Waste Water System Improvements. The Hawkins County Water Storage Tank in 

Tennessee created three businesses and allowed for the retention of 14 more during the same time frame. 

Figure 10: Number of Businesses Created or Retained by State, Survey Responses Closed Projects 

(n=30) 

- 

23 

12 

1 

- 

10 

6 

1 

2 

1 

18 

- 

- 

 - 5 10 15 20 25

AL

GA

KY

MD

MS

NC

NY

OH

PA

SC

TN

VA

WV
Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

Final Report 65

The business establishment outcomes were primarily the result of sewer and water system 

improvements, with 36 new or retained businesses due to water system improvements and 27 due to 

sewer improvements, as shown in Figure 11. As noted previously, the Jefferson Waste Water System 

Improvements in Georgia created or retained a large number of businesses due to a water system 

investment, four and 19, respectively. The largest business creation or retention associated with a water 

system project was the Hawkins County Water Storage tank in Tennessee, which created three new 

establishments and retained an additional 14. 

Figure 11: Number of Businesses Created/Retained by Self-Identified Infrastructure Type, Survey 

Responses Closed Projects (n=30) 

In addition to impacts on jobs and businesses, the infrastructure investments also contributed to 

improvements to households. Based on survey responses, 51 of the closed ARC infrastructure 

investments, those which provided a numerical response, improved more than 57,000 households. The 

largest shares of those improvements were in Kentucky and Tennessee, with more than 25,000 and 

23,000 improved households, respectively. These impressive numbers are primarily due to improvements 

to water systems. The Hawkins County Water Storage Tank in Tennessee has improved more than 

20,000 households, and the Wood Creek Water Treatment Plant Expansion in Kentucky serves more 

than 16,000 households. 

The ARC contributed nearly $25 million in funding to closed projects that totaled $131.8 million from 

2004 to 2010. According to the 42 survey respondents who provided a numerical response, these projects 

leveraged an additional $1.2 billion in private investments. Detail related to both the funding sources for 

project completion and the additional leveraged investments was not available from the survey 

responses. 

1 

2 

- 

3 

- 

- 

27 

5 

36 

 - 5 10 15 20 25 30 35 40

Access Road

Airport

Highway

Industrial Site Development

Intermodal Facility

Rail

Sewer System

Telecommunications

Water System
66 Final Report

Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

Anticipated vs. Actual Impacts 

When applying for ARC funding, the prospective grantees are asked to estimate the impacts of their 

projects. An additional assessment of the success of projects is to examine the actual outcomes compared 

to the perceived outcomes. This comparison was made for only the closed projects that had numerical 

values for both ARC predicted outcomes and survey responses, the number of which varied by question 

and is indicated in the third column of the table below. The results indicate that actual impacts for 

households served and jobs created exceed the anticipated outcomes. Two of the metrics fell short of 

reaching anticipated outcomes – job retention and businesses served. Job retention was about 1,000 jobs 

short of projections with grantees predicting that there would be 2,354 jobs retained and the actual 

results for these projects were 1,379 jobs retained. Grantees predicted that there would be 1,029 

businesses served but the actual results showed only 714 businesses served for the 21 cases that provided 

responses. It is important to note that the sample sizes are limited to only those projects that provided 

numerical responses in both their grant application and in their survey response, which reduces the 

sample size and may not reflect all outcomes. 

Table 7: Comparison of Predicted and Actual Household Served and Jobs Created and Retained 

Predicted Actual n

Households Served 13,148 27,488 37

Job Creation 4,181 5,051 21

Job Retention 2,354 1,379 8

As seen in Table 7, respondents anticipated serving approximately 13,000 households and actually served 

more than 27,000. This is more than double the amount initially anticipated. More information would be 

required to determine the exact explanation for why actual outcomes were greater than predicted 

outcomes, but some possible explanations include greater impacts due to a combination of this grant 

with other related investments, missing responses, or a misunderstanding of the question. Job creation 

saw better than anticipated outcomes among these projects, 5,051 compared to 4,181. 

Perceived Project Outcomes 

In addition to questions seeking quantitative data, the evaluation survey also contained questions to 

qualitatively assess the long-range project outcomes. These measures describe how grant recipients 

perceive the long-range impacts of the projects on the economy, the environment, and the quality of life 

of the communities. Grantees were asked a series of questions related to the project and asked to respond 

according to the following scale: 

 None: Project had little to no impact on trend 

 Slight: Project impact was not large enough to reverse or stabilize trend 

 Moderate: Project impact contributed to the stabilization or reversal of trend 

 High: Project impact was responsible for significant improvement in trend 

Questions were asked regarding economic impacts, competitiveness impacts, and environmental 

impacts. Of the 197 survey responses, 157 (80 percent) provided answers to the questions related to 

project outcomes. 
Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

Final Report 67

Ratings were sought to the following statements about long-term economic impacts of ARC projects: 

1. Attracting new residents or stabilizing the area’s population

2. Attracting new jobs or increasing employment at existing businesses 

3. Creating new sources of income or increasing income for local residents 

4. Increasing local business sales or the value of business assets (such as equipment, real estate) 

5. Increasing the value of household assets (such as homes, land, farms) for local residents 

6. Increasing the value of community assets (such as community buildings, schools, infrastructure, 

parks) 

The range of success on these projects varied by question, as shown in Figure 12, though those who 

responded viewed their project as having the most success at increasing the value of community assets –

26 percent believed there was a moderate improvement and 35 percent a high improvement. Only 20 

percent of respondents saw no improvement and 19 percent believed there to be a slight improvement in 

the value of community assets. 

Respondents also found the projects to have high or moderate success at both attracting/stabilizing the 

population and increasing employment. Respondents found the projects least successful at creating new 

sources of or increasing income for local residents; 38 percent believed that there was no impact while 

only 16 percent believed that there was a high impact. Overall, the ARC projects were viewed by the 

grantees as able to reverse or stabilize an economic trend approximately half of the time. 

Figure 12: Qualitative Survey Responses Regarding Economic Impacts (n=197) 

0

10

20

30

40

50

60

Attracting New

Residents

Employment

Growth

Increasing

Income

Increasing Sales

or Value of

Business Assets

Increasing the

Value of

Household Assets

Increasing the

value of

Community

Assets

High Moderate Slight None No Response
68 Final Report

Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

The second set of qualitative questions sought long-term outcomes related to the following 

competitiveness measures: 

1. Improving the stability and sustainability of the local economy 

2. Improving the efficiency of business operations or public services 

3. Improving the productivity of students, employees, businesses, land, or other assets 

4. Improving the skill level of the workforce 

5. Increasing the viability of local businesses 

6. Improving access to markets for local products, businesses, artisans, and entrepreneurs 

7. Reducing the cost of doing business 

The findings of these questions are shown in Figure 13. Of the measures, respondents found the projects 

to be the most successful at improving the stability and sustainability of the local economy. Eighty-five 

percent indicated that the impact ranged from slight to high and only 15 percent of respondents believed

that there was no impact. With respect to improvements in the efficiency of operations, 35 percent of 

respondents believed that their project had a high impact, 22 percent moderate, and only 19 percent did 

not see any impact. The respondents found their projects to be least successful at improving work-force 

skill levels and access to new markets. Fifty-five percent indicated there was no impact. This finding is 

not unsurprising as the infrastructure investments evaluated as part of this study do not typically impact 

worker skills. 

Figure 13: Qualitative Survey Responses Regarding Competitiveness Measures (n=197) 

The final category of qualitative impacts related to the environmental outcomes of the investments. Of 

the economic, competitiveness and environmental impacts, grantees found their projects to be the least 

successful at improving environmental outcomes, as shown in Figure 14. The following are the topics 

posed as questions that were presented to the grantees: 

0

10

20

30

40

50

60

70

80

90

100

Stability and

Sustainability of

the Local Economy

Efficiency of

Business

Operations/Public

Services

Productivity of

Students,

Employees,

Businesses, Land

Skill Level of the

Workforce

Viability of Local

Businesses

Access to New

Markets

High Moderate None Slight No Response
Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

Final Report 69

1. Impact on improving air or water quality 

2. Impact on improving quality and/or access to land and natural resources 

3. Impact on waste reduction or improving waste management (reuse, recycling, alternative energy, 

biofuels) 

4. Impact on improving energy security and independence 

5. Impact on improving energy efficiency or conservation 

Of the environmental impact measures, the greatest success was related to improvements in air and 

water quality; 43 percent of respondents believed that their project had a high impact while 41 percent 

believed there was no impact. From the grantees’ perspectives, the projects also had mild success –

approximately 48 percent reporting at least a slight improvement – at improving the quality of or access 

to land and natural resources. At least two-thirds of respondents found no impacts at all in regards to 

improvements of waste reduction, energy security or energy efficiency. 

Figure 14: Qualitative Survey Responses Regarding Environmental Measures (n=197) 

5.2.2 Econometric Analysis 

This section of the report provides the methodology and results of econometric analyses conducted on 

both the full ARCnet database for 2004 to 2010 and the data collected through the online survey of grant 

recipients previously described. Some background on the tools utilized to conduct the analyses is 

provided, as well as the econometric approach employed and the results of the two analyses. ARC’s level 

of investment in projects and project results are examined, and an assessment of the extent to which 

project direct outcomes can be used to help predict results is provided. Specifically, the findings are 

designed to: 

0

20

40

60

80

100

120

 Air or Water Quality Quality and/or

Access to Land and

Natural Resources

Waste Reduction or

Improving Waste

Management

Improving Energy

Security and

Independence

Improving Energy

Efficiency or

Conservation

High Moderate None Slight No Response
70 Final Report

Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

1. Measure intermediate project outcomes (e.g., miles of road, linear feet of water and sewer 

systems, or numbers of businesses served) and establish the typical range of program 

outcome/cost ratios; 

2. Measure final economic development results (i.e., jobs created) and establish the typical 

range of economic result/cost ratios; 

3. Develop models that predict project outcome/cost ratios in terms of how they vary by type 

of project and characteristics of the setting (including its population, density and economic 

cost structure); and 

4. Develop models that predict economic development result/cost ratios in terms of how they 

relate to project type, intermediate project outcomes and characteristics of the setting 

(including economic profile). 

The third and fourth items also enable further use of study findings for grant review and program benefit 

evaluation. The steps in this process are indicated below. 

Those latter uses are enabled by the use of ARC-LEAP, which is a software system supported by ARC 

that was initially designed to enable the Commission and its Local Development Districts (LDDs) to 

assess their region’s economic performance and growth opportunities. It was later redesigned and 

updated to enable ARC to evaluate grant funding proposals, by helping applicants define realistic 

expectations for program outcomes and economic development results. The initial prototype of this 

program evaluation function was developed in 2011 for broadband and telecom-related grants, and later 

updated in 2012 to cover education grants. 

Databases 

The first step in the methodology was to collect information regarding types8 of recent infrastructure 

investment projects and their outcomes. To accomplish this, two datasets were analyzed – the full 

universe of projects contained in the ARCnet database, 811 cases, and a subset of 197 of these cases for 

which survey responses were received. Not all of these cases contained sufficient information for analysis, 

and the limitations and sample size reductions will be discussed. Analysis of both datasets allows for a 

comparison of job impacts predicted by grant recipients at the time of application in the ARCnet 

database and the actual job impacts reported by those that responded to the survey. Because the various 

project types have unique characteristics that can lead to very different outcomes, with the exception of 

water and sewer, the types were not aggregated. 

 

8

 For statistical analysis, the project types considered were based on the ARC classified type. In some cases, these 

varied from the self-identified type indicated by the survey respondent, which were used in the previous section.
Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

Final Report 71

The analysis showed that in both datasets, less than half of the projects provided information on job 

impacts and only three of the eleven project types had sufficient cases with job data to enable statistical 

analysis.9 Table 8 shows the number of each of these types of projects and the number with job data for 

both the full database and the survey results. The top portion of the table indicates types that did not 

have sufficient cases for statistical analysis, and the bottom half shows the three types that do contain 

enough information to analyze. 

Table 8: Number of Projects and Projects with Jobs Data by Type for Full ARCnet and Survey 

Responses 

ARCnet Database (Predicted Jobs) Survey Results (Actual Jobs)

Project Type Total 

Projects

Projects with Job 

Data

Total 

Projects

Projects with Job 

Data

Project Types with Insufficient Cases to Analyze

Airport 1 1 0 0

Business Site Development 1 1 0 0

Gas Line 4 3 1 0

Highway 1 0 1 0

Intermodal 3 1 1 0

Rail 3 0 0 0

Telecommunications 65 5 19 1

Water Tank 34 9 0 0

Subtotal 112 20 22 1

Project Types with Sufficient Cases for Statistical Analysis

Access Road 116 76 32 15

Industrial Site Development 86 61 18 9

Water/Sewer 497 161 125 47

Subtotal 699 298 175 71

Total 811 309 197 72

Table 8 shows that after filtering both datasets for: 

 Project types deemed too underrepresented for statistical analysis: 

 Access road, water/sewer, and industrial site development projects with no job impact data; and 

 Outlying projects that had job impact or spending data so extreme as to skew the analysis and 

decrease predictive power (see Appendix E for details on the outlier detection and removal 

process, as well as a replication of the regression analysis in Table 12 including outliers), 

 

9

 In order to expand the number of observations, water system and sewer system projects were combined to create 

a single “water/sewer” category as they have similar outcomes. 
72 Final Report

Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

there were 298 cases from the ARCnet database and 71 cases in the survey response data with actual job 

outcomes. Water/sewer projects were the most represented in both databases while industrial site 

development was the least represented. As mentioned previously, projects in the ARCnet database 

contain information on predicted job creation provided by ARC grant applicants, while projects from 

the survey responses provided information on actual job creation. 

Comparison of Predicted Job Impacts to Actual Job Impacts 

Before proceeding with the regression analysis, a comparison of job creation predictions to actual job 

creation after project implementation was done for the 31 grant recipients that provided job data in both 

ARCnet and the survey, shown in Table 9. This provides a glimpse into the accuracy of predictions made 

by recipients before projects were undertaken as compared to the actual results of the projects, in terms 

of outcomes.

Table 9: Comparison of Predicted and Actual Job Creation 

Predicted Job Creation Actual Job Creation

Project Type Min Max Average Median Min Max Average Median

Access Road 50 500 170.4 115 0 1,000 174.6 45

Industrial Site Development 50 1,200 248.1 100 0 500 120.3 70

Water/Sewer 6 530 101.0 45 0 3,000 324.4 55

N 31 31

The results for access roads, in Table 9, show that while some grant recipients reported job creation 

estimates that were less than their predictions when they applied for the grant (as reflected by the 

decreased minimum, maximum, and median actual job creation values), an upward revision by one or 

more recipients caused the average value to increase slightly. Among industrial site projects, lower 

minimum and maximum values, a lower average, and a lower median value also reflect a general pattern 

of downward revision, indicating overestimation of job creation at the time of grant receipt. Unlike the 

other categories, water/sewer projects in this sample underestimated the likely impacts at the time of 

grant application. While at least one recipient did revise their job creation value down to zero in the 

survey, an overall upward revision is indicated by a higher maximum, average, and median actual job 

creation value. 

In addition to comparing predicted job impacts to actual job impacts, the average rate of job creation per 

million dollars in total project spending, including funds from all sources, was also calculated. This 

analysis was expanded to include all projects with job data for these three project types from Table 8 in 

order to assess program-level effectiveness, or the impact of total spending, rather than the accuracy of 

survey respondent predictions, as analyzed in Table 9. While the regression analysis in the next section 

isolates the effect of total spending while controlling for other variables, this analysis demonstrates how 

job creation spending rates vary by project type, even when actual job creation values are considered. 

Access road projects create the most jobs per million dollars in total spending, as shown in Table 10, 

while water/sewer projects create the fewest. These ratios fall for each project type in terms of actual job 

creation, reflecting the earlier finding that a sample of recipients may have overestimated their job 

creation at the time of grant request. 
Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

Final Report 73

Table 10: Average Job Creation by Project Type for Predicted and Actual Jobs 

Jobs per Total Spending ($1 Million)

Project Type Predicted Job Creation Actual Job Creation

Access Road 280 275

Industrial Site Development 121 66

Water/Sewer 84 48

N 298 71

The share of ARC spending that contributed to total project spending is also important to consider. 

While it is difficult to ascertain which jobs ARC spending was responsible for creating, especially in the 

case of projects with multiple sources of funding, ARC’s share of support does vary quite considerably 

across project types. When considering the 298 projects in the ARCnet database, ARC was, on average, 

responsible for five percent of access road spending, 27 percent of industrial site development spending, 

and 18 percent of water/sewer spending. These disparities indicate that access road projects are reliant 

on diverse funding sources while industrial site development and water/sewer projects are reliant on 

ARC spending.

Statistical Analysis 

The second step in the methodology was to use statistical analysis, in the form of linear regressions,10 to assess the 

assess the extent to which variation in predicted and actual job creation could be explained by differences in project 

differences in project spending levels and geographic settings – including local economic distress and local 

local population density – with the latter effectively representing rural/urban differences. The analysis seeks to 

seeks to explain variation in both predicted and actual job creation because the survey responses contain too few 

too few projects to generate statistically significant results, which are shown in 

Table 13. An analysis and interpretation of the larger dataset of predicted values is shown in 

Table 12. Though, on average, actual job creation values are lower than predicted values due to the 

overestimation previously described, the analysis of predicted job values is justified on the grounds that: 

1. A larger dataset generates results in which more confidence can be placed; 

2. Project-level spending does not change across the ARCnet and survey cases; and 

3. Project-level geographic settings – economic distress and density – do not change across the 

ARCnet and survey cases. 

Because the size and settings of projects do not change across the two datasets, the following analysis 

provides great value in its contribution to the understanding of how changes in project type, spending, 

and geographic setting affect job creation outcomes. The choice of explanatory variables to include in 

this analysis, as shown in Table 11, was influenced by economic development theory as well as a practical 

understanding of project-level job creation dynamics. 

Table 11: Explanatory Variables Tested in Regression Analysis 

Explanatory Variable Description Source

 

10 Various functional forms were tested before proceeding with a linear model. These included double-log and 

semilog functional forms, as well as models that did not suppress the constant term. 
74 Final Report

Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

Total Spending Total project spending in constant 2011 dollars ARCnet

Population Density Persons per square mile ARCnet

Density x Total Spending Interaction of population density and total spending Calculated by EDR Group

Economic Distress ARC-generated county economic distress index11 ARCnet

Distress x Total Spending Interaction of economic distress and total spending Calculated by EDR Group

It is hypothesized that spending on a project is directly and positively related to job creation outcomes; 

through an infusion of outside (or exogenous) funding, project managers are able to hire construction 

workers, for example. While not directly related to hiring made possible by a new project, population 

density and economic distress in a project’s surrounding county are also hypothesized to affect rates of 

job creation and the impact of spending by ARC and other funders. 

It is expected that population density will exhibit a positive relationship with project-level job creation. 

Even when controlling for spending, projects in more urban counties, defined as those with higher 

population density, have access to a larger, more mobile labor force and the ability to support job creation 

in other industries. The interaction of density and spending, indicated as population density multiplied 

by total spending, may have the opposite effect on job creation given the difficulty of construction in 

more urban settings as compared with rural settings. Delays and extra precautions associated with 

congestion and interference with public spaces in urban areas may result in less efficient use of funds. 

Economic distress in a project’s surrounding county, as defined in Footnote 11, is hypothesized to have a 

positive relationship with job creation. This is anticipated because counties with higher levels of 

unemployment, income, and poverty are more likely to have lower wages caused by either an oversupply 

of or lack of demand for workers, a process referred to as “wage determination.” With lower prevailing 

wages in a county, projects may hire more workers, even when controlling for spending and other factors. 

In the case of predicted job creation, this relationship may also materialize out of a bias on the part of 

grant recipients in more distressed counties to overestimate job creation. The interaction between 

distress and spending, indicated as economic distress multiplied by total spending, is also expected to 

have a positive relationship with job creation, as project funds may be allocated more effectively toward 

job creation given lower prevailing wage rates in distressed counties. 

 

11 The Appalachian Regional Commission (ARC) uses an index-based county economic classification system to 

identify and monitor the economic status of Appalachian counties. The system involves the creation of a national 

index of county economic status through a comparison of each county’s averages for three economic indicators—

three-year average unemployment rate, per capita market income, and poverty rate—with national averages. The 

resulting values are summed and averaged to create a composite index value for each county. Each county in the 

nation is then ranked, based on its composite index value, with higher values indicating higher levels of distress. 
Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

Final Report 75

The regression equation representing the series of hypothesized relationships is 

 ( ) ( ) ( )

 ( ) ( )

where 

 

This equation specifies how job creation is a function of project type, spending level, population density, 

and economic distress. Interaction terms are included to reflect how the spending effect differs as density 

and distress vary. Finally, the regression has no constant term as jobs associated with a project cannot be 

created in the absence of project spending and associated efforts. 

The analysis of variations in predicted job creation shows that total spending, population density, and 

economic distress all affect job impacts. Full results, presented in terms of linear regression coefficients 

for factors affecting the predicted number of jobs created by project type, are summarized in Error! 

Reference source not found.Table 12. This analysis shows that predicted job creation, as hypothesized, 

rises with project expenditures: it is estimated that an additional $1 million in project spending results in 

151 access road jobs, 52 water/sewer jobs, and 129 industrial site development jobs. 

Table 12: Regression Results: Predicted Job Creation (ARCnet Database) 

Explanatory Variable

Coefficient by Project Type

Access Road Industrial Site Water/Sewer

Total Spending ($1 Million) 151.4** 129.1*** 51.9**

Density (10 persons/square mile) 6.02** 1.43 0.51

Density*Total Spending ($1 Million) -.595* -0.128 0.049**

Distress (10 units) 4.65* 3.36*** 5.50***

Distress*Total Spending ($1 Million) -2.04 -0.577*** -.0.310**

Percent Explained R

2 

= 0.515 R

2 

= 0.657 R

2 

= 0.486

Model Significance (F-score) *** *** ***

N 76 61 161

Note: Linear regression through the origin: Dependent (predicted) variable is “jobs created.” The constant term was suppressed 

so that there would be no jobs predicted if no spending occurred. 

*Significant at 90% confidence level; **Significant at 95% confidence level; ***Significant at 99% confidence level. 

Population density also has a positive relationship with access road job creation, affirming the hypothesis 

and suggesting that 10 additional persons per square mile in a county is associated with approximately 

six jobs. The interaction between population density and total spending has an effect on access road and 

water/sewer job creation, but the impact differs by project type. The negative relationship between this 

interaction term and job creation for access road projects affirms the hypothesis that spending on access 

road projects is less effective in more urban counties where congestion may hinder construction. For 

water/sewer projects, however, the results suggest that spending becomes more effective at creating jobs 

as population density increases. 

Economic distress is estimated to have a positive relationship with job creation for all three project types. 

A 10-unit increase in a county’s ARC distress index (see Footnote 11 for the definition) is associated with 

close to five access road jobs, six water/sewer jobs, and just over three industrial site development jobs. 
76 Final Report

Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

While the impact of economic distress on job creation is not nearly as strong as that of total spending, 

the influence of this variable is still significant and in-line with the earlier hypothesis. The interaction 

between economic distress and total spending is related only to water/sewer and industrial site 

development job creation. The influence of this variable occurs in the opposite direction of what was 

hypothesized; while higher economic distress alone is associated with greater job creation, when 

controlling for spending, spending actually becomes less effective at creating jobs among projects located 

in more distressed counties. 

Across the regression estimates for project type – access road, water/sewer, and industrial site – the 

explanatory variables explained between 49 and 66 percent of the variance in predicted job creation. Due 

to the low number of projects containing job impact data in the survey results as shown in Table 8, 

however, the regression estimates provide little insight into the variations in actual job creation among 

access road and water/sewer projects. The data provide no insight into industrial site development job 

creation, as data were especially sparse for this project type. These results can be seen in Table 13. 

Table 13: Regression Results: Actual Job Creation (Survey Results) 

Explanatory Variable Coefficient by Project Type

Access Road Industrial Site Water/Sewer

Total Spending ($1 Million) 561.5* N/A 276.2

Density (10 persons/square mile) 1.09 N/A -16.0

Density x Total Spending ($1 Million) 0.480 N/A 1.75

Distress (10 units) 1.65 N/A 0.11

Distress x Total Spending ($1 Million) -4.02 N/A -1.19

Percent Explained R

2 

= 0.933 R

2 

= 0.264

Model Significance (F-score) ** -- **

N 15 9 47

Note: Linear regression through the origin: Dependent (predicted) variable is “jobs created.” The constant term was suppressed

so that there would be no jobs predicted if no spending occurred. 

*Significant at 90% confidence level; **Significant at 95% confidence level; na = not available due to insufficient sample size.

Role of Intermediate Outcomes 

The third step in the methodology was to collect information from the survey data regarding the most 

commonly available measures of intermediate program outcomes that occur apart from job creation. The 

goal of recording these intermediate measures is to draw an association between project-level 

expenditures and physical outcomes, acknowledging the fact that not all project types are expected to 

create jobs. Of the survey responses, only the access road and water/sewer project types had enough 

cases with both total spending and intermediate outcome information to analyze. A total of 16 access road 

projects provided both spending and intermediate outcome data, while 63 water/sewer projects provided 

both pieces of information.12

Among access road projects, the minimum length of road constructed was 0.05 miles, while the 

maximum was three miles, with an overall average of 0.62 miles. The length of piping developed by

water/sewer projects also varied greatly; the minimum length developed was 185 feet, while the 

maximum was 195,739 feet, with an average of 34,858 feet. Though industrial site development project 

 

12 Note that the number of projects with both spending and outcome data is different than the total number of 

projects that provided actual job creation information in Error! Reference source not found.. 
Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

Final Report 77

outcomes were not included in the regression analysis, they did exist; the minimum number of businesses 

served was one, while the maximum was 24. On average, these projects served nine businesses each. 

The regression results suggest that total spending has a positive relationship with intermediate outcomes 

for both access roads and water/sewer projects. As shown in Table 14, an additional $1 million in access 

road spending results in 0.333 miles of road. The same investment is also estimated to result in 

approximately 11,000 linear feet of pipe infrastructure if spent on water/sewer projects. For this sample of 

available data, the regression estimates explained between 27 and 53 percent of the variation in 

intermediate outcomes. 

Table 14: Regression Results: Intermediate Outcomes as a Function of Total Spending ($1 Million) 

(Survey Results) 

Project 

Type Intermediate Outcome Coefficient

Percent 

Explained 

(R

2

)

Model 

Significance 

(F-score)

Cases with Both 

Spending & 

Outcome Data (N)

Access Road Miles of Road .333** .526 ** 16

Water/Sewer Linear Feet 10,938** .272 ** 63

Note: Linear regression through the origin: Dependent (predicted) variable is intermediate outcome. The constant term was suppressed so that 

there would be no intermediate outcome predicted if no expenditure occurred. 

**Significant at 95% confidence level. 

A similar analysis was performed to explore the relationship between intermediate outcomes and actual 

job creation. Industrial site development projects were again excluded from this analysis due to the low 

number of cases, but 15 access road projects provided both intermediate outcome and job creation data 

and 47 water/sewer projects provided both pieces of information.13 The results in Table 15 below suggest 

that the intermediate outcomes associated with access road and water/sewer projects are positively and 

significantly related to actual job creation. The estimates suggest that an additional mile of access road 

developed is associated with 144 jobs, and an additional 10,000 linear feet of water/sewer piping is 

associated with 10 jobs. The regression estimates explain between 11 and 32 percent of the variation in 

job creation. 

Table 15: Regression Results: Actual Jobs Created as a Function of Intermediate Outcome (Survey 

Results) 

Project 

Type

Intermediate 

Outcome Coefficient Percent Explained (R

2

)

Model 

Significance 

(F-score)

Cases with Both 

Job Creation & 

Outcome Data (N)

Access Road Miles of Road 144.1** .324 ** 15

Water/Sewer Linear Feet

(10,000) 10** .111 47

Note: Linear regression through the origin: Dependent (predicted) variable is “jobs created.” The constant term was suppressed so that there 

would be no intermediate outcome predicted if no expenditure occurred. 

** Significant at 95% confidence level. 

 

13 Again, note that the number of projects with both job creation and outcome data is different than the total 

number of projects that provided actual job creation information in Table 9 and differs from the data in Table 14. 
78 Final Report

Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

Systematic Underestimation of Job Creation 

To explore in greater detail how well the regression equations were able to estimate predicted and actual 

job creation beyond a measure of percent explained (R2), an analysis of residuals was performed. 

Residuals represent the difference between job creation values observed in both the full ARCnet and 

survey datasets and the values estimated by the regression equations. The details of this analysis can be 

found in Appendix F. The results indicate that while the regression-estimated job creation values are 

very close to those observed values for the majority of projects, there was a small group for which the 

regressions consistently under-estimated job creation values either predicted in ARCnet or confirmed as 

actual in the survey. 

This finding suggests that some projects may have non-observed factors causing them to generate 

significantly greater-than-expected job creation. The most logical explanation for this, which is 

supported by some case studies, is that certain ARC-funded infrastructure projects were bundled with 

other non-infrastructure economic development actions to create a comprehensive package. Any such 

non-infrastructure actions, such as other grants, tax incentives, or loans, would not be captured by the 

inputs into the ARCnet database or the survey questions, yet they would tend to increase job creation 

beyond what would otherwise be expected from the project spending alone. This bundling can involve 

multiple grantors at multiple levels of government, each collecting grantee data separately. In the absence 

of coordination, outcomes reported by projects receiving significant amounts of support from multiple 

agencies could thus appear greater than expected. Given this finding, it may be a useful exercise to ask 

grantees about other sources of funding or related projects in the future. 

5.3 Performance Measure Conclusions 

Overall, the bulk of ARC investment was in water and sewer infrastructure projects. While ARC 

sponsored many projects in both Kentucky and Mississippi, as well as projects in other ARC states, the 

largest share of investment was spent in West Virginia. Throughout the region, most of the projects were 

in transitional counties, but the largest share of funding was spent in distressed counties. 

Based on the survey results, water projects generated the most benefits in terms of jobs created or 

retained, businesses served, and households served. Additionally, projects in Kentucky seemed to have 

the highest benefit in terms of households and businesses improved. 

Based solely on the survey responses, ARC funding of water system, industrial site development, sewer 

system and access road projects offered the highest return on investment. Of the studied cases, water 

systems projects have created or retained 6,005 jobs and 36 businesses. Industrial site development 

created or retained the second-highest number of jobs with 1,045 and three businesses. Sewer systems 

created or retained fewer jobs, 304, but the second highest number of businesses, 27. Access road projects 

created or retained 700 jobs and one business. 

It is worth noting that predictions or anticipated outcomes of grant recipients were often lower than 

actual outcomes. For example, survey respondents predicted that they would serve approximately 13,000 

households but actually served nearly 27,500 – more than double the anticipated amount for the 37 cases 

with responses. Job creation was greater than anticipated, with 5,051 jobs compared to 4,181 predicted 

for the 21 cases with responses. Job retention, however, fell short of predictions only keeping 1,370 jobs 

rather than the 2,354 jobs predicted among the eight cases with sufficient information. For many projects 

and outcomes, projects were predicted to be less successful as job and business generators than they 

actually were. 
Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

Final Report 79

The funding contributed to attracting new businesses and jobs to the communities as well as improving 

the efficiency of business operations. ARC funds accounted for 20 percent of total project costs for those 

projects that completed surveys. According to the grantees, ARC funding had a significant effect on the 

outcome of the projects. Without this funding, 35 percent of the projects for which responses were 

gathered would not have happened. 
Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

80 Final Report

6 CASE STUDIES OF BEST PRACTICES FOR INFRASTRUCTURE 

The team conducted case studies of 13 projects, one in each state in the region. The case studies were 

intended to be exemplary in character in order to provide ARC and communities considering similar 

projects with insights into the ingredients of successful projects. Exemplary projects were identified 

through review of survey data and discussions with ARC program staff and state program managers. 

ARC defines exemplary projects as follows: 

 Met or exceeded outcome projections 

 Unexpected outcomes 

 Consistency with ARC objectives and strategies 

 Projects that achieved multiple objectives/strategies 

 Particularly strong local commitment (e.g., financial contribution; participation of key actors) 

The case studies focused on project planning and implementation, challenges encountered and how they 

were addressed, economic and community impacts, and “lessons learned” for communities undertaking 

similar projects. Information was obtained through review of ARC documents and interviews with key 

project participants, beneficiaries, and observers. Three of the case studies were conducted on-site and 

the remaining ten through telephone interviews. In addition to geographic diversity, the case studies 

were selected to obtain a mix of project types. The following sections contain the detailed reports from 

each case study. Basic information on each of the case study projects is presented in the following table. 

Table 16: Case Study Summary 

Grantee Project Title Project Description Project Category & 

Type

ARC 

Funding

Total 

Cost

Primary 

Outcomes

Allegany County, 

MD 

Barton Business 

Park Sewage 

Treatment Plant

Construction of a 

sewage treatment 

plant for 150-acre 

Business Park. 

Targeted users 

include advanced 

manufacturing and 

defense-related 

production 

facilitates

Business 

Development/ 

Sewer System

$371,000 $1.94 

million

364 jobs 

created; $12 

million 

Leveraged 

Private 

Investment 

(LPI)

Dickenson County, 

VA

Big Caney Water Phase 2 of five-phase 

project to repair 

water infrastructure 

and extend water 

service to homes and 

businesses in 

unincorporated 

parts of county

Community 

Development/Water 

System 

$300,000 $3.1 

million

453 

residences 

and 10+ 

businesses 

served

Carroll Tomorrow, 

Carroll County 

(GA)

Burson Center for 

Business 

Development

Development of 

small business 

incubator providing 

affordable space for 

small, early-stage 

businesses for 

periods ranging from 

Business 

Development/ 

Business Incubation

$216,882 $1.9 

million

362 jobs 

created; $42 

million LPI 
Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

Final Report 81

Grantee Project Title Project Description Project Category & 

Type

ARC 

Funding

Total 

Cost

Primary 

Outcomes

one to five years.

Village of 

Canaseraga, NY

Canaseraga 

Wastewater 

Collection and 

Treatment 

System

Construction of new 

wastewater 

treatment system to 

replace failing 

residential septic 

systems

Community 

Development/ 

Sewer System

$150,000 $4.59 

million

215 

residences 

and 

businesses 

served; 155 

jobs 

retained

Borough of Canton, 

PA

Canton Water 

System 

Improvements

Replacement of 75-

year old water mains 

and the construction 

of water storage 

tank

Community 

Development/ 

Water System

$285,000 $3.7 

million

750 

households 

and 70 

businesses 

served; 200 

jobs 

retained

City of Clinton, TN Clinton I-75 

Industrial Park 

Sewer Line

Improvements to 

wastewater 

collection and 

transmission system 

to support 

construction of auto 

parts production 

plant

Business 

Development/ 

Sewer System

$450,000 $1.26 

million

560 jobs 

created; 

$65+ million 

LPI 

Town of Dobson, 

NC

Dobson I-77 

Infrastructure

Extension of 

municipal water and 

wastewater service 

to a site of planned 

commercial 

development at 

interstate highway 

exit

Business 

Development/ 

Water and Sewer 

Systems

$200,000 $2.2 

million

77 jobs 

created; 

$18.25 

million LPI

City of Muscle 

Shoals, AL

Muscle Shoals 

Industrial 

Infrastructure

Utilities and road 

improvements in a 

new industrial park 

to support 

construction of auto 

parts production 

facility 

Business 

Development/ 

Industrial Site 

Development

$500,000 $2.73 

million

800 jobs 

created; $60 

million LPI 

City of 

Owingsville, KY

Owingsville/Bath 

County 

Industrial Park 

Wastewater 

Treatment Plant

Installation of 

wastewater 

treatment plant to 

meet the expansion 

needs of food 

processing plant

Business 

Development/ 

Sewer System 

$500,000 $3.04 

million

222 jobs 

created and 

retained; 

$26.2 

million LPI 

SC Budget & 

Control 

Board/Powdersville 

Water District

Powdersville 

Water District 

Water Storage 

Tank

Construction of 

water storage tank

Community 

Development/ 

Water System

$500,000 $1.69 

million

410 jobs 

created 

Village of Racine, 

OH

Racine Water 

Treatment Plant 

and Storage 

Replacement of 

deteriorating water 

facilities with new 

water treatment 

Community 

Development/ 

Water System

$457,000 $1.96 

million

315 

households 

served; 55 

jobs created 
Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

82 Final Report

Grantee Project Title Project Description Project Category & 

Type

ARC 

Funding

Total 

Cost

Primary 

Outcomes

Facility plant and ground 

water storage tank

and retained 

Canaan Valley 

Institute, WV

Wastewater 

Solutions for 

Small 

Communities 

Support for 

development of 

decentralized 

wastewater treatment 

facilities in small, 

unincorporated 

communities in 

southern WV’s coal 

region through 

technical assistance 

to nonprofit 

organizations

Community 

Development/ 

Sewer System

$96,811 $176,865 77 

residences 

and 1 

business 

served; 

additional 

900 

residences 

projected 

City of Water 

Valley, MS

Water Valley 

Sewer & Water 

Improvements

Repair of 

deteriorated sewer 

lines in residential 

area and part of 

downtown, and 

extension of water 

and sewer service to 

another residential 

area

Community 

Development/ 

Sewer and Water 

Systems

$152,547 $602,547 354 

residences 

and 15 

businesses 

served; 100 

jobs 

retained

6.1 Barton Business Park Sewage Treatment Plant – Cumberland, MD 

The project involved the construction of a sewage treatment plant for the Barton Business Park in 

Cumberland, Maryland. This investment turned the industrial park into a highly attractive location for 

new industry. Targeted users include advanced manufacturing and defense-related production facilities. 

The Appalachian Regional Commission provided $371,000 in project funding, approximately 25 percent 

of total costs. The project was completed in 2006. 

6.1.1 Community Profile 

Allegany County, located in northwest Maryland, is part of the Cumberland, MD-WV Metropolitan 

Statistical Area, with a 2010 population of 75,087. The Barton Business Park is located on US 220 near 

Cresaptown and seven miles south of Interstate 68. The county has experienced minimal population 

growth in recent years, with an increase of only 0.2 percent between 2000 and 2010, compared to nine 

percent statewide and 9.7 percent nationally. 

Employment in the county has suffered over recent years. Between 2001 and 2011, employment decreased 

by 2.2 percent in contrast to statewide employment and national employment growth of two and one 

percent, respectively. The local unemployment rate has generally exceeded the state rate and matched or 

exceeded national rates during the past decade. In 2011, the county unemployment rate of 8.5 percent was 

1.5 percentage points above the state rate and .04 percentage points below the national level. 

The county’s industry mix includes a large base in education and healthcare with 8,182 jobs. Other 

significant industries include art and entertainment, manufacturing, and retail trade, employing 3,711, 

3,147, and 2,728, respectively. As compared to the five-year average for 2006-2010, 2011 industry 

employment in Allegany County shows substantial decreases in retail trade (982 jobs) and construction 
Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

Final Report 83

(552 jobs). Art and entertainment is the only industry that has seen significant job growth over this 

period with 761 jobs gained. Education and healthcare saw a minor increase over this period, an increase 

of 89 jobs. 

Allegany County is suffering economically compared to the state and the nation. Median household 

income during the 2006-2010 period averaged 53 percent of the state level and 73 percent of the national 

level. The poverty rate was 5.9 percentage points above the state level and 0.7 percentage points above 

the national level. The county was classified as “transitional” in both 2004, when the ARC grant was 

awarded, and 2012. A transitional county is defined as economically positioned between the 25 percent 

most distressed and 25 percent most robust U.S. counties. 

Allegany County’s age distribution shows a smaller adolescent (0-19) population than the state and 

national levels and a much higher elderly (65+) demographic. This indicates that in coming decades 

Alleghany County will have a smaller workforce and a larger elderly population leading to a potentially 

smaller tax base. 

6.1.2 Project Description 

The project involved the construction of a 50,000-gallon-per-day package sewage treatment plant that 

created tremendous potential for industrial development at the 150-acre business park. Funding was 

allocated in 2004, and the project was completed in 2006. The initial project cost estimate was 

$1,635,000, which was then lowered to $1,480,000 of which $237,000 was to be funded by an ARC grant, 

$1,000,000 from the state of Maryland, and $243,000 from local sources. The original bid for construction 

was $1,727,000 but final project cost was $1,954,054. The applicant later requested, and was granted, an 

additional ARC grant of $134,000 that was used to contribute to the additional project costs. The final 

breakdown of funding was $371,000 from ARC grants, $1,093,595 from the state of Maryland, and 

$489,459 from local sources. 

6.1.3 Project Planning and Implementation 

Allegany County has suffered from high unemployment in recent decades stemming from the closure of 

major industries such as PPG, Celanese, and Kelly-Springfield. The 150-acre Barton Business Park land 

lacked the necessary water, sewer, and electrical infrastructure to support new industries. This project 

gave Allegany County marketable land to attract new or expanding industries in an area where job 

creation was desperately needed. It was anticipated that industry relocation to the Barton Business Park 

could create between 800 and 1,000 jobs, potentially addressing up to 1.3 percent of local unemployment. 

The Allegany County Commissioners awarded the design of the Barton Business Park wastewater 

treatment plant to Rummel, Klepper and Kahl on July 26, 2001, in the amount of $75,764. Since the 

development of the business park would likely be by sections, the plant had to be designed to 

accommodate flows as low as 15,000 gallons per day with an ultimate capacity of 200,000 gallons per day. 

At the time of the project, Allegany County funds were, and continue to be, extremely tight due to a 

declining tax base. 

Before the project could proceed to construction, Allegany County had to retain an archaeological data 

recovery firm to mitigate two archaeological sites in the path of the proposed sewer outfall. The contract 

was awarded to R. Christopher Goodwin on March 25, 2004 in the amount of $153,168. The sewage plant 

contract was awarded to Lashley Construction Company, Inc. on April 14, 2005, and the Notice to 

Proceed was given May 31, 2005. The completion time for the project was 300 consecutive business days. 

During the construction process, there were nine change orders. Change Order 1, issued on July 13, 2005, 

involved changing doors, changing the foundation on the control building, adding both water and sewer 
Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

84 Final Report

lines, and widening the road. Change Order 2, issued on October 7, 2005, called for the installation of a 

concrete floor in the process building and constructing additional fencing along the access road per the 

Maryland Historical Trust. Change Order 5 was to install a gate at the entrance. Change Order 6 was to 

cut a new opening for air conditioning in the control building. Change Order 7 was to install a new 

exhaust fan. Change Order 8 was to set UV units in parallel rather than in series per plans. And, finally, 

Change Order 9 was for an additional 5” PVC to relocate a transformer. The water line, sewer main, road 

widening, concrete floor, and fencing led to an approved 42-day extension to the contract time. The 

project was completed in June 2006. 

According to the grant recipient, the initial bid was for $1,727,000 and was $227,054, or thirteen percent, 

over budget for a total cost of $1,954,054. The nine change orders led to a 42-day approved extension in 

the project time. There were no significant differences between the initial plans and the completed 

project. 

6.1.4 Economic and Community Impacts 

As a direct result of the project, Allegany County was able to market eight lots for new and expanding 

development. The lots range in size from 3.3 acres to 43 acres. Since completion in 2006, American 

Woodmark Corp. invested $12 million in a 250,000-square-foot manufacturing plant on the site. This 

plant currently employs 364 full-time employees earning at least $10 per hour. The project has also 

attracted other businesses to the area. Chessie Credit Union bought land adjacent to the business park in 

2008 where it plans to build a branch location. Allegany County has included the building of a 50,000-

square-foot shell building in the business park in its Capital Improvement Plan to satisfy a demand that 

has been expressed specifically in regards to the business park. 

To date, the economic impacts have not met the expected job creation figures of 800 to 1,000, though the 

potential exists if more businesses decide to locate at the business park. American Woodmark expects to 

eventually increase its workforce to 500 employees. While the project boasts job creation, it remains 

unclear whether the community views the outcomes as successful. 

6.1.5 Lessons Learned 

While it has attracted some business, the Barton Business Park project was not a resounding success. A 

few lessons can be learned from the project’s struggles.

 Forecast Potential Project Obstacles or Risks. The project planners did not anticipate that the 

proposed sewer outfall would lie on archaeological sites. It should be noted that careful investigation 

of potential obstacles/risks can avoid project delays and allow for projected costs that involve all 

mitigating factors; in this case, archeological data recovery. As a general note to future planners, 

projects on or near the Potomac River will likely encounter an archeological site. 

 Be Realistic About Job Creation Timeframe. The project anticipated the creation of 800 to 1,000 

jobs. Six years after the project’s completion, only one company has located at the business park and 

job creation sits at 364. Project planners must differentiate between the potential for site 

development and real interest in site development. A sound marketing strategy at the time of project 

completion may have helped drive industrial interest. 

6.1.6 Conclusion 

The Barton Business Park sewage treatment plant has failed to attract the level of industrial development 

anticipated at the time of project conception. The project exceeded budget and time expectations as well 
Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

Final Report 85

as encountered an unanticipated problem with archaeological sites that cost the cash-strapped Allegany 

County an additional $153,168. The project outcomes demonstrate the importance of sound project 

planning and implementation strategies.

6.2 Big Caney Water Project Phase II – Dickenson County, VA 

Phase 2 of the Big Caney Water Project was one phase of what has become a five-phase project to repair 

water infrastructure and extend water service to most of the homes, businesses, and public facilities in 

Dickenson County, Virginia. ARC provided a grant of $300,000 for the project, about 10 percent of the 

total project cost of $3.124 million. The design phase of the project began in late 2005 and construction 

was completed in late 2007. 

6.2.1 Community Profile 

Dickenson County, located in rural southwestern Virginia, had a population of 16,395 in 2010. Its largest 

town, Clintwood, had a population of 1,414. Traditionally heavily dependent on the coal industry, the 

county has seen its population decline dramatically in the past 60 years along with the drop in coal 

mining employment. Between its peak in 1950 and 2010, the county lost one-third of its total population. 

During the past decade, population has stabilized somewhat, declining by only three percent, although 

this continued downward trend compares unfavorably to statewide and national growth of 13 percent 

and 9.7 percent, respectively. 

The county has experienced fairly strong employment growth during the past decade, driven in large part 

by the increase in coal mining employment. Total wage and salary employment increased by 15.4 percent 

between 2001 and 2011, to 3,710. This compared to an increase of only 4.1 percent statewide and a decline 

of 0.2 percent nationally. In numerical terms, total employment growth was about 500, with over 400 of 

that in the coal industry. The county’s unemployment rate also declined relative to state and national 

averages. In 2011, it stood at 8.1 percent, 1.9 percentage points above the state level, but 0.8 percent below 

the national level. However, the county’s labor force participation rate, the number of workers employed 

or actively searching for employment, was very low – only 40.3 percent, compared to 67.4 percent 

statewide and 65 percent nationally. This is not the result of an aging population since the percentage of 

working age residents is only slightly below state and national averages. 

The county economy remains heavily dependent on mineral extraction, primarily coal mining. Mining, 

quarrying, and oil and gas extraction comprised 27 percent of total wage and salary employment in 2011. 

Other important industries are healthcare and social assistance, with 13 percent of total employment, and 

retail trade, with 11.2 percent. Employment in mineral extraction increased by 80.9 percent between 2001 

and 2011, while employment in healthcare and social assistance declined by 1.2 percent. In retail trade, the 

decrease was 6.5 percent. 

The economic well-being of county residents is far below state and national averages. Median household 

income averaged $29,080 during the 2006-2010 period, only 47.4 percent of the state average and 56 

percent of the national average. The poverty rate of 19.1 percent was 8.8 percentage points above the state 

average and 5.3 percentage points above the national average. 

The county was categorized by ARC as “at-risk” in 2012, an improvement over the “distressed” 

categorization in 2004, when the ARC grant was approved, reflecting its somewhat brightening 

economic picture. At-risk counties rank between the worst 10 percent and 25 percent of the nation’s 

counties, while distressed counties rank among the worst 10 percent of counties. 
Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

86 Final Report

Recognizing the risks and limitations of dependence on the energy industry, county government officials 

and local and regional economic development professionals have been working to diversify the county 

economy. They have experienced some success in developing recreational and cultural tourism and have 

attracted a few significant employers, including a large call center for the federal government retirement 

system. However, the county has faced a number of serious barriers to diversification, most notably a 

limited number of large developable sites, and limited transportation, telecommunications, and water 

and sewer infrastructure. The Big Caney water project was conceived as a comprehensive effort to 

address the deficiencies in the county’s water infrastructure.

6.2.2 Project Description 

The project was the second phase of a comprehensive multi-year, multi-million-dollar initiative to 

renovate parts of the county’s existing water system, which had seriously deteriorated since their 

construction in the 1960s, and extend water service to previously un-served areas of the county. Phase 2 

funds were used to replace 14 miles of old service lines and construct a 200,000-gallon water storage 

tank. The total project cost was $3,123,514. In addition to the $300,000 ARC grant, other funding 

included a state Small Cities Community Development Block Grant (CDBG) grant of $500,000, funding 

from the Virginia Department of Health, split into a grant of $1,010,955 and a zero-interest long-term loan 

of $520,790, and a local contribution of $170,177. 

6.2.3 Project Planning and Implementation 

Beginning in the mid-1960s, Big Caney Water Corporation, a private, nonprofit corporation formed by 

local citizens, constructed a water system that supplied running water to portions of Dickenson County. 

Big Caney was the only water source for the county’s unincorporated areas until the mid-1980s, when the 

Dickenson County Public Service Authority (PSA) was established. The PSA is an autonomous public 

authority with a board of directors appointed by the county board of commissioners. It was initially 

tasked with extending water service to areas of the county not served by Big Caney or municipal water 

systems.

By the late 1990s, the Big Caney system had seriously deteriorated. As a private corporation, it did not 

have access to most public funding sources and was not able to pay for adequate system maintenance and 

equipment replacement through its own revenues. Its water lines were in poor repair, losing more than 

2.4 million gallons of water each month, almost half of total water production. Its water tanks and pump 

stations were also in very poor condition, and it was drawing its water supply from a source that did not 

meet state health standards. Meanwhile, the PSA was expanding and, with access to government 

funding, was on a sounder operational and financial footing. In 2002, after lengthy discussions, the two 

systems agreed to merge under the management of the PSA. 

With the merger complete, the PSA and its consulting engineer developed a plan to integrate the two 

systems and rebuild the Big Caney system. The latter would involve replacing all the tanks, pump 

stations, and transmission lines. Planning was conducted in close consultation with the county 

supervisors to ensure county support. Rather than attempting to raise all the funds for the project at one 

time, the PSA broke it into four phases, which would enable it to size each phase to fit within the grant 

limits of ARC, the state’s Small Cities CDBG program, and other key funders. The phasing also enabled it 

to prioritize the work based on the extent of deterioration of different parts of the system. 

The PSA was assisted with grant writing and grant administration by the Cumberland Plateau Local 

Development District. ARC did not participate in Phase 1 of the project; the United States Department of 
Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

Final Report 87

Agriculture Rural Development funded this phase. ARC, in addition to participating in Phase 2, also 

provided part of the funding for Phases 3 and 4. 

Ron Phillips, the PSA’s executive director, reported that funders were initially reluctant to support the 

project because they were more accustomed to funding projects that served areas with no prior access to 

running water in order to promote new economic growth. However, through discussions with funders, 

the PSA was able to persuade them of the important role improving water quality and service reliability 

could play in enhancing the quality of life and economic prospects for the county. Moreover, because the 

project was relatively expensive on account of the area’s mountainous terrain, funding it internally 

without grants and low-cost loans from other sources would result in prohibitively expensive water 

rates for the county’s primarily low- and moderate-income population. Once persuaded, funders were 

willing to provide the support needed to bring the system up to acceptable standards. 

Phase 2 began in late 2005 and was completed roughly according to schedule in late 2007. As with the 

other phases, the construction was overseen by a consulting engineer with assistance from the PSA staff. 

The final cost was $3,123,515, about $278,000 above the original estimate. The additional cost involved 

the installation of new meters and service lines to properties. The PSA was originally expected to cover 

this cost, but did not have sufficient funds to do so. The additional cost was met by a funding increase of 

about $108,000 from the Virginia Department of Health and an increase in the county contribution of 

about $170,000. 

Phases 3 and 4, which were eventually combined, were completed in 2010. The PSA later added a Phase 5, 

which was in progress as of the end of 2012. With the completion of Phase 5, virtually the entire county 

will have access to water service. A Phase 6 is planned to replace undersized spur lines to improve 

pressure to isolated areas. The PSA has also installed several generators to maintain uninterrupted 

service during power failures. 

6.2.4 Economic and Community Impacts 

Phase 2 of the Big Caney water project improved the quality and reliability of water service to 453 

households, 84 more than the 369 projected in the ARC grant application. Replacement of water lines, 

many of them seriously undersized, resulted in increased water pressure and reduced water loss. This 

was particularly important for residences at the far end of the system, which sometimes lost water for a 

day or more during periods of particularly low pressure. 

In addition to improved and expanded residential service, the project also improved service or facilitated 

the construction of businesses and public facilities not anticipated at the time of the ARC application. 

These included: 

 Improved water service to about 10 small convenience stores and other community-serving 

businesses in the areas served by the project; 

 Service to two new educational facilities – a combined middle school, high school, and careertechnical center campus, and an elementary school – which were needed as a result of school 

consolidation and could not have been constructed at the selected locations without the 

improvements; 

 Improved service to the Breaks Interstate Park, which supported the construction of a water 

park and 10 tourist cabins; and 

 New service to a structural metals fabricator with 25 employees. 

The project is also estimated to save the county approximately $35,000 annually through reduced water 

loss, according to an engineering report. 
Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

88 Final Report

Moreover, the project should be viewed as part of the larger initiative to rebuild the entire Big Caney 

system as well as to extend service to virtually the entire county. Through the Big Caney and other PSA 

projects, the percentage of households receiving water service increased from only 34 percent in 1992 to 

93 percent in 2012. 

This initiative has had important community and economic impacts beyond improving water service to 

residents and businesses. Perhaps most important, it has improved the value of residential properties and 

strengthened the residential real estate market. It has enhanced the value of residences that previously 

had poor or no service, and has opened up areas served by both the existing system and system 

expansions to new residential development. The result has been increased investment in existing housing 

as well as increased construction of new housing. County officials and economic development 

professionals believe that improved housing quality and availability is helping to stem the population 

outflow, convincing more young adults to remain in the area and attracting new residents such as 

retirees who earlier left the area to find work. In fact, while it cannot be conclusively attributed to these 

improvements, the county’s population decline has been slowing – from 11 percent in the 1980s, to seven 

percent in the 1990s, to only three percent in the most recent decade. 

The initiative has also provided a foundation for economic diversification. It has already supported 

tourism development through improved water service to Breaks Interstate Park and will enable the 

development of more lodging facilities and second homes. It is hoped that it will also help to attract a 

diverse range of businesses to the county. When searching for a business location, employers give careful 

consideration to quality of life for their employees, and the availability of good-quality housing is an 

important factor in that calculation. In addition, while the county’s current and planned industrial parks 

are served by municipal water systems, areas served by the PSA are now better positioned to attract 

smaller business facilities. 

Finally, the initiative has had positive fiscal implications for the PSA and the county. Drastically reducing 

water loss has resulted in significant cost savings for the PSA, kept water rates reasonable, and reduced 

the possibility that the PSA will ever require a county subsidy. In addition, new home construction and 

increases in property values have generated new tax revenues for the county. 

6.2.5 Lessons Learned 

While PSA staff, county officials, and local economic development professionals all recognized the 

importance of improving the county’s water system at the outset, they expressed surprise at how much 

impact the initiative has already had. Notes Ron Phillips, “We always had a vision, and we felt it was 

going to work, but we didn’t foresee how well it would work. The rewards we are seeing in sustainability 

and improved quality of life are considerable.” Phillips and others also point to the critical importance of 

ARC and other funders. Without these resources, they say, the improvements would have taken far 

longer to achieve. 

Participants in the project point to a number of lessons learned:

 Develop a collaborative culture. The PSA’s collaborative approach extends back to its merger 

with the Big Caney Water Corporation, which was carefully negotiated to achieve a positive 

outcome for both parties. During the Big Caney project, the PSA worked closely with the county, 

municipalities, local development district, and state health officials to bring all ideas and 

perspectives to the table and distill them into a plan on which all stakeholders could agree. This 

collaborative approach continued through all phases of the project.
Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

Final Report 89

 Know where you are going and take your funders with you. The plan to rebuild the Big Caney 

system and extend water service throughout the county was ambitious and costly, particularly 

for economically depressed Dickenson County. The PSA and its partners developed a long-term 

vision for the project, and were able to convey to funders its importance to the county’s future. 

At the same time, they recognized that the project could not be taken on all at once and would 

have to be broken into phases. This enabled funders to understand the project’s overall funding 

needs and at the same time see that it could be sequenced to meet their funding restrictions. 

 Be driven but realistic. Ron Phillips acknowledges that the technical and financial challenges 

presented by the project sometimes appeared insurmountable, but the strong commitment of the 

PSA and its partners would eventually yield solutions. He advises, “Never go into a project and 

throw up your arms and say it can’t be done. There are always bumps, but stay focused, develop 

good relationships, and keep strong people around you.” At the same time, he notes the

importance of adjusting plans to align with funding limitations, stating, “you look at your 

funding sources, you look at your priority list, and you have to cater the project to the funding.”

 Keep key constituencies informed. When a project involves substantial public expenditures, 

water line construction along roadways, and potential service disruptions, local leaders and the 

public expect to be kept informed. The PSA made sure to keep communication lines open and, as 

a result, encountered few complaints. 

 Recognize and value your human resources. The PSA understood the importance of engaging 

skilled, highly qualified people in the project, and maintaining their involvement through 

respectful relationships. When it found that the Big Caney Water Corporation’s as-built 

drawings were deficient, it was able to rely heavily on the staff it had retained from the Big Caney 

Corporation who had worked on the system and had extensive knowledge of it. While it 

competitively bid its contracts, it made sure that local contractors were fairly paid so that, in 

Phillips’ words, “we don’t nickel and dime them out of business.” It also developed a strong, longterm relationship with its consulting engineer, who later became the county engineer.

6.2.6 Conclusion 

Phase 2 of the Big Caney water project exceeded expected outcomes in terms of households served and 

contributed to the larger initiative to improve and extend water service to most of Dickenson County. 

The project significantly improved quality of life for county residents and strengthened the economic 

environment in a number of important ways, including promoting residential development, increasing 

quality housing choices for existing and new residents, and supporting tourism growth. The project 

shows that even ambitious, technically complex, and costly projects can succeed if they are undertaken 

with sound planning, tenacity, collaboration among key stakeholders, and the cultivation of human 

talent. These lessons will serve the community well as it tackles other infrastructure challenges, 

including improving wastewater treatment, telecommunications, and transportation. 

6.3 Burson Center for Business Development – Carrollton, GA 

The Burson Center for Business Development is a small business incubator located in the City of 

Carrollton in west central Georgia. The facility provides affordable space for small, early-stage businesses 

for periods ranging from one to five years. It also serves as a central location for a range of business 

support services for tenants and other businesses in Carroll County and the surrounding region. It was 

developed and is operated by Carroll Tomorrow, a public-private county-wide economic development 
Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

90 Final Report

partnership. ARC provided a grant of $216,882 to support project development costs, approximately 20 

percent of total costs, excluding the value of the property itself, which was donated. The center opened 

in 2006. 

6.3.1 Community Profile 

Carroll County, located in west central Georgia, is an edge county of the Atlanta Metropolitan Statistical 

Area with a 2010 population of 110,527. Carrollton, its largest municipality, is 50 miles west of downtown 

Atlanta. The county has experienced rapid population growth in recent years, with an increase of 27 

percent between 2000 and 2010 compared to 18 percent statewide and 10 percent nationally. 

Employment in the county has increased at a relatively rapid pace. Between 2002 and 2011, employment 

increased by almost 15 percent in contrast to flat statewide employment and national employment 

growth of only one percent. While employment growth has been robust, the unemployment rate has 

generally exceeded state and national rates during the past decade, and the difference has increased 

somewhat during the recent recession. In 2011, the county unemployment rate of 10.7 percent was 0.9 

percentage points above the state rate and 1.8 percentage points above the national level. 

The county’s industrial mix is strongly oriented to manufacturing. About one-quarter of all private wage 

and salary employment is in manufacturing. The manufacturing sector has remained relatively robust 

during the past decade, experiencing net employment growth between 2002 and 2011 (although it shed 

jobs from a peak in 2006 through 2010) in contrast to steep losses at both the state and national levels. 

The continuing strength of manufacturing can be largely attributed to a shift from declining industries 

such as textiles and apparel to growth industries such as fabricated metals, electronics, and automotive 

equipment. The second and third largest private sector employers are health care and retail, both about 

16 percent of total private employment. 

The county is somewhat less well off than both the state and the nation. Median household income 

during the 2006-2010 period averaged 92 percent of the state level, and 88 percent of the national level. 

The poverty rate was 1.4 percentage points above the state level and 3.5 percentage points above the 

national level. The county was classified as “transitional” in 2004 when the ARC grant was awarded and 

again in 2012. A transitional county is defined as economically positioned between the 25 percent most 

distressed and 25 percent most robust U.S. counties. 

6.3.2 Project Description 

The Burson Center is a 24,500 square foot small business incubator located in the Carrollton Industrial 

Park. The facility has 23 leasable furnished offices, 5,000 square feet of warehouse space with loading 

docks, a 2,000 square foot conference room, a library, a break area, and administrative offices. The center, 

which opened in mid-2006, is owned and operated by Carroll Tomorrow, a public-private partnership 

chartered to address the community and economic development needs of Carroll County. Carroll 

Tomorrow covers the cost of staffing, which includes the executive director and an administrative 

assistant, and covers any operating shortfalls, which typically run between $30,000 and $35,000 

annually. 

Tenants may lease space for a period of up to five years, but typically graduate in the third year if not 

earlier. Tenants are expected to achieve one of the following goals by the end of the third year: 1) annual 

gross sales of one million dollars or more; 2) growth to more than 10 employees; 3) acquisition by a larger 

firm; or 4) consistent profitability or income exceeding expenditures by 75 percent. Lease rates are 

structured to encourage tenants to reach one or more of these goals as quickly as possible, starting at 
Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

Final Report 91

significantly below the local market rate and then increasing in year three and again in year four to 

approximately market rate. 

The incubator provides several services included in the lease rate. These include copiers and printers, 

media equipment, a networked telephone system, high-speed internet, reception and clerical support, 

24-7 secure building access, mail/shipping service, and a research library. Tenants also have access to a 

number of business assistance services, including: business plan development; seminars and trainings; 

counseling and coaching; industry and market research; legal and accounting assistance; graduate 

assistants and interns; mentors and peer-to-peer advice; conventional and alternative financing; and 

international trade assistance. In addition to its own staff, the center draws on the expertise of an 

attorney and accountant, who are given free space in exchange for assisting other tenants, Georgia Tech’s 

Procurement Technical Assistance Center and Manufacturing Extension Partnership, both of which have 

on-site offices, and the University of West Georgia’s Small Business Development Center, Center for 

New Business Ventures, and Richards College of Business faculty and students. 

The center is housed in a renovated industrial facility donated by its namesake, Dr. John Burson, a local 

physician and community leader. The construction cost of approximately $1.3 million was funded by a 

combination of federal, state, and local grant dollars, including ARC ($215,882), the U.S. Economic 

Development Administration ($648,000), the State of Georgia ($216,118), the Development Authority of 

Carroll County ($100,000) and the City of Carrollton ($130,000). The donated property was valued at 

approximately $600,000. The Community Foundation of Carroll County also provided Carroll 

Tomorrow with a five-year $25,000 annual grant which can be used to support operating costs. 

6.3.3 Project Planning and Implementation 

The idea for a business incubator emerged several years before construction even began. In the late 1990s, 

business and community leaders saw fundamental changes occurring in the local economy, and were 

concerned that outside forces rather than the community’s own vision were shaping its future. The 

county’s traditional manufacturing economy, particularly its textile and apparel industry, was suffering 

from plant closures and job losses brought on by increasing international competition. Meanwhile, 

Carroll County was increasingly moving into the orbit of the Atlanta metropolitan region, raising the 

fears that the county would become a bedroom community, with a loss of its unique identity. 

In 1999, the community, led by the CEOs of two of the county’s largest employers, embarked on an 

economic development planning process, assisted by a well-known Atlanta-based economic 

development consultant. The planning process, undertaken with widespread community involvement, 

led to the development of a five-point economic development strategy. Two major elements were the 

creation of Carroll Tomorrow to lead the county’s economic development efforts, and an increased 

emphasis on promoting local entrepreneurship. The entrepreneurship component was viewed as 

building on the county’s considerable local entrepreneurial history. At about the same time, a group of 

entrepreneurs, some of whom were also involved in the strategic planning process, began to pursue the 

establishment of a small business incubator. They hired a consultant to perform a feasibility study that 

laid out a path for establishing a nonprofit incubator. The two groups then joined forces, and in 2001 

assigned Carroll Tomorrow the task of planning and developing the incubator. 

The planning process, led by Carroll Tomorrow’s new executive director, was extensive and thorough. 

Conducted over a two-year period, it included market analysis, plans for facility design and renovation, 

financial projections, and discussions with business assistance organizations. Interviews with and trips 

to other incubators were part of the process. Finally, national trends and “best practices” were studied 

using such resources as the National Business Incubation Association. The design evolved through 

several iterations as concepts were proposed, discarded, researched, and refined. 
Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

92 Final Report

The next phase of the process, which took another two years, involved securing a property and obtaining 

funding. The Burson property was identified as a strong candidate in light of its size, layout and location, 

and negotiations were successfully concluded with Dr. Burson for donation of the property. With a local 

match in hand, proposals for grant funding were submitted to Economic Development Administration 

(EDA), ARC and the One Georgia Equity Fund. While the grant process was complicated and timeconsuming, sufficient funding from these sources was secured to begin construction. Once construction 

began, additional costs were covered by the Carroll County Development Authority and the City of 

Carrollton. The cooperation and joint commitment by the county and city to support the project was 

unusual and reflected the broad stakeholder engagement achieved in the project planning and 

development process. 

The renovation was designed according to guidelines published by the National Business Incubation 

Association. Particular attention was paid to the interaction between tenants, leading to the design of 

common space surrounded by tenant offices to serve as informal meeting space and encourage discussion 

among tenants. Flexibility was incorporated into the design. Most of the tenant offices use demountable, 

movable walls combined with removable flooring and subflooring to allow rapid access to wiring and 

cabling. This was intended to provide flexibility to accommodate the varied needs and continually 

changing tenant mix characteristic of an incubator facility. In addition, it was intended to provide an 

architectural and aesthetic look to the facility intended to attract high-quality tenants at market or nearmarket rates. 

Construction began in mid-2005 and was completed by mid-2006. 

6.3.4 Economic and Community Impacts 

During the six years from its opening in mid-2006 through mid-2012, the Burson Center has hosted more 

than 79 businesses, of which eleven have met formal graduation requirements and 49 others are 

considered positive outcomes (firms that have left before reaching one of the defined graduation 

benchmarks, but remain successfully in business). The 11 formal graduates have created 192 jobs and 

attracted $11.1 million in equity investment. All 60 formal graduates and positive exits have created 362 

jobs and attracted $42 million in investment. Since initial lease-up, the center has maintained an average 

occupancy rate of 87 percent. 

Tenants cite a number of reasons for the center’s contribution to their success. These include the quality 

of the facilities and the positive image this creates among clients, networking opportunities with other 

tenants, increased credibility with potential financing sources, the support from the staff in terms of 

direct assistance, arranging assistance from other sources, and facilitating connections with potential 

clients, and the presence of valuable resources within the center such as the free accounting and legal 

services. 

Beyond these tenant outcomes, the center has become a business resource center for both startup and 

established businesses throughout Carroll County and beyond, significantly magnifying its impact. Since 

its inception, it has hosted more than 16,000 visitors for over 1,300 seminars, expos and programs. These 

include seminars presented by the Small Business Development Center, government procurement fairs, a 

“Business Success Luncheon” series organized by the center director, and presentations by state 

economic development organizations. The center’s conference room is also used for meetings by a variety 

of local and regional economic development, business, and civic organizations. 

Less tangible, but perhaps no less important, has been the center’s impact on the perception and reality 

of the community as a supportive entrepreneurial environment. Business and community leaders note 

that the center, with its wide range of business resources, has become the “go-to” place for budding 
Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

Final Report 93

entrepreneurs and has played a significant role in encouraging the growth of entrepreneurship within the 

region. In addition, it has become a model of success that has drawn visitors from many other 

communities and organizations, and created the image of a community that values and nurtures 

entrepreneurship. This, in turn, has increased confidence within the community in its ability to shape its 

economic future. More broadly, the center is held up as an example of successful collaboration among a 

variety of stakeholders that has paved the way for increased cooperation on other local projects.

6.3.5 Lessons Learned 

The Burson Center is widely viewed within the community as a strong success. Several lessons can be 

drawn from the favorable results of this project. 

 Consider the project in the context of larger economic development objectives. The development 

of the Burson Center was not pursued in isolation, but rather as part of a broader community 

strategy that placed a strong emphasis on entrepreneurship development. The foundation of a 

comprehensive strategic plan with strong community support increased the likelihood that the 

project would gain support of key stakeholders and complement other efforts to promote local 

entrepreneurship.

 Plan carefully. The project participants engaged in a thorough planning process before proceeding 

with the project. This included engaging professional assistance to conduct a feasibility study, 

exploring models and best practices, and thoroughly considering such issues as size, design, financial 

structure, tenant services and resources, and operations. While observing other successful 

incubators, they took pains to customize the project to meet local needs and conditions.

 Structure the project to be financially sustainable. The project leaders calculated that the project 

would be unable to take on any debt because, as designed, it would not generate sufficient revenue to 

service debt, and Carroll Tomorrow, the operator of the incubator, could not develop a sufficient 

revenue base from dues and contributions to do so. As a result, they focused on securing a donated 

building and grant funding from multiple sources to cover for all renovation costs. They also ensured 

that Carroll Tomorrow’s revenue base would be sufficient to cover projected gaps in operating 

revenues. 

 Foster a spirit of cooperation. The project was unlike any other undertaken in the community. The 

project leadership recognized early on that a wide variety of stakeholders would need to be engaged 

and relied upon for material support if the project were to succeed. Great pains were taken to reach 

out to city and county government, business and civic leadership, and higher education institutions 

to involve them in the process and gain their support.

 Establish metrics to measure results. Carroll Tomorrow developed very clear metrics to assess the 

track record of the incubator in helping incubator businesses to succeed. This enables incubator 

management to determine whether the incubator is achieving its objectives, to make changes if 

necessary, and to report to the community how its “investment” is performing.

 Maintain a long-term focus. Carroll Tomorrow and other business and community leaders 

recognize that developing and growing a local entrepreneurial base is a long-term process. They had 

the patience to take the time required to ensure that the incubator was well-designed and 

professionally managed and, while they are rigorous in measuring the progress of incubator tenants, 

recognize that the larger economic changes they are seeking to promote will take more than a few 

years to materialize. As one community leader commented, “this is a marathon, not a sprint.”
Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

94 Final Report

6.3.6 Conclusion 

In summary, the Burson Center has made a strong contribution to Carroll County’s efforts to promote 

entrepreneurship, economic diversification and job growth. A strong consensus has developed among 

community leaders that the project has met and even exceeded expectations. The project outcomes 

demonstrate the importance of establishing a strategic focus for economic development, careful planning, 

stakeholder engagement and collaboration, and developing and applying clear metrics to measure project 

results. 

6.4 Canaseraga Wastewater Collection and Treatment System – Canaseraga, NY 

Canaseraga, New York, is a small village of 550 residents in Allegany County, part of the state’s Southern 

Tier. Between 2004 and 2008, the village constructed a new wastewater treatment system to replace 

residential septic systems, many of which were failing and discharging contaminated water into a local 

river. The Appalachian Regional Commission provided a $150,000 grant in support of the project, 3.3 

percent of the total project cost of $4.59 million. 

6.4.1 Community Profile 

Canaseraga, like many small upstate New York communities, was sustained by traditional 

manufacturing until the 1970s. At that point, it was a bedroom community of nearly a thousand, with 

most of its residents commuting to bustling manufacturing centers like Hornell, Dansville, and Genesee. 

Its small downtown was thriving, with many retail and eating and drinking establishments. 

Today, as these jobs have disappeared, Canaseraga’s downtown consists of only a grocery store, barber 

shop, and a barroom. “The population is leaving,” Mayor Robert Ames notes. “People are going places for 

jobs.” Between 2000 and 2010, the village continued a trend of declining population, with a loss of 7.4 

percent, compared to a 2.1 percent gain statewide and a 9.7 percent gain nationally. Among those who 

remain, incomes are relatively low. Median household income during the 2006-2010 period averaged only 

64 percent of the statewide level and 70 percent of the national level. At the same time, there are fewer 

individuals at the low end of the economic scale, with a poverty rate of 9.1 percent, 5.1 percentage points 

below the state average and 4.7 percent below the national average. 

Despite a decline in manufacturing employment, including a decline of about 20 percent between 2001 

and 2011, workers in Allegany County continue to rely upon manufacturing for a considerable amount of 

employment. Approximately 16 percent of total county employment is in manufacturing. The largest 

manufacturers still remain in small cities like Hornell. Other large non-manufacturing employers include 

the local public school system, located at the village’s edge, and the Livingston Correctional Facility, 

located to the north in Sonyea. 

In fiscal year 2012, as in 2005 when the project was initiated, ARC continued to categorize Allegany 

County as a “transitional” county, or an area “transitioning between strong and weak economies.” Total 

employment declined by 3.1 percent during this period, compared to slight growth of 0.3 percent 

statewide and a slight decline of 0.2 percent nationally. The county’s unemployment rate averaged about 

0.5 percentage points above the statewide average and 0.3 percentage points above the national average 

between 2001 and 2011. 
Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

Final Report 95

6.4.2 Project Description 

The Canaseraga Wastewater Collection and Treatment System project consisted of the construction of 

5.4 miles (28,400 linear feet) of sewer and a 0.1-million-gallon-per-day treatment plant. It serves 215 

households and 18 businesses. The project was undertaken to replace residential septic systems, many of 

which were failing and discharging waste into a local river. The total cost of the project was $4.59 

million. In addition to the $150,000 ARC grant, funding was obtained through a $1.879 million United 

States Department of Agriculture (USDA) Rural Development Grant, a $2.161 million zero-interest 

USDA Rural Development Loan, and a $400,000 Small Cities Community Development Block Grant 

(CDBG) grant from the State of New York. Construction began in late May 2004 and was completed in 

late May 2008, spanning four years. 

6.4.3 Project Planning and Implementation 

The village had long been aware of its sewage issues, and their environmental, health, and economic 

impacts. Many property owners had been financially unable to properly maintain their septic systems, 

leading to widespread leaks and failures as the housing stock aged. When the systems failed, these 

homeowners had to choose between finding money for costly repairs (sometimes in excess of $10,000), 

abandoning the property altogether, or diverting the sewage into storm drains or the local river. For 

those choosing the latter two options, an environmental and health hazard was created for residents, 

their neighbors, and those using the river nearby or downstream. It also created a significant economic 

issue, depressing property values, and reducing recreational hunting and fishing. Wildlife was observed 

to be increasingly scarce, a fact attributed to increasing river pollution, forcing hunters to look elsewhere. 

Local fishermen were also becoming concerned about diseases in locally caught fish. 

In the mid-1990s, the village began to receive citations from New York State for sewage treatment 

compliance issues. However, the citations were rescinded when it became clear that the community did 

not have the financial resources necessary to construct the required sewer system. 

In late 2001, a consulting engineer for the Village of Canaseraga, having contacted the USDA Rural 

Development office in Bath, New York, on an unrelated matter, mentioned the village’s wastewater issues 

to J.C. Smith, USDA’s local manager. Smith then contacted village leaders, volunteering to help them find 

a way to address the problem. 

The consulting engineer prepared a design for the system with an estimated cost of approximately $4.6 

million. During 2002, Smith worked with the village to prepare applications for USDA funding, state 

Small Cities CDBG funding, and an ARC grant. They also conducted an environmental review and 

prepared the documents required to comply with the State Environmental Quality Review Act. Soon, the 

pieces began to fall into place. 

Later that year, USDA Rural Development issued an eligibility letter of grant support for $1.879 million, 

and also offered the village a zero-interest loan of $100,000. The New York Environmental Facilities 

Corporation also offered a $2.061 million low-interest loan from its USDA-funded State Clean Water 

Revolving Loan Fund. With almost 90 percent of the funding in hand, local leaders began to hold local 

information meetings about the project and its impacts and worked to pass a village referendum in 

support of the project. All this occurred in careful synchronization with the April CDBG funding due 

date in order to submit the strongest application possible. 

In late 2003, the village was notified of its $400,000 CDBG award through the Governor’s Office for 

Small Cities. With most of its funding now committed, demonstrating a feasible, actionable project, the 

village’s ARC funding application was scored highly, and ARC awarded the final $150,000 needed to 
Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

96 Final Report

fund the project. ARC had rejected two earlier applications because of concerns about the project’s 

financial feasibility. 

The final design and construction of the system occurred between 2004 and 2008. No major delays or 

cost-overruns were encountered. The village’s superintendent of public works did report that in order to 

complete the project within budget, the village did not build planned protective structures around 

certain pieces of equipment. This exposes this infrastructure to the elements and makes monitoring the 

equipment more difficult, particularly during the winter months. 

6.4.4 Economic and Community Impacts 

Village and county officials believe that the project’s greatest impact has been on improving local 

environmental quality and resident health. Notes Wilson Rowley, the local superintendent of public 

works, “I remember when I was a kid, trout fishing in the creek. You could literally see someone’s gray 

water running off. There still are rural areas like that. People try to hide it. [Canaseraga] was like that. 

[But now] most people, overall, are real thankful [that we’re not].” Agrees County Administrator John 

Margeson, “The best thing to come out of this project is correcting the threat to public health. Any future 

economic development impact will be icing on that cake.”

According to Wilson, with the river cleaner, wildlife has returned to the area, a boon for the many local 

hunters and gamesmen. And the systems will also benefit financially in the long run. “I know that people 

are going to complain about debt service charges. But they don’t always remember that they don’t ever 

have to pump their septic tanks again. It alleviates a lot of worries on the taxpayer. If the power goes out, 

we still have sewer service. And you never have to worry about septic service backing up. If it does, on 

your property, the village is going to take care of it. It has all been beneficial.”

Approximately 215 households are now being serviced by the sewer system. In addition, an estimated 155 

local jobs were retained. Further, as Margeson notes, it allows for possible future development, 

leveraging a growing nearby tourist attraction, the Swain ski area. “The introduction of the sewage 

system makes it more attractive to individuals of some means who might want to put up seasonal homes 

in the village to take advantage of its proximity to Swain. Further, within the village proper, we could see 

possibly some light industrial development and substantial residential development if the infrastructure 

is extended west towards Swain.”

6.4.5 Lessons Learned 

 Tackling big projects in small areas takes dedication and patience. Projects serving small, rural 

communities, as ARC projects often do, may take a considerable amount of time to launch, 

convincing funders of project viability, especially given the naturally high cost per-person-served of 

such endeavors. In Canaseraga, the process took over six years from initial conception to completion. 

For the first financial commitment alone, a crucial $1.85 million USDA Rural Development grant, the 

village had to wait two years. However, once the first funder committed, successive financing became 

more easily available, concluding with the $150,000 ARC grant. It was through this often arduous 

process that a group of dedicated local leaders were able to complete a project previously thought 

financially infeasible. However, by the time the project came to fruition, leaders had achieved a high 

level of local buy-in, developed a sound, well thought out plan, and honed an effective case for why 

this project was necessary. It was all accomplished by having a process through which stakeholders 

learned from failure and did not get discouraged in the pursuit of their goals. 
Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

Final Report 97

 Addressing health and environmental concerns is a foundational step to successful economic 

development. For economic development to occur, many of Appalachia’s small villages first require 

infrastructure investments to address basic quality of life concerns. In the Village of Canaseraga, 

health and environmental issues were paramount, with sewage leaking into natural water sources, 

and impacting the health and aesthetics of the village itself. Addressing these needs was the main 

catalyst for the project, and it has successfully addressed these issues. However, in doing so, it has 

also made the village a better place to live and to work, with new businesses now able to connect to 

modern sewer amenities. The small downtown, too, can offer small businesses the option of 

connecting to a sewer line. And while residents will bear the costs of the project’s operations and 

debt service, they will never have to worry about septic tank leakage or maintenance in their yards. 

The project has allowed Canaseraga to compete on a more even playing field with other similar 

communities for business and residential development by first addressing its health and 

environmental concerns. 

6.4.6 Conclusion 

Through the 1990s, Canaseraga’s failing septic systems threatened the village’s health and economic wellbeing. Starting in 2001, dedicated local officials, supported by USDA Rural Development, were able to 

demonstrate the need for the project and piece together funding, including persuading village residents 

to pass a referendum to take on substantial debt and ensure its financial viability. Today, the village 

benefits from a centralized sewage system and, with it, a higher quality of life and a greater chance to 

compete for residents and businesses that expect such modern amenities. 

6.5 Canton Water Improvement Project - Canton Borough, PA 

The Canton Water Improvement project consisted of the replacement of five miles of water mains and 

the construction of a 300,000-gallon water storage tank. The project was necessary to replace the 75-

year-old distribution lines that were in poor condition and losing over 65 percent of their treated water 

supply through leakages. The water tank was necessary to expand storage capacity and to improve the 

services for the entire community. The Appalachian Regional Commission (ARC) provided a grant of 

$285,000, approximately eight percent of the total project cost of $3.7 million. 

6.5.1 Community Profile 

Canton Borough is located in the southwest corner of Bradford County, Pennsylvania. It had a 

population of 1,976 in 2010, an increase of 9.4 percent over 2000. Its growth was more rapid than 

statewide growth of 3.4 percent and approximately on par with national growth of 9.7 percent. 

Bradford County is classified by ARC as a non-metro area adjacent to a small metro area and had a 

population of 62,622 in 2010. At the time the ARC grant was awarded in 2006, the county was classified 

as a “transitional county,” according to the ARC economic classification system, and continued to be 

classified as such in 2012. A transitional county is defined as economically positioned between the 25 

percent most distressed and 25 percent most robust U.S. counties. 

The county’s economic performance was relatively weak between 2001 and 2009, but began to improve 

significantly between 2009 and 2011, primarily as a result of increased shale gas development. Between 

2001 and 2009, county wage and salary employment declined by 7.5 percent, compared to declines of 1.5 

percent and 0.8 percent, respectively, at the state and national levels. However, between 2009 and 2011, 

this relationship was reversed, with county employment growing by 4.9 percent, compared to respective 
Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

98 Final Report

declines of 0.3 percent and 0.2 at the state and national levels. During this latter period, employment in 

the gas industry grew by over 1,000 jobs and from less than one percent to five percent of total county 

employment. The county’s unemployment rate generally tracked state and national averages during the 

2001-2009 period, but fell significantly relative to state and national averages in 2010-2011. 

The three major industries in Bradford County are manufacturing, retail, and healthcare and social 

assistance. In 2011, these industries accounted for approximately 18 percent, 12 percent, and 20 percent of 

wage and salary employment respectively. Between 2001 and 2011, manufacturing experienced a 

significant employment decline, while retail remained stable and healthcare and social assistance grew 

moderately. As noted, while the mining sector is still relatively small, it has added over 1,000 jobs 

between 2009 and 2011. 

Borough residents suffer from relatively low economic status. Median household income averaged 

$28,382 during the 2006-2010 period, little more than half state and national levels. The poverty rate of 

29.7 percent was more than double both state and national levels. 

6.5.2 Project Description 

The Canton Water System Improvement project consisted of the replacement of five miles of water 

mains and the construction of a 300,000-gallon water storage tank. The reconstruction of the old system 

and the water tank started early in 2006 and took nearly 16 months to complete. The project replaced a 

water distribution system that was built in the early 1900s and had outlived its useful life. The former 

system no longer provided the adequate supply needed to fight major fires, and more than 65 percent of 

the treated water supply was lost through leakage. 

The upgrade project was completed at a total cost of $3.7 million. ARC provided a grant of $285,000. 

Other funding sources included $50,000 from Pennsylvania’s Community Development Block Grant

(CDBG) program, $3.2 million in loans from United States Department of Agriculture Rural 

Development, and $162,400 in local funds. 

ARC funds were needed to increase the scope of the improvements and alleviate the financial burdens on 

the community. ARC funds mainly contributed to the decision to build the water tank, a project that 

likely would not have otherwise been completed due to lack of funds. While water rates were increased 

by 40 percent to achieve the project, they would have been 45 to 50 percent higher had ARC funds not 

been available. 

6.5.3 Project Planning and Implementation 

The initial impetus for the project came from two of the community’s major employers, which were 

under pressure from their insurance companies to increase water flow and storage for fire protection. 

Therefore, the project was needed to provide an adequate infrastructure to serve the community, retain 

existing businesses, and attract new industries. 

Another factor that contributed to undertaking the project was the necessity to separate an existing 

combined sewer system. The sanitary and stormwater were in the same piping, resulting in the discharge 

of raw sewage during a significant rain event. Although ARC funds were only used for the water line 

improvement portion of the project, the water upgrade parallel to the sewer system upgrade made sense 

due to significant cost savings from mobilizing and repaving. 

The project planning started in 2002 and construction started late in 2006. The initial project cost was 

estimated at $3.2 million. The engineering firm Larson Design Group was hired to conduct daily 

inspections, and the Canton Borough Authority (CBA) provided personnel to assist the inspection and 
Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

Final Report 99

construction crews in locating existing mains and service lines. According to CBA staff, no major 

obstacles or challenges were encountered during the planning and implementation phase of the project. 

However, two unanticipated events increased the project cost by $0.5 million. First, additional highway 

restoration was required on one state highway due to an unstable base; and, second, the number of water 

main fittings needed was significantly underestimated. These unexpected events brought the total 

project cost to $3.7 million. The cost overrun was covered with an additional loan, and the project 

engineer, Larson Design Group, provided $100,000 for its estimating error. Despite these changes, the 

project was completed within the anticipated timeframe. 

6.5.4 Economic and Community Impacts 

The projected outcomes for the project were to minimize water loss by half (to 30 percent), increase firefighting capacity, and to provide adequate water supply to residents, the Shop Vac plant, and to the 

former Paper Magic plant to encourage a new company to utilize the facility. The project met the grantee 

and community expectations. It improved the service for 750 households, 70 commercial customers, and 

retained nearly 200 jobs. While two miles of the system still require replacement, the project reduced 

water loss from nearly 65 percent to less than 20 percent. Given that the operating efficiency improved 

more than anticipated, CBA has been able to sell water that had been previously lost through leakage to 

gas companies. The sale of the water has generated an additional $1 million in annual revenue over two 

years. This additional revenue is used to pay the 40-year USDA loan. It is expected that once the loan is 

paid, rates will come down and the additional revenue will be applied to new projects and to a new 

equipment replacement fund.

Overall, the project both alleviated environmental concerns and improved the economic potential of the 

area. It improved the water quality, reduced water losses to less than 20 percent, served the projected 

households and businesses, and created an unexpected revenue source. In addition, while the community 

lost Shop Vac when the company relocated its facility, it gained another company, in part because of the 

infrastructure project. Cudd Energy Services relocated its Pennsylvania headquarters into an empty 

Parker Hannifin plant, hiring or relocating more than 150 workers. 

6.5.5 Lessons Learned 

Several lessons can be drawn from the outcome of this project: 

 Identify the basic needs of the community in order to maximize its economic and social 

development. The water system upgrade was a priority because the inadequate system was 

jeopardizing the social and economic development of Canton Borough. The inefficiencies were 

limiting the expansion of major employers, were causing excessive expenditures by local government, 

and were a risk to human health. The infrastructure upgrade exceeded the expectations of 

stakeholders, retained and attracted new businesses, and generated a new revenue source for Canton 

Borough. 

 Pay careful attention to project design. The reconstruction of the water distribution system was 

carried out in parallel with the sewer system upgrade to reduce costs and maximize resources. As 

part of the strategic planning process, different types of water main materials were evaluated to 

compare advantages and disadvantages. Polyethylene pipes were selected due to unique physical 

characteristics: low weight, corrosion resistance, high-impact resistance, and flexibility. Leaks were 

substantially reduced, increasing the efficiency of the system. 
Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

100 Final Report

 Conduct routine project inspections. The project’s success was due in large part to the diligent 

coordination between CBA and the project engineer. Daily inspections conducted by the project 

engineer ensured that the construction was carried out according to the plans and specifications. 

Canton Borough Authority also assisted the construction team in locating existing mains and service 

lines to reduce errors.

 Keep the community informed. A factor to consider on future projects and recommended for other 

communities is increasing coordination with the community. Downtown businesses received several 

water advisories from CBA, but they were not prepared for the project and CBA did not do enough to

prepare them. 

 Develop a contingency plan. The original cost of the project was projected at $3.2 million, yet the 

final cost increased by $500,000 as a result of unexpected events and the estimating error by Larson 

Design Group. Although the engineering firm provided $100,000, CBA had to make an additional 

loan to cover the overrun. CBA recommends that future grantees allocate a contingency fund or 

secure access to credit markets for additional loans to cover unexpected expenses. 

6.5.6 Conclusion 

During 2007, the Canton Borough Authority replaced nearly 60 percent of its 75-year-old water 

distribution system because of the poor condition of distribution lines and inadequate water pressure to 

serve businesses or to fight major fires. A total of five miles of water line were replaced and a 300,000-

gallon water storage tank was constructed. The reconstruction of the water line eliminated major leaks 

in the system and immediately improved operating efficiencies and fire protection levels. The project 

served 750 households and 70 employers in the Borough and Canton Township. An unexpected but 

positive outcome is that as a result of the improved efficiency, CBA is selling water to gas drilling 

companies and collecting additional revenues.

6.6 Clinton I-75 Industrial Park Sewer Line – Clinton, TN 

In 2006, the City of Clinton, Tennessee’s Utilities Board completed major improvements to the 

wastewater collection and transmission system in the city’s I-75 Industrial Business Park. The project 

was undertaken to accommodate the needs of Aisin Automotive Castings, an automotive component 

supplier that wanted to build a production facility in the park. The ARC awarded the city a grant of 

$450,000, which covered 30 percent of the project’s total cost of $1.49 million. 

6.6.1 Community Profile 

The City of Clinton, located in Anderson County, Tennessee, just northwest of Knoxville, had a 

population of 9,841 in 2010. The city experienced modest population growth of 4.6 percent between 2000 

and 2010, but lagged the growth rate at both the state and national levels by more than half. 

ARC categorized Anderson County as “transitional” in fiscal year 2013, as it also did in 2005, or an area 

“transitioning between strong and weak economies,” ranking between the “worst 25 percent and the best 

25 percent of the nation’s counties.” The county experienced employment growth of 5.3 percent between 

2001 and 2011, while state and national employment declined by 4.6 percent and 3.3 percent, respectively. 

Unemployment rates during this period averaged slightly below both state and national rates. In 2001, 

the county unemployment rate was 8.4 percent, compared to 9.2 percent statewide and 8.9 percent 
Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

Final Report 101

nationally. The county has traditionally, and continues to, rely heavily on manufacturing jobs. In 2011, 

fully 25.6 percent of jobs were in manufacturing, only a slight decline from a decade earlier. 

The economic status of county residents is about average relative to the state. The average median 

household income during the 2006-2010 period was 95 percent of the state level, while the poverty rate of 

14.4 percent was 2.1 percentage points lower. Compared to the nation as a whole, the county is 

somewhat less well-off, with median household income 79 percent of the national level and the poverty 

rate 0.6 percentage points higher. 

At the time of the Aisin sewer line project, the community had been hit hard by the downsizing of the 

Department of Energy’s (DOE) facilities in neighboring Oak Ridge, Tennessee, which employed many 

local residents. Over the previous 10 years, the DOE facility had reduced its staff from 20,000 to 12,800. 

Further, as Terry Bobrowski, executive director of East Tennessee Development, noted, “A lot of folks 

[have been] retiring” in the area, and the younger families that have remained are finding themselves 

“unable to keep their kids at home with good paying jobs,” leading to a young professional brain-drain 

and an increasingly older population, with 19.3 percent of local residents age 65 or older – over six 

percentage points higher than the United States average.

6.6.2 Project Description 

To address Aisin’s infrastructure needs, the Clinton Utilities Board undertook major improvements to 

the park’s wastewater collection and transmission system: the installation of 6,800 linear feet of sewer 

line, the construction of two new pump stations, one at the Aisin site and the rehabilitation of another. 

The plant was expected to produce an average of 180,000 gallons of wastewater per day. The final cost of 

the project was $1.26 million. In addition to the ARC grant of $450,000, funding sources included 

$450,000 from the state of Tennessee’s Fast Track Infrastructure Development Program, and 

contributions of $225,000 by Anderson County, $75,000 by the City of Clinton, and $299,550 by the 

Clinton Utilities Board. 

6.6.3 Project Planning and Implementation 

In 2004, Aisin Automotive Castings, a subsidiary to Aisin Seiki Limited, one of the world’s top 

automotive component suppliers, indicated interest in building a metal casting facility in Clinton, 

Tennessee’s Interstate 75 Industrial Business Park. The new park, one of Anderson County’s largest, was 

set on 252 acres along the Interstate and had been, to that point, largely vacant. However, Aisin also 

indicated that the park’s current wastewater collection infrastructure would not be able to handle the 

expected 180,000 gallons of wastewater released per day by the prospective plant. “We had the basic 

infrastructure there,” Greg Fay, general manager of the Clinton Utilities Board recalled. “What we

needed was a main section of line in order to move the waste forward.”

Aisin’s investment was expected to be significant – approximately $65 million to construct and equip a 

280,000-square-foot production facility. It expected to create 400 new jobs after three years, with 

average wages of $16.00 per hour, plus benefits. For the community, suffering from workforce reductions 

by one of its major employers, the Department of Energy, the investment would be a much needed 

employment generator. 

Local leaders convened to see what could be done to meet Aisin’s infrastructure needs. Engineers 

determined that in order to accommodate Aisin approximately one mile of sewer line, two new pump 

stations, and the rehabilitation of another would have to take place. The cost would total approximately 

$1.26 million. Given that the local utility, the Clinton Utilities Board, serving only 6,000 customers, could 

not absorb such an investment alone, “everyone pulled together,” Fay noted. Everyone gave “as much as 
Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

102 Final Report

everyone could stand, which was sufficient to make it work.” In addition to contributions by the city, the 

county, and the utilities board, grants were also obtained from the state of Tennessee’s Fast Track 

Infrastructure Development Program (FTIDP) and the Appalachian Regional Commission. 

From the start of implementation to project completion, local stakeholders met and spoke regularly after 

setting expectations for financial contributions and project responsibilities. The Clinton Utilities Board 

was responsible for overseeing construction. 

The project experienced some normal weather delays, but was completed on schedule in early 2006. 

However, costs exceeded the original budget by a considerable amount. “East Tennessee is nothing but a 

rock bed,” Fay reported. When the contractors hit rock when digging the sewer line, they exercised the 

“fair” terms of their contract to charge more. The additional overruns amounted to slightly over $200,000, 

bringing the total final project cost to $1,494,500. The Clinton Utilities Board covered these overages 

from its own reserves, raising its initial contribution from $67,700 to $299,550. 

6.6.4 Economic and Community Impacts 

The payoff from the installation of the Clinton I-75 Industrial Park sewer line has been significant. Aisin 

moved into the industrial park in late 2004 and created the promised 400 jobs. In late 2007, it doubled its 

manufacturing space from 240,000 square feet to 480,000 square feet, and added another 160 positions to 

bring its workforce to 560. Exercising an initial option, it arranged to purchase 36 additional acres of 

industrial park land at a subsidized rate of $100 per acre. 

Aisin has become the county’s second largest taxpayer and is widely expected to soon top the list. Even 

with a 25 percent, 10-year tax abatement on the original plant, which was part of a larger incentives 

package arranged separately from the infrastructure investment, Aisin pays $525,402 a year in county 

property taxes and $136,233 in city taxes. These figures do not include payroll taxes or the sales and 

property taxes generated by Aisin employees, which would bring the tax dividends from this investment 

to well over a million dollars per year. 

Noted Terri Bobrowski, “This was one project that really has not only met initial expectations, but has 

consistently provided additional benefits over time. Aisin is a great employer and corporate citizen, a 

poster child for good investment on the public side.”

Further, since Aisin’s arrival, the industrial park and the adjacent area have expanded significantly. The 

industrial park is now close to capacity, with another 10 to 12 industrial businesses establishing 

themselves within it or nearby. The total available acreage in the 252-acre park is down to less than 25. 

“It also opened up an entire Interstate exit,” Bobrowski reported, with a Wal-Mart, a Toyota dealership, 

and several restaurants and hotels springing up since. A Junior Achievement Center has also opened a 

15,000-square-foot facility that serves the region with a business learning center and regular 

entrepreneurial programming for local students. 

Bobrowski estimated that 50 new commercial businesses have since opened, all using the sewer system 

built for Aisin, and that that 1,500 new jobs have been created from subsequent development that relied 

on that infrastructure. 

6.6.5 Lessons Learned 

All interviewees agreed that the project exceeded all previously held expectations. The ultimate $1.5 

million investment was likely recovered in tax revenue gains alone by the second year of Aisin’s 

operation. Further, Aisin has gone on to expand its employment base and its property holdings. The once 
Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

Final Report 103

vacant industrial park is now near capacity, and the surrounding area, also once vacant, hosts 

restaurants, hotels, and approximately 50 new commercial businesses, all of which are using the 

infrastructure built out for Aisin. 

While the build-out may not have been the sole contributor for this growth, it was the investment that 

allowed for Aisin’s arrival that spurred other development in the industrial park and along the Interstate 

that may not have occurred otherwise. It was a catalytic investment. 

However, even with the most positive economic implications that resulted from the investment, 

interviewees noted that many lessons were learned throughout the process: 

 “Committed” funding is not necessarily the same as having money in-hand. Five different 

organizations contributed funding to the Clinton project. Each funding organization often had its 

own standard set of use restrictions and methods for funding distribution. For instance, Bobrowski 

noted, as East Tennessee Development served as the Clinton project administrator, it was “tough 

tracking wage rates,” with the state and federal guidelines requiring different levels of pay. 

Ultimately, he said, “it ended up costing the project money to deal with the complexity, in terms of 

the detail required.”

 Each funding source may come with its own restrictions and requirements of which grantees 

need to be continuously aware. “When you mix money,” Bobrowski stated, “there’s always 

administrative cash flow problems,” which naturally arise. Some funding sources may only pay for 

certain expenses, while others want the project administrator to pay for everything up-front and 

then get reimbursed. This may then require the project administrator to access lines of credit from 

local banks, which may not always be available, especially in small communities, and which will also 

add previously unaccounted for interest payment expenses to the project. Having a clear, detailed 

understanding of the restrictions and parameters of the funding from all partners from the start of 

the project can allow for a smoother transaction process. 

 Having trust in your local partnerships is crucial to smooth project planning and 

implementation. All interviewees noted the importance of trust and partnership in what went into 

making the Clinton project a particular success. Greg Fay of the Clinton Utilities Board noted, “Trust 

is key. We didn’t have any trouble with any of the partners, but that’s about relationships. I’ve been 

here for 35 years. You have to have a relationship with your city council, your county commission. 

That’s imperative...you have to sometimes take people where they might be uncomfortable going at 

first. And that takes professional comfort and trust.” All interviewees knew each other and thought 

highly of one another. This sense of trust allowed all of the key players to effectively work together to 

get the proposal done, win funding, and implement the project.

 Informed and knowledgeable staff can more effectively oversee implementation. Terry 

Bobrowski noted that the Clinton Utilities Board had a local wastewater specialist on staff who was 

invaluable throughout the process. The knowledgeable individual consistently conducted quality 

checks on the materials and construction. “He understood what was going into the ground and was 

able to make field adjustments as needed.”

 When looking for contractors, do not just look at the dollar value; look at what will be gained 

by that investment. When putting a contract out to bid, as in Clinton, Greg Fay emphasized that 

while it was important to ensure that the bid process is open and fair, the contractor ultimately 

selected must be capable of doing the job at hand and getting it done on time. He warned not to just 

“hire people one knows” or ones that give the “lowest bid.”
Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

104 Final Report

 Sometimes supporting smaller, locally grown projects can pay greater dividends than other 

statewide attraction efforts. Terry Bobrowski added that when looking at infrastructure 

investment projects like the one for Aisin in Clinton, it is important to put it in the perspective of 

how many jobs would be created. “Just a little bit of money can make a lot of difference,” he noted. 

While “tens of millions were spent by Tennessee to land [the now defunct] Saturn,” this small 

infrastructure grant had an immediate payoff and opened up an entire area for subsequent 

development. 

6.6.6 Conclusion 

Greg Fay called the Clinton I-75 Industrial Park sewer line project, funded in part by ARC, and Aisin’s 

resulting build-out, the “gift that keeps on giving.” The I-75 Industrial Park in Clinton went from being 

vacant to now running at near full capacity. The local area around the park, once empty, now holds over 

40 new businesses, including restaurants, hotels, and an entrepreneurial education center. The gap 

funding the ARC was able to provide directly resulted in Aisin, which now employs over 600 individuals 

from across the region, locating in the area. Indirectly, it helped to create the infrastructure needed for 

the area to flourish, with one interviewee approximating the creation of well over 1,000 additional jobs. 

Noted Terry Bobrowski, “This project would not have happened without an ARC grant. They were a 

critical partner. And the results have been tremendous.”

6.7 Dobson I-77 Infrastructure Project – Dobson, NC

In 2006, the Appalachian Regional Commission (ARC) provided a grant of $200,000 to support a $2.2 

million extension of municipal water and wastewater service to the site of a planned commercial 

development at I-77 Exit 93 outside the Town of Dobson in Surry County, North Carolina. The project 

serves various commercial and residential customers, but its primary purpose was to provide the basic 

infrastructure needed for the construction of the Village at Shelton Vineyards, a hotel-centered 

commercial complex by the town’s interstate exit conceived by the owners of Shelton Vineyards, a large 

nearby vineyard and winery that is open to tourists. 

6.7.1 Community Profile 

Surry County is a predominantly rural county with a population of 73,673 in 2010, located in northwest 

North Carolina next to the Virginia border. The county is split between the Piedmont and the Blue Ridge 

Mountains, which makes it a part of both the urbanizing, agricultural Piedmont Triad region of northcentral North Carolina and the more rustic Appalachian foothills region of western North Carolina. It is 

also considered a part of the Yadkin Valley. Dobson, the county seat, lies at the center of the county and 

had a population in 2010 of 1,586. There are three other incorporated municipalities in Surry County: the 

Towns of Elkin and Mount Pilot, and the City of Mount Airy. Through most of the 20th century, textile 

manufacturing and tobacco cultivation were central to Surry County’s economy, but after several decades 

of decline in these industries, Surry County has a more diverse, but smaller, industry base with a much 

larger role for the service sector. Farming and processing of agricultural products still play a significant 

role in the economy, but livestock has replaced tobacco as the most important commodity. 

In 2012, Surry County was designated as an ARC “transitional” county, the same designation it had in 

2006 when the ARC grant was approved. ARC defines transitional counties as being in the middle 50 

percent of counties in terms of level of distress. While it is far from depressed, Surry County 

underperforms across many different indicators when compared with North Carolina and the country as 

a whole. Surry County did grow in population between 2000 and 2010, but it only grew by 3.4 percent, a 
Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

Final Report 105

rate much lower than the state’s 18.5 percent growth rate and the nation’s 9.7 percent growth rate. Surry 

County’s median household income in the 2006-2010 period was $37,294, 82 percent of the state median 

and 72 percent of the national median. In that same period, it had a poverty rate of 16.9 percent, higher 

than the state’s 15.5 percent rate and the country’s 13.8 percent rate. Its unemployment rate for 2011 was 

11.3 percent, 0.8 percentage points higher than the state rate and 2.4 points above the national rate. 

Perhaps the most drastic statistic in Surry County is its shrinking employment. Between 2002 and 2011, 

payroll employment in the county fell 18 percent, from 33,166 to 27,206, even as state employment rose by 

approximately two percent. Losses in the manufacturing sector, where employment fell 58 percent from 

9,328 to 3,958, account for almost all of the county’s employment decline. The manufacturing job loss was 

itself concentrated in what was once the county’s most important industry, textiles. In 2001, there were 

6,123 jobs at the county’s apparel manufacturers and textile mills; in 2010, only 875 remained.

Job growth in other industries generally did not counteract the decline in manufacturing jobs. Some 

parts of the service sector, such as education and healthcare, have grown. 

Tourism is one of the few bright spots in the county’s employment numbers. Between 2002 and 2011, the 

county’s private sector employment in leisure and hospitality grew from 2,100, about six percent of total 

employment, to 2,679, about 10 percent of total employment. In this area, Surry County has actually 

outperformed the state’s growth level by almost eight percentage points. This was a result of sustained 

local, county, and regional efforts to promote tourism and build tourism-related industries. Part of this 

involved finding ways to take better advantage of existing natural, cultural, and historical attractions, 

like scenic roads, state parks, traditional music, and the Andy Griffith-related sights of Mount Airy (or 

“Mayberry” as it is known to fans of the show). But another side of the story was the creation of 

something almost entirely new in the area: winemaking and wine tourism. In 2000, when Ed and Charlie 

Shelton founded their winery outside Dobson, there was only one other winery in the region. Today, the 

Yadkin Valley, an American Viticultural Area (AVA) since 2003, has over 30 wineries that attract tourists 

who want to visit wineries and explore the area. About 12 of these wineries are in Surry County, due in 

large part to Shelton Vineyards and the Shelton-Badgett Center for Viticulture and Enology at Surry 

Community College in Dobson. 

6.7.2 Project Description 

The Dobson Exit 93 infrastructure project consists of a water and sewer system extension that brings 

services from the Town of Dobson to the area around I-77 Exit 93, a site of existing and planned 

commercial development three miles west of the main part of town. The town owns and manages the 

systems. 

The water extension was designed to serve commercial customers around the interchange as well as 

households and churches. Its major elements are a booster pump station, about 19,500 linear feet of eightinch water lines, and a 200,000-gallon water tower. The pump station was built with the ability to 

connect to additional water systems, and it was subsequently used to provide water for a different 

interstate interchange development project several miles north of Dobson. 

While the sewer system travels along approximately the same route as the water line, in order to reduce 

costs and prevent overdevelopment of farmland, it was designed with a long force main so that only the 

area around its destination would be easily served. The primary components of the sewer extension are 

two pump stations by the interchange, 4,000 linear feet of eight-inch gravity sewer, and 18,700 linear feet 

of two- and four-inch sewage force main. 

The completed project cost $2,263,368. Non-local funders included ARC ($200,000), the North Carolina 

Rural Economic Development Center (Rural Center) ($500,000), and the Golden LEAF Foundation 
Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

106 Final Report

($200,000), a nonprofit that uses a portion of North Carolina’s Tobacco Master Settlement money to 

support economic development in distressed communities, especially those economically dependent on 

tobacco. The rest of the funding was split among the town of Dobson ($451,968), Surry County 

($411,400), and the Sheltons ($500,000). 

6.7.3 Project Planning and Implementation 

The idea of extending water and sewer infrastructure to the unincorporated area around I-77 Exit 93 goes 

back decades. Surry County officials had always imagined that the county’s highway interchanges would 

become prime sites for commercial and industrial development, but lack of a county water and sewer 

system and the high cost of extending the independent municipal utilities meant that servicing these 

areas would be very difficult. Most of the interstate exits remained unserviced and underdeveloped. 

Before this project, Exit 93 had a small motel, a gas station, a restaurant, farms, and little else, despite 

being one of the flattest and most easily accessible parts of the county. 

By the year 2000, efforts to expand the county’s utility services had increased in urgency. Due to the rapid 

loss of manufacturing firms during the 1990s, the municipal water and sewer systems were operating 

significantly below capacity. Dobson was becoming dependent on a single water customer, a poultry 

processing plant, which consumed over half of the town’s output. In 2001, the county worked with the 

municipalities and an engineering firm, Adam-Heath Engineering, to study the area’s water and sewer 

capabilities, assess the county’s needs, and develop a 20-year capital plan. 

The plan included recommendations for the extension of services to many unincorporated parts of the 

county and interconnections between the municipalities. It also placed a priority on expanding services 

to the county’s highway interchanges, starting with Exit 93 that was tentatively scheduled to receive 

both water and sewer services by 2005. Because the Dobson interchange was just three miles from the 

town’s border, the utility lines would be relatively inexpensive. In addition, the town had a special 

incentive to participate in the project—under North Carolina’s satellite annexation rule, Dobson could 

incorporate the land of any consenting property owners less than three miles away as long as it provided 

them with municipal services. 

An exciting opportunity for development around the interchange came along in 2004, when Ed and 

Charlie Shelton, two wealthy businessmen who had grown up outside Mount Airy, came to the town 

with plans for a large development and a request for utility services. Five years earlier, the brothers had 

established Shelton Vineyards, a vineyard and winery southeast of Dobson. They became increasingly 

devoted to promoting and supporting winemaking and winery tourism in the region. Not long after, they 

helped set up a viticulture and enology program at Surry Community College in Dobson, and took the 

lead in advocating for an American Viticultural Area designation for the Yadkin Valley, which was 

approved in 2003. By 2004, Shelton Vineyards had grown into the largest estate vineyard in North 

Carolina, and the Sheltons were looking to expand the tourism angle of their business further and make a 

bigger impact on the region. 

Their plan for the Village of Shelton Vineyards was to build a large hotel-centered development on about 

80 acres of land they had purchased on the east side of the interchange. The project would require utility 

service to go forward. The original plan for the Village at Shelton Vineyards that was proposed to 

infrastructure funders had four phases. The first, scheduled for completion in 2007, included a 100-room 

all-suites hotel and an upscale restaurant, totaling $7.5 million of investment and providing 62 new jobs. 

The second phase, scheduled for 2008, would invest $5.5 million in a convention center, another 

restaurant, a bank branch, and a drug store. By 2010, they would add offices, a shopping center, another 

hotel and another restaurant, doubling their previous investment, and between 2015 and 2020, they 
Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

Final Report 107

hoped to have added enough retail to bring the project’s total value to $44.75 million, with 496 attached 

jobs. 

The Sheltons’ initial discussions with the town and county about putting in new infrastructure went 

well, and each stakeholder agreed that it would be in their interest to get an extension built. Both the 

town and the developers thought that an annexation arrangement would be in their long-term interest. 

The town would expand its tax base, permanently add a large piece of land with high potential for 

development, and develop a deeper relationship with the Sheltons. The Sheltons would receive all 

municipal services, and they also thought it was more likely that a liquor-by-the-drink referendum 

would pass in the town (one did in 2008). County officials supported the project not only because of the 

direct economic development potential, but also because the extension would add to their long-term 

goals for water and sewer expansion. 

The town, which had taken ultimate responsibility for the project but had few financial resources, began 

to explore options for funding. Early on, it was able to get a preliminary agreement from the Rural Center 

to provide $500,000 of funding. In 2005, the town hired Adams-Heath Engineering, the firm that had 

designed the 2001 20-year capital improvement plan, to conduct a needs assessment, find funding 

sources, and help plan the project. 

The total projected project costs were $2,150,200. The Sheltons agreed to commit $200,000, conditional 

on it being counted as prepayment for assessment, and the town and county each committed to spending 

$203,500. The town hoped to cover the gap with non-local funding, but the grants it received from ARC, 

the Rural Center, and the Golden LEAF Foundation totaled only $900,000. Not wanting to endanger this 

funding, the stakeholders made a deal to split the remaining costs. The town and the county each agreed 

to pay $411,400 and the Sheltons agreed to pay $500,000, in exchange for the right to print the name of 

its development on the new water tower. 

Once zoning was in place at the county and the project was approved, a design was finalized, the town 

hired a construction firm, and the infrastructure project went forward. The construction went relatively 

smoothly, but there were a few “hiccups.” At one point, some of the farmers who opposed the projects 

were getting ready to sue, but the parties were able to sit down together and defuse the issue. The project 

was completed in April 2007, slightly over budget, and the hotel was finished just one month later. 

6.7.4 Economic and Community Impacts 

The Exit 93 infrastructure project has significantly improved the county’s infrastructure, its tourism and 

wine industries, and its capacity for further economic development. Most directly, it allowed the 

Sheltons to build the Village at Shelton Vineyards, a development that brought jobs, taxes, and tourists 

into the area, while adding greatly to its image and its attractiveness as a destination. 

The first stage of the Village at Shelton Vineyards was completed basically as planned, though the 

opening of a restaurant was delayed for several years. The 102-room hotel, a Hampton Inn and Suites 

owned and operated by the Shelton family, includes a wine bar and other features that serve to promote 

the region’s wineries. A service station, which the Sheltons purchased, was rebuilt and reopened as 

Village Market at Shelton Village, a complex that includes a gas station, a Dairy Queen franchise, and a 

unique gift shop and convenience store that sells wine and crafts from the area. In 2011, the Sheltons 

made a deal with BB&T, which built a branch bank near the other businesses. In 2012, they opened Bello 

Vino, an Italian fine-dining restaurant associated with the winery. 

Due to the recession and the ensuing economic slowdown, the plans for retail businesses, a convention 

center, and other new investments have not yet gone forward. The Sheltons now expect that future 
Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

108 Final Report

growth will be at a much slower pace than was originally envisioned, but they continue to look for 

opportunities to build businesses there that would accord with their vision for the area. 

In total, four buildings were built at the Village, and Dobson gained four businesses as a result of the 

infrastructure project. Private investment in the development totaled $18.25 million, surpassing ARC’s 

expected outcome for leveraged private investment, which was $7.5 million. The Village has provided 66 

jobs, which exceeds the ARC’s projection of 62 jobs created. The access and marketing the hotel provides 

has also made a major impact on the Sheltons’ business, greatly enhancing their ability to bring in new 

visitors and bring back old visitors. The hotel has allowed the Sheltons to host weddings at their winery 

and to hold other public and private events, gatherings, and festivals that draw people to the region. This 

increased demand has led to an additional 11 jobs at the Shelton Vineyards winery and restaurant. 

The development has also had a significant effect on the area’s tourism and wine industries more 

generally. About one-third of the hotel’s guests are there primarily to visit area wineries and see other 

attractions. Other guests often come knowing nothing about the area’s wineries and other attractions, 

which make them prime targets for outreach. Since the hotel was built, the number of wineries has 

almost doubled in both Surry County and the Yadkin Valley as a whole, in spite of the recession. One 

business that served many of the hotel guests, a restaurant in Dobson, invested in a major renovation and 

expansion and now hosts weddings and other events. The hotel has also had a catalytic effect in terms of 

other lodging businesses—it was the first of its kind in Surry County, but since it opened, two similarly 

sized hotels have opened. Payroll employment in the county’s accommodations industry has more than 

doubled, from 68 jobs in 2006 to 160 jobs in 2011. The Sheltons believe that their entry into the hotel 

business showed that it could be done in Surry County and convinced others to follow in their footsteps, 

just as they had done with their winery. 

Another important way the project facilitated tourism growth was through the establishment and 

funding of the Dobson Tourism Development Authority (TDA). Before this project, Dobson did not have 

occupancy tax revenue or a TDA. Because of the Exit 93 project, the town was able to establish a TDA, 

and collect a six percent levy on rooms. Each year, the tax brings about $100,000 to the TDA. The TDA, 

which is headed by Ed Shelton, has used the funding to beautify the downtown, build a new town 

website, advertise the town’s attractions, and promote local events such as a fiddlers’ convention. The 

TDA also contributes some of its funding to a new county-level tourism board, which promotes the 

county as a whole, focusing on wine and heritage tourism. 

The project has also led to the strengthening of relationships within the county and the improvement of 

the area’s utilities. When the project began, Dobson was somewhat “self-contained,” even though it was 

the county seat. With an independent, isolated utility system, the town’s board had no real reason to 

work with the county or with any of the other municipalities. This has changed in the years since the 

Exit 93 infrastructure extension was built, and today they have a close working relationship. Dobson has 

worked with the county on several new water projects, most significantly an extension from the town to 

an interstate exit northwest of Dobson, which used the interconnection point built by the Exit 93 

project. The experience gained by the town and the county during the Exit 93 project, which had been 

the only one of its kind, has helped them better plan and implement similar shared projects and intermunicipal connections. 

Finally, the Exit 93 project has had a major effect on the way the town is perceived, an effect 

compounded by the recent construction of the Shelton-Badgett Center for Viticulture and Enology at 

Surry Community College. On the most basic level, it improved the image of the town by cleaning up the 

Exit 93 area, which had previously consisted of dilapidated low-cost construction. Due in part to the 

Dobson TDA marketing, Dobson is now thought of as a potential destination, and some have even termed 

it the “unofficial capital of the Yadkin Valley wine region.” This project has shown local residents how 
Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

Final Report 109

change can be beneficial, and the Sheltons have promoted a vision for the region’s future that many of 

them could get behind. 

6.7.5 Lessons Learned 

 Sometimes it pays to wait. County and town leaders waited for a good opportunity to come along 

before they went forward with plans for a water and sewer extension. In addition to the advantage of 

having attached jobs provided when competing for grants, having some guaranteed new utility 

customers and an increased tax base ensured that at least some costs could be covered. Also, having 

private sector partners as funders and allies was crucial to getting the project completed. The 

Sheltons’ funding, influence, and expertise made the project much easier than it would have been 

otherwise.

 Connect projects to strategies built on a region’s underlying strengths. Surry County has a 

competitive advantage in tourism and wine production, and over the past few years, county and 

municipal leaders have begun to recognize that. By focusing on things like winemaking, specialty 

agriculture, outdoor recreation, and heritage tourism, Surry County has chosen a path that is 

outward looking, but that would, by necessity, preserve or even enhance its rural, small town 

character. Because the Village at Shelton Vineyards was part of this broader strategy, the 

infrastructure extension is having an outsized impact, one that goes far beyond the businesses it 

directly serves.

 Meaningful projects can help a community define its path. For more than a century, Surry County 

was defined by tobacco and textiles. The decline of these industries, particularly the major loss of 

textile manufacturers in the years since 1990, left the county not only without jobs but also without a 

strong identity. The push for viticulture and tourism in the past ten years is not just a strategy for 

bringing dollars into the region by building up certain sectors; it is about a broader vision for the 

county’s future. Perhaps more importantly, it has gotten residents to think of Dobson as an actual 

destination, part of an interdependent regional tourism economy.

 Stakeholders should be truly committed to a project. This project was designed through a process 

that ensured that the town and county governments, the developers, and area residents would all 

support it. The Sheltons, the city, and the county all took part in the project’s planning and 

contributed significant funding to the project. This helped to ensure that everything went smoothly 

and that the final outcome would be to each partner’s liking. When a grant that the town had 

expected to receive did not come through, they all contributed money to fill the gap because they 

considered themselves equal partners. What could have been a major roadblock was just a speed 

bump because each stakeholder had engaged in the planning process and had something to gain from 

the final project.

 Be realistic about outcomes. ARC chose to base its expected outcomes for job creation and private 

investment only on Phase I of the planned developments. This ended up being a wise decision, 

because after the economic recession hit, development stalled, and subsequent phases have not been 

completed. In cases like this where there is a particular development associated with a project, 

construction timelines should be looked at carefully and developers should be clear about how 

circumstances might affect their plans. The Dobson project was successful, but if conservative 

projections had not been used, it might not have been viewed that way. 
Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

110 Final Report

6.7.6 Conclusion 

The Exit 93 infrastructure extension allowed a major piece of development to go forward and provided 

the groundwork for future investment in the new service area. It brought in businesses that provided 

much needed jobs and tax dollars. More broadly, the project boosted two increasingly important parts of 

the regional economy – tourism and winemaking – by increasing the area’s visibility and flow of tourists. 

Despite the fact that the highest hopes for the Village at Shelton Vineyards have yet to be realized, this 

project should be considered a successful example of how highly targeted infrastructure investment can 

spur growth, create jobs, and help a community suffering from manufacturing decline redefine itself as a 

21st century tourist destination. 

6.8 Muscle Shoals Industrial Infrastructure – Muscle Shoals, AL 

The City of Muscle Shoals, Alabama, was awarded an ARC grant to construct utilities and road 

improvements in a new industrial park in order to support construction of a production facility by a large 

auto parts manufacturer. ARC provided funding of $500,000, 18 percent of the total project cost of $2.73 

million. The grant was approved in August 2006, and the project was completed in late 2006. 

6.8.1 Community Profile 

Muscle Shoals is located in northwest Alabama, approximately 20 miles south of the Tennessee border. 

With a population of 13,146 in 2010, it is the largest municipality in largely rural Colbert County. The 

county’s total population in 2010 was 54,428. Colbert and neighboring Lauderdale County comprise the 

Florence-Muscle Shoals Metropolitan Statistical Area, with a 2010 population of 147,137. Muscle Shoals, 

along with the adjacent municipalities of Florence, Sheffield, and Tuscumbia, are known collectively as 

the Quad Cities. Muscle Shoals experienced solid population growth during the 2000-2010 decade, with 

an increase of 10.2 percent, compared to 7.5 percent statewide and 9.7 percent nationally. At the same 

time, Colbert County as a whole experienced a slight population decline. 

Colbert County has experienced improvement in its economic performance during the past decade. 

While employment growth has been modest, with an increase of 2.4 percent during the 2001-2011 period, 

it has still exceeded state and national levels, with declines of 2.2 percent and 0.2 percent, respectively. 

While unemployment rates significantly exceeded both state and national averages at the beginning of 

the decade, the gap was erased by the end of the decade. These improvements are reflected in ARC’s 

upgrading of the county’s economic status from “at-risk” when the project was approved to “transitional” 

by 2012. At-risk counties rank between the worst 10 percent and 25 percent of the nation's counties, 

while transitional counties between the worst 25 percent and the best 25 percent of the nation's 

counties. 

The county has been a strong manufacturing center for decades. Manufacturing employment declined 

during the 1980s and 1990s with the loss of textile firms and manufacturers of products such as chemicals 

and metal products for the Tennessee Valley Authority as TVA down-sized. However, manufacturing 

still comprised almost 20 percent of total county employment in 2011, the greatest share of any industry, 

and actually experienced modest employment growth between 2001 and 2011. Other important 

industries are healthcare and social assistance and retail trade, both of which grew between 2001 and 

2011, healthcare by almost 75 percent. 

The economic well-being of Colbert County residents is, on average, somewhat lower than state and 

national levels, but the county also has a lower proportion of individuals at the low end of the income 

scale than the state. Median household income during the 2006-2010 period averaged $38,610, compared 
Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

Final Report 111

to $42,081 statewide and $51,914 nationally. The poverty rate was 15.7 percentage points, 1.4 percent 

below the statewide level but 1.9 percentage points above the national level. 

6.8.2 Project Description 

The City of Muscle Shoals constructed utilities and road improvements in a new city-owned industrial 

park to accommodate the needs of North American Lighting (NAL), a Japanese-headquartered 

automotive parts supplier, related to the construction of a new 80,000-square-foot manufacturing plant. 

The improvements to the 440-acre park adjacent to the Northwest Alabama Regional Airport included 

widening of a two-lane roadway in front of the park, construction of a new one-quarter-mile access road 

into the property, water lines, sewer lines, and a 500,000-gallon water storage tank. The project was 

initially estimated to create 320 jobs and leverage $21 million in private investment. In addition to the 

$500,000 ARC grant, funding for the $2,731,000 project included a U.S. Economic Development 

Administration (EDA) grant of $1.33 million, a $750,000 grant from the Alabama Department of 

Transportation (ALDOT), and a $151,000 local contribution. 

6.8.3 Project Planning and Implementation 

Having suffered through significant job losses and rising unemployment in the area through the 1980s 

and 1990s, local government officials and economic development leaders recognized that they needed to 

develop a more effective strategy to attract new jobs and diversify the area’s economy. While this would 

require a higher level of regional collaboration, local governments in the area had a history of competing 

rather than cooperating around economic development. 

The first major breakthrough in regional collaboration came when the Alabama Retirement System 

proposed investing in a new hotel, conference center, and professional-level golf courses in Florence. The 

local cost share was too much for the City of Florence and Lauderdale County to cover on their own, so 

the county commissions of Lauderdale and Colbert, along with the mayors of all the two counties’ 

municipalities, agreed to enact a two-cent gasoline tax covering both counties that would stay in place 

until the municipal bonds issued for the project were paid. 

With the successful completion of this project, the local governments were more inclined to work 

together when in early 2005 North American Lighting approached the Shoals Economic Development 

Authority (SEDA), which handled industrial recruitment for the two counties, about the possibility of 

locating its first southeastern plant in the area. Working with SEDA, NAL determined that the best 

location for the plant would be a large parcel of unimproved industrial land acquired by the City of 

Muscle Shoals in 2003 for a new 473-acre industrial park at a cost of approximately $3.3 million. Once 

NAL had satisfied itself that Muscle Shoals was the right location and negotiated an incentive package 

with the state, it sat down with the city to discuss its site preparation needs and negotiate a timetable for 

completing site improvements. The timetable was aggressive – NAL expected to open the plant in late 

2006. The city agreed to clear and level the site with its own funds by April 2006 so that plant 

construction could begin. It did not, however, have the financial capacity to pay for road and utilities 

improvements. 

At this point, the city began to approach potential federal and state funders, including ARC, EDA, and 

ALDOT, for funding of the site improvements. Each of these agencies agreed to fund part of the estimated 

$2.731 million project, leaving the city to provide the remaining $151,000. In addition to taking 

responsibility for the site improvements, the city donated the 40-acre site to NAL. 

NAL also requested local funding for workforce training through the state’s AIDT Program. The city did 

not have the funds to meet this commitment alone so, working through SEDA, it approached the two 
Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

112 Final Report

counties and the three adjacent municipalities to request a cost-sharing arrangement. Recognizing the 

regional importance of the project, all of the other governments agreed to contribute toward the cost. An 

equitable cost-sharing formula was worked out and the six governments contributed a combined amount 

of $1 million. 

With NAL’s aggressive schedule in mind, city officials worked hard to ensure that they could hold up 

their end of the bargain. Once the engineering plans were finalized, they let the project out for 

construction. A working group of the mayor, a representative of the engineering firm, and local public 

works staff met weekly and sometimes daily to review progress and ensure the project remained on 

schedule. One serious challenge encountered because of the aggressive completion schedule was to 

invoice and receive payment from funders to meet the payment schedule of the contractor. While 

receiving disbursements from funders involved paperwork and delays, ARC and other funders showed 

the greatest flexibility possible to keep the project moving. The project was completed in late 2006, 

within the required timeframe and on budget. 

6.8.4 Economic and Community Impacts 

The project significantly exceeded expectations. The initial projections of 320 jobs and $21 million in 

leveraged private investment were met when the plant scaled up its operations to full capacity. Then, in 

2011, NAL doubled the size of the plant and hired hundreds of additional workers, bringing total 

employment to 800 and leveraged private investment to $60 million. These jobs are also relatively wellpaying. 

While the impact of a single plant cannot typically be correlated with improvements in overall economic 

indicators, the impacts can be discernible in a small county like Colbert. While the growth rate for total 

wage and salary employment in Colbert lagged both state and national rates between 2001 and 2007, 

county employment remained stable between 2007 and 2011 while state and national-level employment 

declined by 7.1 percent and 4.4 percent, respectively. Differences in manufacturing employment trends 

are even more pronounced. While manufacturing employment declined significantly at the county, state, 

and national levels between 2001 and 2007, county manufacturing employment increased by 12.3 percent 

between 2007 and 2011, compared to declines of 19.9 percent and 15.4 percent, respectively, at the state 

and national levels. The plant may also account, in part, for the relatively strong population growth 

within Muscle Shoals between 2000 and 2010. 

Demonstrating that Colbert County can be a competitive location for automotive industry suppliers has 

also generated additional automotive industry investment. In September 2012, TASUS, another Japanese 

automotive supplier, announced plans to build a $19.1 million facility with 135 jobs in the FlorenceLauderdale Industrial Park. The area has also received inquiries from other automotive suppliers and 

hopes to attract additional plants. The fact that the area is located within 200 miles of 14 automotive 

assembly and engine plants positions it well to grow its automotive sector. 

The project has solidified collaboration in economic development among the area’s two counties and 

their municipalities. Shortly after the completion of the NAL plant, county and municipal officials 

unanimously agreed to create an area-wide economic development fund through imposition of a one-half 

percent increase in the local sales tax. Funding priorities are decided by a board of local officials from the 

two counties and the four major municipalities, and channeled through SEDA. 

The project has also generated less tangible, but still important, benefits. For example, the local 

community college, working with NAL, has improved its workforce training programs. The increase in 

relatively high-quality jobs may encourage more young people to stay in school and remain in the 

community when they reach adulthood. NAL has become a strong partner with the community in 
Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

Final Report 113

charitable and other civic activities. The community has become more outward looking through its 

contacts with NAL and its Japanese employees and is seeking to position itself as a location for 

additional foreign direct investment. In addition, local officials and economic developers believe that the 

community has gained a more positive view of its future, thus stimulating additional civic engagement 

and investment. 

6.8.5 Lessons Learned 

A number of important lessons emerged from the project: 

 Be prepared for a timely response to economic development opportunities. While a small 

community and region with limited economic development capacity, Muscle Shoals and the rest 

of the Shoals region responded to their economic development challenges proactively. They 

prepared a Community Economic Development Strategy (CEDS) focusing on economic 

diversification, purchased industrial land, and prepared preliminary engineering plans for its 

development. As such, they were well positioned when a major industrial prospect, working 

under its own tight deadlines, imposed a short timeframe for the preparation of an industrial site 

for its new plant. Moreover, local officials organized to exercise close oversight of site 

preparation to ensure that it was completed according to schedule and to work closely with 

funding sources to ensure that funding delays did not slow the project. Had the region not been 

well prepared, the NAL plant may well have located in another community.

 Collaborate as a region. Having competed for industrial prospects in the past, county and 

municipal officials came to realize that in an era of global competition and international 

investment, demonstrating unity and collaboration is critical to attracting firms like NAL. 

Extending the collaborative culture that had begun to gel in the early part of the decade, the 

communities shared financial burdens and together offered a welcoming and supportive 

environment for NAL that helped to persuade the firm that Muscle Shoals was the best location 

for its new plant.

 Maintain close communication with the new firm at all stages of the process. Local economic 

development professionals and government officials noted the importance of maintaining close 

communication with key staff of the new firm, both before the final site selection decision and 

during the development process. They stressed the importance of developing close relationships, 

listening to and understanding the firm’s needs, and solving any problems quickly as they arise. 

As underscored by the NAL executive in charge of the project, “Some of the competitors were 

more intent in telling us how great they were, but didn’t listen to what we were looking for and 

didn’t tailor their proposals to our needs. Folks in Alabama did it right.”

 Work out payment logistics with funders at the outset. Muscle Shoals officials noted the 

difficulty of keeping the project moving on schedule because of delays by funders in processing 

payment requests. While all funders are bound by certain financial constraints, city officials 

noted that funders worked as creatively as possible to keep payments flowing. Working out 

these issues at the outset of the project may help to avoid project delays. 

6.8.6 Conclusion 

The location of the NAL plant in Muscle Shoals greatly contributed to the local and regional economies 

and significantly exceeded expectations. The project created a large number of relatively well-paying jobs 

and demonstrated that the region could compete in attracting automotive suppliers in the southeastern 
Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

114 Final Report

U.S.’s rapidly expanding automotive sector. Key factors in the success of the project were the 

groundwork laid even before NAL approached the community, the collaborative approach of local 

governments, and the strong and responsive relationship developed with the firm. 

6.9 Owingsville/Bath County Industrial Park Wastewater Treatment Plant – 

Owingsville, KY 

The City of Owingsville installed a 300,000-gallon-per-day wastewater treatment plant to meet the 

expansion needs of Custom Food Products, a producer of value-added meat and poultry products 

situated in the Bath County Industrial Park. The ARC provided a $500,000 grant to the project, 

approximately 17 percent of the total cost of $3.04 million. 

6.9.1 Community Profile 

Owingsville, Kentucky, is a small city in Bath County, about 50 miles to the east of Lexington. The city, 

unlike many Appalachian communities, has been fortunate in recent years to experience some population 

growth – up 2.8 percent to 1,530 between the 2000 and 2010 Censuses, though still below growth rates 

for the state (7.4 percent) and the nation (9.7 percent) for the same period. Its population is also older 

and poorer than that of the U.S. as a whole. A full 22 percent of the city’s citizens are over the age of 65, 

nine percentage points higher than the U.S. average. The city’s median income, conversely, at $25,820, is 

less than half of that of the country ($51,914) and barely over two-thirds of that of the Kentucky 

($41,576). Its poverty rate, at 28.4 percent, is also over twice that of the nation (13.8 percent) and nearly 

11 percentage points higher than the state (17.7 percent). 

Bath County has traditionally relied heavily on manufacturing as a source of employment. However, 

manufacturing employment has declined in recent years, including during the past decade. In 2001, it 

comprised 25 percent of total county employment; by 2011, its share of employment had declined to nine 

percent. The loss of manufacturing jobs was the primary factor in an overall decline in county wage and 

salary employment of 19 percent. 

The ARC characterized Bath County, in fiscal year 2012, as being “distressed” or among the “most 

economically depressed counties” in the country; this is a deterioration in its status from 2007, when 

ARC listed the county as “at-risk,” or as an area “at risk of becoming economically distressed.” The 

county’s unemployment rate has remained higher than that of the state by anywhere from seven 

percentage points in 2005 to just over two percentage points in 2012. 

At the time of the Owingsville project, interviewees reported that the local area was already “struggling 

to absorb” the closure of one of three manufacturing plants in the Bath Industrial Park and the loss of 100 

jobs. This made proactive job-retention efforts even more critical. 

6.9.2 Project Description 

The City of Owingsville is the water and wastewater service provider for the Bath County Industrial 

Park. As of 2006, Owingsville’s wastewater treatment facilities consisted of a 200,000-gallon-per-day 

lagoon wastewater treatment plant and 13.5 miles of collection line. Its lagoon treatment plant was at the 

point of exceeding its design capacity with the inflow of industrial waste from Custom Food Products, 

even before discussions about the expansion commenced. The lagoon treatment plant was not designed 

to handle large volumes of industrial wastewater. 
Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

Final Report 115

To accommodate Custom Food Products, the City of Owingsville constructed a 300,000-gallon-per-day 

sequencing batch reactor (SBR) type wastewater treatment plant across from the Bath County Industrial 

Park. The city constructed the plant on land donated by Bath County. Construction included placing 

collection lines to intercept the existing sanitary sewer system, which transported sewage from the 

industrial park to the existing lagoon wastewater treatment plant, and to reroute the flow to the 

proposed new wastewater treatment plant via new sanitary sewer collection lines. The project was to 

include approximately 3,375 linear feet of 6- and 10-inch diameter PVC sanitary sewer lines. 

Construction started in August 2008 and was completed in mid-2010. 

6.9.3 Project Planning and Implementation 

In 2007, Owingsville’s largest employer, Custom Food Products, situated in the Bath County Industrial 

Park, notified city officials that it was looking to expand and was not certain that it could do so at its 

current location. Custom Food Products produces “safe, high quality, value-added meat and poultry 

products” to food processors across the country.

14 That year, one of Custom Food Products’ largest 

customers, ConAgra Foods, itself a multi-national food producer and distributor, asked Custom Food 

Products to expand into the ready-made sausage and biscuit market. This expansion would require not 

only additional space and staff, but also infrastructure that could handle the type of waste generated by 

such production. Custom Food Products spoke with city and county officials, advising them of the 

company’s plans and of its inability to expand at its current location given the sewage infrastructure 

improvements that would be required. 

Owingsville’s wastewater treatment system at the time, a 200,000-gallon-per-day lagoon-style 

processing facility, was primarily designed to treat residential waste, not the waste generated by Custom 

Food’s expanded plant. It was also already running at capacity. 

Owingsville received this news on the backside of several other announcements of local layoffs and was 

already in the process of absorbing the loss of 100 jobs from another plant closing in the Industrial Park. 

Local policymakers met and determined that the city needed to find a way to enhance its wastewater 

collection and treatment system to keep Custom Food Products. The Owingsville mayor and the Bath 

County executive met with Custom Food Products leaders to ascertain the company’s needs, hired 

consultants to help assess the wastewater capacity and needs, and then traveled across the state to view 

other similar wastewater treatment facilities until they found a model that would work for Owingsville. 

A package plant, 300,000-gallon-per-day sequencing batch reactor (SBR) wastewater treatment plant 

was determined to be the best fit for the city’s industrial needs. The cost of the project was estimated at 

just over $3 million. 

The city then hired Bell Engineering to undertake the engineering work required for the package plant. 

To meet the expected $3.04 million cost for the project, city and county officials worked together to 

apply for and receive $1.2 million from the federal government through the Community Development 

Block Grant program; $1 million from EDA; and $500,000 from the Appalachian Regional Commission. 

Another $338,997 was granted from the Kentucky Infrastructure Authority Revolving Loan Fund B. 

Construction started on connecting the package wastewater treatment facility in August 2008 and was 

completed in mid-2010. Construction was reported to have gone smoothly, with one exception – in May 

2010, a significant flooding event caused considerable damage to the plant and the park’s facilities. FEMA 

and contingency funds assisted with clean-up, including the replacement of lab furniture and reworking 

 

14 http://www.customfoodproducts.com/about.php 
Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

116 Final Report

of electrical systems. Even with these added costs, the project ultimately came in under budget, a savings 

that would have been even greater had the flooding event not occurred. 

6.9.4 Economic and Community Impacts 

This project directly supported the retention of Custom Food Products’ 122 jobs at the Bath County 

Industrial Park. The firm also created, as promised, 60 new jobs as a result of its expansion, plus an 

additional 42 jobs in subsequent expansions, 53 of which are reported to have gone to low- to moderateincome individuals. As a whole, investing in the infrastructure to retain an expanding major area 

employer resulted in 122 jobs retained and over 100 new jobs created. The plant’s payroll before the 

project was at $3.6 million, and has since increased by another $2.2 million. Custom Food Products also 

invested $26.2 million to expand its physical plant. 

The construction of a second wastewater treatment facility also allowed the city’s primary sewer line to 

absorb businesses and residences that did not have service. This included the Bath County Educational 

and Marketing Center, a $400,000 United States Department of Agriculture Rural Business Enterprise 

Grant recipient, designed to give local farmers the ability to produce and market their value-added 

goods15 as well as 105 homes just across the county line (though still in the sewer district). Additionally, 

the new capacity allowed the Ridgeway Nursing and Rehabilitation Facility to expand, increasing its 

beds from 60 to 99. These new connections added tax revenue to city coffers and benefitted local 

businesses and individuals who needed proper sewage connections. 

While the Custom Food Products facility is currently the only tenant in the Park, Owingsville Mayor 

Gary Hunt is optimistic about future growth, especially with the newly enhanced infrastructure. “One of 

the factories is going to sell at auction,” he commented. “There has been lots of interest.” Having the 

second sewer plant with lines that can be re-run to easily and effortlessly accommodate such growth, he 

noted is “great insurance” for the future.

Despite the recent economic hits, Mayor Hunt reported, “The last few years have been very progressive.” 

Beyond the Custom Food Products expansions, Owingsville has “improved its downtown, ...[instituted] 

a new streetscape program, new sidewalks, new lights,” often through other ARC funding. Since it 

started these revitalization efforts, supported by tax dollars from employers like Custom Food Products, 

Owingsville has seen several new businesses open in the city’s downtown, and it has started to see its 

population begin to grow, giving the city and the area great hope for additional future development. 

6.9.5 Lessons Learned 

 Investing in infrastructure can play a critical role in municipalities retaining large employers.

Municipal investments in infrastructure projects can both retain current jobs and position a 

community for future economic growth. In the Owingsville case, Custom Food Products required the 

wastewater infrastructure in order to expand. Without it, the firm would have moved elsewhere. The 

quick reactions of city and county officials, working together to draw up appropriate plans and find 

the appropriate financing to pay for the project, resulted in the construction of the necessary 

infrastructure to retain one of the area’s largest employers. Without it, Custom Food Products’ 122 

jobs would have been lost, and the area would not have benefited from the 100-plus additional jobs 

the firm has since created through its expansions. Another $2.2 million has also been added to local 

payrolls. 

 

15http://blogs.usda.gov/2010/08/31/new-marketing-center-for-local-producers-opens-in-kentucky-thanks-torecovery-act-support-from-usda/
Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

Final Report 117

 Increasing infrastructure capacity can have many indirect benefits. While Custom Food 

Products’ requirement for increased sewage capacity was the main driver for the addition of the new 

plant, it was not the only one. According to the city’s official project summary, during the period of 

July 2003 through February 2006, the city’s lagoon plant exceeded its design capacity in 17 of the 32 

months. Residences and even some local businesses were not able to be connected to the city sewer 

due to these capacity constraints. With the addition of a second plant, providing excess capacity, 105 

new homes and the Bath County Agricultural Complex have been added to the sewer system. 

Increased sewage capacity also enabled the Ridgeway Nursing and Rehabilitation Facility to 

increases its beds. Further, as Owingsville’s current mayor noted, it also left the Park more attractive 

to future businesses looking at the area as a possible plant site: “[We were] very appreciative to get 

the sewer plant. Once you’ve got it, you’ve got it. While it’s not at full capacity [now, it is] still a good 

selling point for future growth.”

 Developing systems for tracking funding and ensuring that funds are used appropriately can be 

an invaluable project management technique. The project’s grant manager from the area’s local 

development district stated that one of the most valuable steps she took through the process was a 

small, simple one done from the start: tracking the funding sources and their use. “Before we 

submitted the first application, we invited all funders to the table. We sat down around the table and 

brainstormed and talked about how we could best approach this,” she recalled. “When you’re trying 

to get something done quickly, you cannot afford to make mistakes. For example, if you get half-amillion in CDBG funding...it needs to be used in a particular way. You can’t use that for engineers, for 

instance. [So we tracked] the money. I designed a grid and tracked funding across components. It 

also saved time, and allowed people [coming in] who weren’t [already] familiar with the project to 

take over, if needed.” Funding requirements are often complicated, and become only more confusing 

when multiple funds are combined in one pool. Having simple processes from the start of a project to 

clearly depict which funds can be used for which parts of construction can help a project save time 

and money. 

6.9.6 Conclusion 

The ARC grant allowed the City of Owingsville to build the infrastructure necessary to retain one of its 

leading employers in the Bath County Industrial Park and enabled the firm to proceed with its expansion 

plans. Custom Food Products continues to thrive, and has since nearly doubled its employment to over 

200. Bath County and Owingsville have both benefitted from increased employment tax revenues. 

Further, the extra capacity from a second sewer line has taken pressure off the city sewer system, 

allowing other entities that previously did not have service, including 105 homes and an agricultural 

center, to connect to the infrastructure (and pay related fees). A local nursing home, given the ability to 

expand its sewer use, was also able to increase the number of beds available. 

Bath County and the City of Owingsville have faced considerable challenges over the past decade, 

including high poverty rates and low median incomes, but local officials’ ability to work together to 

retain businesses, supported by ARC funding, has helped to mitigate jobs losses and set the stage for new 

economic activity. 

6.10 Powdersville Water Storage Tank Project – Powdersville, SC 

Powdersville Water District (PWD) received ARC funding in 2008 to construct a one-million-gallon 

above ground water storage tank to better serve customers in and around the community of 
Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

118 Final Report

Powdersville, South Carolina. The primary purpose of the project was to support business development 

and job creation in the area, particularly along a high-growth highway corridor spanning parts of 

Anderson and Pickens Counties. The ARC awarded the project a $500,000 grant, 30 percent of the total 

budget of $1,687,000. 

6.10.1 Community Profile 

Powdersville is an unincorporated community in Anderson County near the Pickens County line. 

Anderson and Pickens Counties are part of the eight-county Greenville-Spartanburg-Anderson 

Consolidated Statistical Area, located in the northwestern corner of the state on the Georgia border. 

Powdersville is a fast-growing suburb of Greenville, which lies 10 miles to the northeast. Its population 

increased 42.1 percent between 2000 and 2010, to 7,618. The combined population of Anderson and 

Pickens was 306,350 in 2010, an increase of 10.8 percent from 2000. This compares to growth rates of 15.3 

percent and 9.7 percent, respectively, at the state and national levels. 

While both Anderson and Pickens Counties experienced population growth during the past decade, 

employment declined. Between 2001 and 2011, wage and salary employment declined by 9.8 percent in 

Anderson and by 3.9 percent in Pickens. By comparison, employment at the state and national levels 

declined by only 0.2 percent and 0.3 percent, respectively. At the same time, unemployment rates during 

this period averaged only slightly higher in Anderson than statewide and actually lower in Pickens. This 

likely reflects the fact that many workers in both counties commute to Greenville and elsewhere to find 

work. Both counties’ unemployment rates were still consistently above U.S. rates, as was the state’s.

The largest industries in both counties are manufacturing, including many automotive suppliers to the 

BMW assembly plant in nearby Spartanburg, retail trade, healthcare and social assistance, and 

accommodation and food services. Industry trends in both counties have been similar, with 

manufacturing declining dramatically between 2001 and 2011 (by almost one-third in both counties), 

retail declining slightly, and both healthcare and social assistance and accommodation and food services 

expanding considerably. 

Powdersville is a relatively prosperous community, much more so than the rest of Anderson County or 

Pickens County. Its median household income during the 2006-2010 period averaged $60,616, while its 

poverty rate averaged only 6.4 percent, far better than state and even national levels. In contrast, these 

indicators for the two counties were about equal to state levels and considerably less favorable than 

national levels. Both counties were classified by ARC as “transitional” in 2008 when the ARC grant was 

awarded and again in 2012. A transitional county is defined as economically positioned between the 25 

percent most distressed and 25 percent most robust U.S. counties. 

6.10.2 Project Description 

The project involved the construction of a one-million-gallon water storage tank with appurtenances 

along a designated “high-growth corridor.” The tank was needed to comply with state regulations 

involving fire flow protection (the amount of water that should be available for providing fire protection 

at selected locations) for existing residences and businesses. By increasing system capacity, it also 

enabled the PWD to support planned residential and commercial development and accommodate future 

development. In addition to the $500,000 ARC grant, the remaining costs were covered by a $1.187 

million low-interest loan from the South Carolina Department of Health and Environmental Control’s 

State Revolving Loan Fund (SRF). 
Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

Final Report 119

6.10.3 Project Planning and Implementation 

The Powdersville Water District has almost 12,000 taps and serves over 30,000 people, mostly in 

Powdersville and the surrounding area of northeast Anderson County, with approximately 80 percent of 

its customers. It also serves a small portion of southern Pickens County, with the remaining 20 percent of 

its customers. PWD played the lead role in the planning process, which began in late 2007. 

Two factors primarily drove the decision to undertake the project. First, there were concerns about 

water pressure differentials in the system. The PWD service area is hilly, resulting in three different 

pressure zones. One zone had very low pressure at the time, while one had much higher pressure. This 

resulted in highly uneven pressure in different areas. The region needed more water storage in order to 

equalize pressure throughout all the zones, increase the reliability of the system, and meet state fire flow 

regulations. Second, rapid population growth in the area was generating significant increases in water 

demand. Increasing capacity would enable the PWD to expand water storage capacity to meet the 

increasing demand associated with new business and residential development. Major development 

projects were already underway at the time, including a medical facility, a hotel, and two educational 

facilities – a high school and a middle school. 

Dyke Spencer, the PWD’s general manager, oversaw the implementation of the project, assisted by 

Gordon Brush, the director of finance and administration. PWD enlisted BP Barber, an engineering firm 

based in Columbia, South Carolina, to design the tank. BP Barber had a long history of work with PWD 

over 25 years and, therefore, had already established a strong working relationship with the district. BP 

Barber created a model of the area as well as measurements and elevation data taken from the region in 

order to create a model for designing the project. Dirk Reis, the grant services administrator for the 

Appalachian Regional Council of Governments, assisted the PWD in preparing funding proposals to 

ARC and the SRF. 

The storage tank was originally designed with a 750,000-gallon capacity, but was increased by 250,000 

gallons during the bid process when it became apparent that the recession had significantly depressed 

material and labor costs, making it possible to build increased capacity to meet future growth needs at a 

reasonable additional cost. The expansion increased the cost of the project by $447,000, from $1.24 

million to $1.687 million. The initial SRF loan of $740,000 was increased to $1.87 million to cover the 

additional project costs. Construction commenced in early 2009 and took 18 months to complete. The 

tank was put online in August 2010, well ahead of the projected completion date of date March 31, 2011. 

6.10.4 Economic and Community Impacts 

To date, the project has had a significant impact on the community. The increased water supply, speed, 

and reliability have helped to stimulate growth, particularly along the designated high-growth corridor. 

Job creation associated with the project has greatly exceeded original projections. Initial job creation 

estimated in the ARC grant proposal was 80. By the time the final closeout report was submitted, the 

grantee reported that the project had already created 110 jobs. And since the completion of the project, at 

least nine more businesses with over 300 employees have opened. The corridor has seen considerable 

growth since the project was completed, due in large part to the water system’s ability to accommodate 

more businesses. Among the new developments are several gas stations, a pharmacy, a value retail store, a 

restaurant, an engineering firm, and a manufacturer. Other businesses have expanded. 

New business development has also improved the quality of life for area residents by increasing the 

availability of retail and service businesses to serve the area’s rapidly growing population. Before the 

project was completed, the area lacked a sufficient retail and service base to meet the needs of the 
Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

120 Final Report

community. The jobs created also enable more area residents to work closer to home rather than having 

to commute to Greenville or other urban centers to find employment. 

More generally, the project may position Anderson and Pickens Counties to capture more regional 

employment growth. While employment in both counties has declined during the past decade, 

employment in adjacent Greenville County has increased. With increased water capacity, more regional 

businesses may consider the parts of Anderson and Pickens Counties served by the PWD more desirable 

locations to start up or expand. 

Overall, the community and those involved directly in the project consider it a success in terms of 

increased employment opportunities, additional tax revenues, and higher quality of life. Moreover, 

incorporating additional capacity has created room for additional business and residential development. 

6.10.5 Lessons Learned 

Those involved in the project cite a few lessons learned from their experience: 

 A long-range perspective is needed to anticipate future infrastructure needs, particularly in 

growth areas. Those involved in the project came away with the understanding that it is 

important to always look to the future and what will be required to accommodate future growth. 

By anticipating expansion in the Powdersville area, especially as it has increasingly taken on the 

character of a bedroom community to the City of Greenville, it only made sense that more water 

storage capacity would be required to support this growth. Positioning the project to serve an 

area already designated a high-growth corridor also ensured that growth would be channeled to 

areas that were equipped to accommodate development. 

 A larger initial investment may save money in the long run. In some instances, it makes sense 

to design an infrastructure project with spare capacity, even if it is not needed to meet immediate 

needs. By taking advantage of low construction costs during the recession, the PWD was able to 

build in the additional capacity that will very likely be needed in the not-too-distant future at a 

reasonable cost. Waiting to expand the system until economic growth accelerated would likely 

have cost PWD customers more in the long run. 

6.10.6 Conclusion 

Overall, the Powdersville water storage tank project can be considered a sound investment. Prior to this 

project, the area did not have a sufficient, reliable water supply to support the commercial development 

necessary to serve a growing population. The construction of the one-million-gallon water storage tank 

created over 100 jobs initially by facilitating the development of planned projects such as the expansion 

of the area high school and middle school, a medical facility, and a hotel. Over time, additional 

development has occurred, and more jobs have been created. With the expanded water system, Anderson 

and Pickens Counties, which have been lagging behind adjacent Greenville County in employment 

growth, have greater capacity to support business development, laying the groundwork for further 

employment growth and increased quality of life. 

6.11 Racine Water Treatment Plant and Storage – Racine, OH 

The Village of Racine, Ohio, received funding to replace its deteriorating water facilities with a new 

300,000-gallon-per-day water treatment plant and a 394,000-gallon ground water storage tank, and in 

response to flooding during the fall of 2004, to raise the system’s wellhead and install new electrical 
Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

Final Report 121

systems, allowing for uninterrupted water service for residents during such events. The system was 

designed not only to serve the 315 households in the community, but also to allow shovel-ready 

investment to occur and to position the village for longer-term economic development. The ARC 

provided funding of $457,000, 23 percent of total project costs. 

6.11.1 Community Profile 

The Village of Racine, Ohio, is a small rural community in Meigs County in southeast Ohio. Located 

along the Ohio River, just across from West Virginia, it is a community of 675 individuals, mostly lifelong 

residents. The village’s population has declined by 9.6 percent between 2000 and 2010, compared to 

growth of 1.6 percent statewide and 9.7 percent nationally. 

Racine’s citizens largely find employment outside the village, traveling north to the University of Ohio, 

west to two aluminum plants, or to one of several nearby power plants. Locally, many residents find 

employment within the Southern Local School District, based in Racine. The District, which serves 720 

students, includes an elementary school (K-8) and a high school (9-12). The schools are widely 

considered to be the center of the community, and the school district regularly works with local leaders 

to make the area more hospitable for growth. 

Meigs County was categorized as a “distressed” county by ARC in fiscal year 2013, as it was when initial 

ARC funding for the water project was approved in 2004, making it among the “most economically 

depressed counties” in the nation. The county’s unemployment rate has consistently exceeded both state 

and national averages by significant margins during the past decade. Between 2001 and 2011, 

unemployment averaged 10.7 percent, about four percentage points above both state and national levels. 

In 2011, it stood at 13.5 percent. The county has also experienced significant job loss during this period. 

Between 2001 and 2011, total employment declined by 26.1 percent, compared to declines of 8.6 percent 

and 0.2 percent, respectively, at the state and national levels. 

At the village level, median housing income averaged $31,250 during the 2006-2010 period, only 66 

percent of the statewide average and 60 percent of the national average. The village’s poverty rate of 

nearly 35 percent is more than twice that of both Ohio and the U.S. 

6.11.2 Project Description 

The village constructed a new 300,000-gallon-per-day-water treatment plant and a 394,000-gallon 

ground water storage tank. With a grant supplement, it was also able to make necessary wellhead 

improvements, including raising the wellhead and installing a new electrical service. These investments 

allowed the village to have regular access to clean, treated water, even during floods, with lower longterm maintenance costs. It also gave the village the capacity to handle future growth. The project’s final 

cost of $1,956,900 was funded by a combination of federal, state, and local dollars, including two ARC 

grants totaling $457,000, a $500,000 grant from the federal Environmental Protection Agency, an 

$865,000 State Small Cities Community Development Block Grant (CDBG) grant, and a $124,900 local 

match, which combined a local cash contribution with a loan from the Ohio Water Development 

Authority. 

6.11.3 Project Planning and Implementation 

In the early 2000s, Racine undertook a master planning process to combat the area’s population loss and 

create a stronger foundation for future economic growth. One of the areas identified as being most in 

need of investment was the village’s water system. Its water plant required extensive and expensive 
Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

122 Final Report

ongoing maintenance for normal operations, was already operating 23 to 24 hours per day, and had a 

severely corroded aeration system that did not meet industry standards. In addition, the system did not 

fully meet the village’s water needs, requiring additional water to be purchased from other communities. 

When a developer was evaluating the area as a possible site for a new power plant, the village 

government, led by Mayor Scott Hill, a strong infrastructure advocate, started meeting with various 

community stakeholders and possible funders to discuss a permanent water solution. Hill believed that 

investments in infrastructure were essential for the village’s future. Beyond the possibility of the power 

plant being sited near Racine, Mayor Hill also believed that the investment was essential to attracting 

new residential and commercial growth and to more effectively serving existing users. The elementary 

school, built in 2000, already needed to have a new pump installed to improve its water pressure, and 

then had to replace its boilers due to the hardness of the water. With the possibility of a major new 

investment in the high school, the water project was imperative. 

With water rate increases not a feasible option for financing a project expected to cost well over a million 

dollars, Hill began to seek external funding. Racine had a history of receiving state and federal funds, and 

of efficiently managing these funds, making Hill’s task easier. Understanding a funder’s reluctance to 

commit funding without support from other funders, Hill also brought funders together on several 

occasions in Racine to see the issues at hand. 

From federal, state, and local agencies, Hill was able to raise $1.76 million for the project, including 

$300,000 from ARC, to build the water treatment plant and storage tank. The investment would enable 

the village to service the needs of the elementary school and other public facilities, existing homes and 

businesses, and to provide the capacity for future growth in and adjacent to the village. 

In September 2004, while the project was in the design phase, serious local flooding caused significant 

damage to Racine’s water supply well and exposed chronic weaknesses in the system that, even with the 

new investment, would not be alleviated. The existing wellhead on the system was prone to flooding, as 

often occurs along the Ohio River, leaving residents without service while supplies and treatment 

facilities are brought back online. Hill once again approached funders and asked for an expansion of the 

project to address this issue during the planned construction, proposing a raised wellhead and new 

electrical service. The supplemental investment was expected to allow Racine the use of its water supply 

during floods, eliminating service disruptions. To the revised proposal, ARC agreed to contribute another 

$157,000, and the village increased its contribution by $40,000, bringing the total project cost to 

$1,956,900. 

The water treatment plant construction generally proceeded smoothly, although Mayor Hill noted 

occasional issues with the consulting engineers. He attributed this to the previously locally owned firm 

being acquired by a larger non-local firm with less of an interest in making the project anything more 

than a “cookie-cutter” exercise. However, by project’s end, Hill believed the community received what 

was originally promised. 

6.11.4 Economic and Community Impacts 

The village has experienced considerable development since the completion of the project. Among the 

most notable developments: 

 The opening of a Dollar General in Racine’s downtown, bringing residents a nearby and affordable 

shopping option; 

 The construction of a new shopping center containing several retail establishments; 

 Construction of a new $2 million branch by the local bank; and 
Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

Final Report 123

 The recent news that a housing developer was planning to build 15 new homes on vacant land, a 

substantial expansion of the residential stock for a community of 315 households. 

These projects would likely not have been undertaken without the improvements to the water system. 

The improvements also facilitated a $10 million investment to renovate and expand the local high school, 

enhancing a major community anchor. 

This development activity speaks to a growing confidence in the area’s potential, all of which, according 

to Perry Varnadoe, director of the Meigs County Economic Development Office, would not have been 

possible without the water system investment. He credits 30 retained and 25 new jobs to the project. He 

also notes that it has made Racine a growth center within the county. “Small towns are always on the 

precipice of going one way or the other. Racine is tipping in the right direction.”

Another benefit of the project is cost savings for the village government. It is now incurring significantly 

lower maintenance costs while servicing more customers. 

6.11.5 Lessons Learned 

Racine’s investment in its new water system has put the community on sounder footing for growth. 

Participants in the project took away a number of lessons: 

 Take advantage of free or low-cost resources when putting together a project proposal. Racine, 

a small community under significant financial limitations, did not have the resources to hire a 

consultant or engineer to develop a comprehensive project plan to present to potential investors. 

Instead, Mayor Hill turned to the Rural Community Assistance Partnership (RCAP), a national 

nonprofit that provides technical assistance to rural communities. It provided Racine with the 

technical assistance and support its leaders needed to put together a viable and credible proposal. 

With RCAP’s assistance, Racine was able to garner the investments it needed for a system that 

would suit its needs.

 To avoid “first funder” hesitation, where possible, convene a meeting of all potential funders.

For projects similar to the one undertaken by Racine, finding a first funder willing to commit money 

to a project is a chronic challenge. In Racine, to overcome this obstacle, Mayor Hill invited all of the 

funders to the village simultaneously to see the need and to discuss the collective resources they had 

available to invest. The community was then also able to demonstrate its own buy-in to the project. 

Racine raised nearly all of the funds it required to fund the water treatment plant and storage facility, 

including those supplemental funds needed after the 2004 flood. 

 Community buy-in is essential for success. While small communities’ investments in projects like 

that of Racine may comprise a relatively small share of total project costs, they can be quite 

substantial on a per-person basis. In Racine, the $124,900 share in loan and local cash contribution 

amounted to over $185 per person. With average household income around $31,000, an investment of 

this magnitude is not insubstantial, especially when other needs are still left unfulfilled. Recognizing 

this, Mayor Hill started “knocking on people’s doors” to get their thoughts and to make the case for 

the project. He later also held several information sessions about the plan. According to Tony Deem, 

superintendent of the Southern Local School District, the mayor “spent a lot of time speaking to a lot 

of people, as he does with any sort of project. He then focuses on what people think needs to be done, 

not what he thinks.” By proactively approaching the citizens of Racine, Mayor Hill not only was able 

to get their input, but also their support, a critical ingredient in any successful project. 
Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

124 Final Report

6.11.6 Conclusion 

The investment in Racine’s water treatment and storage infrastructure gave the area a basic foundation 

upon which additional growth and investment could occur. Thanks to this project, an estimated 55 jobs 

have been created or retained, 15 new homes are being built in a community that has seen little 

investment in new housing stock in decades, the high school is able to undergo a $10 million renovation 

and expansion, and commercial development, including the addition of a new Dollar General downtown 

and a $2 million bank branch location, has occurred. Paired with another of Mayor Hill’s initiatives to 

move the main business district further from the flood zone, Racine has taken significant steps to 

position the village for future private-sector investment and employment growth. 

6.12 Wastewater Treatment Solutions for Small Communities – Canaan Valley Institute, 

WV

The Canaan Valley Institute received ARC funding to support the development of decentralized 

wastewater treatment facilities through technical assistance to nonprofit organizations and public 

entities in small, unincorporated communities in three counties of southern West Virginia’s coal region. 

ARC provided a grant of $96,811, 55 percent of the total cost of $176,865. The grant was awarded in 

October 2008 and the project was completed in September 2010. 

6.12.1 Community Profile 

The project focused on a rural area of southern West Virginia encompassing McDowell County, western 

Raleigh County, and Wyoming County. In 2010, the three counties had populations of 22,113, 78,859, and 

23,796 respectively. 

The area’s economy was dominated by the coal mining industry for many decades and has continued to 

remain highly dependent on coal even as mining employment has declined dramatically. The loss of 

mining jobs beginning in the mid-20th century was accompanied by dramatic population loss as working 

residents left the area in search of other employment. McDowell County, once the highest coal producer 

in the U.S., was most severely impacted, losing an extraordinary 78 percent of its population between 

1950 and 2010. Wyoming County experienced a lesser but still substantial decline of 37 percent. Raleigh 

County, more economically diversified and urbanized, experienced a relatively small loss of 18 percent. 

Population decline in McDowell and Wyoming Counties continued between 2000 and 2010, by 19.1 

percent and 14 percent respectively, while the population of Raleigh County remained stable. In contrast, 

population at the state and national levels grew by 2.5 percent and 9.7 percent respectively. 

Economic data underscore the continuing importance of coal mining to the area’s economy. In McDowell 

and Wyoming Counties, mining comprised 37.8 percent and 24.8 percent of total private employment 

respectively in 2011 and remained the largest single private employer by a wide margin. In more 

diversified Raleigh County, mining still comprised 7.9 percent of total private employment, behind only 

retail and health care. Moreover, general employment trends continue to be strongly influenced by the 

fortunes of the mining industry. Employment growth in McDowell County between 2001 and 2011 was a 

strong 20.3 percent, driven largely by a more than doubling of mining employment. Even in diversified 

Raleigh County, total employment growth of 8.1 percent was strongly influenced by mining employment 

growth, which was second only to health care. In Wyoming County, which experienced a decline in total 

employment of three percent, mining employment grew relatively modestly, too little to counter declines 

in most other industry sectors. Both McDowell and Raleigh counties outperformed the state and U.S. in 

employment growth while Wyoming County underperformed them. 
Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

Final Report 125

The economic status of area residents varies considerably by county, although all three counties lag 

national averages in key indicators of economic well-being. McDowell is the most severely distressed 

while Raleigh is the most prosperous, although the western part of the county included in the project 

area is less so. In terms of employment status, 2011 unemployment rates range from somewhat higher 

(McDowell) to somewhat lower (Raleigh) than state and national averages. More significant, however, 

are labor force participation rates, which show the percentage of adult residents working or actively 

seeking work. These are extremely low both in McDowell and Wyoming Counties, 31.2 percent and 42.2 

percent respectively during the 2006-10 period compared to 55 percent statewide and 65 percent 

nationally. Even Raleigh, with a rate of 50.6 percent lags state and national averages. This indicates that a 

disproportionate number of residents have simply dropped out of the labor force. 

In terms of income levels, McDowell County again fares most poorly. Its median household income of 

$22,154 during the 2006-10 period was only 58 percent of the state average and 43 percent of the national 

average, and the poverty rate of 32.6 percent was almost twice the state average and more than twice the 

national average. In Wyoming and Raleigh Counties, median household incomes and poverty rates were 

roughly on par with state levels but significantly below national levels. 

When the project was initiated in 2008, ARC classified McDowell and Wyoming as “distressed,” and 

Raleigh as “transitional.” In 2012, McDowell and Raleigh held the same classifications while Wyoming 

had improved to “at-risk.” Distressed counties rank among the worst 10 percent of U.S. counties, at-risk 

counties between the worst 10 percent and 25 percent, and transitional counties between the worst 25 

percent and the best 25 percent. 

While mining employment ticked upwards during the 2001-11 period, state and local government 

officials and economic development professionals have recognized the need to diversify the area’s 

economy to compensate for the long-term decline of coal mining employment, stem population outflows, 

and create new employment opportunities. Efforts have been underway to increase recreational and 

cultural tourism through the development of a large network of all-terrain vehicle (ATV) trails and the 

187-mile Coal Heritage Trail, improving state park facilities, promoting local artisanship, and other 

initiatives. Still, the area faces significant barriers to diversification, including poor highway access, 

limited telecommunications infrastructure, and gaps in water and sewer services, particularly outside 

larger municipalities. 

6.12.2 Project Description 

ARC provided funding to the Canaan Valley Institute (CVI) to support the development of decentralized 

wastewater treatment facilities in small, unincorporated communities in the three counties. Most homes 

in these areas are not connected to wastewater treatment facilities and are discharging raw sewage into 

streams, which have become severely impaired by fecal coliform and related contaminants. In many cases, 

centralized sewage systems remain too expensive and impractical to develop and maintain or extend 

from existing municipal systems. CVI worked with local non-profit organizations and public entities to 

support planning and design of non-traditional waste collection and treatment systems, developing 

management structures, training system installers and operators, and securing funding. In addition to the 

ARC grant of $96,811, CVI contributed a local match of $80,054, for a total project cost of $176,865. 

6.12.3 Project Planning and Implementation 

Major flooding that occurred in the area in 2002 created the initial impetus for the project. As rivers and 

streams overflowed their banks, water contaminated by raw sewage flowed into area homes, 

compounding water damage and cleanup costs. The damage was particularly severe in the many small, 
Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

126 Final Report

unincorporated, former mining communities spread throughout the area. Some area residents, searching 

for ways to reduce the impact of future floods, formed citizens’ organizations to explore solutions. One of 

these groups, the Wastewater Treatment Coalition of McDowell County (WTCMC), identified CVI as a 

source of technical assistance, and approached it for help in 2003. 

CVI is a nonprofit organization that assists small communities throughout West Virginia and western 

Maryland to address wastewater treatment and wetlands restoration needs. Its areas of expertise include 

engineering, project development, grants management, training, and GIS services. Its initial work 

involved assisting WTCMC and another citizens’ organization, the Upper Guyandotte Watershed 

Association (UGWA), to prepare wastewater treatment plans for their areas. The plans identified where 

wastewater treatment was needed and what types of solutions were possible, developed action steps, 

and established priorities. Because many small communities were too far from larger communities with 

municipal wastewater treatment systems to make extension of these systems financially feasible, and 

they were located in narrow valleys with insufficient or unsuitable land area for septic systems, the plans 

focused heavily on the use of self-contained “package” wastewater treatment systems designed to serve 

small clusters of users. 

CVI also assisted the UGWA to develop its first pilot project in Helen, a town of about 200 residents in 

Raleigh County. Its role included assisting in selecting the most appropriate treatment system, 

customizing the design of the system, community outreach efforts, engaging with county government 

and state regulatory agencies to obtain necessary funding support and regulatory approvals, and working 

with the area’s public service district (PSD) to enlist its participation in developing and managing the 

system. Eighty-four residences in the community were ultimately connected to the new system. 

In West Virginia, PSDs are autonomous public agencies appointed by county governments and 

responsible for providing water and sewer services outside communities with municipal systems. The 

PSDs in the project area had typically focused primarily on improving water service and were largely 

unfamiliar with package wastewater treatment system technologies. CVI’s role in educating the PSDs 

about package systems and demonstrating the feasibility of installing systems in particular locations was 

therefore critically important. 

This initial work laid the groundwork for the ARC-funded project. Working with the UGWA and the 

WTCMC, CVI targeted six communities across the three counties for design, funding and installation of 

self-contained wastewater treatment systems, and planned to assist at least 20 homeowners in selected 

areas to install individual septic systems. While the goal was to have all of these projects completed 

within five years, individual milestones to be achieved by the end of the grant period were established for 

each community and for the septic tank component. 

The initial grant period was one year. However, CVI’s work progressed more slowly than first 

anticipated, and it requested and received from ARC a one-year no-cost extension. A number of factors 

contributed to the delay, primarily related to the unconventional nature of the package systems. These 

included the challenge of identifying suitable sites and tailoring the design of each system to the unique 

geological and topographical conditions of the site, overcoming reluctance of some PSDs to participate in 

projects, working with regulatory agencies to ensure that the systems met regulatory standards, and 

obtaining needed funds for project design and construction. 

By the end of extended grant period, a new system had been installed in one of the six communities, 

systems were in various stages of development in the five others, and 46 septic tanks had been installed, 

meeting or exceeding all of the established milestones. In addition, CVI assisted in initiating projects in 

three other communities that were not included in the project work plan. 
Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

Final Report 127

6.12.4 Economic and Community Impacts 

CVI, working with its local partners, achieved the following outcomes related to its initial work plan. 

Some of these outcomes reflect progress since the end of the grant period: 

 Installed wastewater collection and treatment systems serving a total of 27 homes and one 

business in the communities of Ashland and Ury. 

 Helped 50 homeowners install residential septic systems with loans from a state low-interest 

loan program. 

 Completed design and obtained project funding for extension of the City of War’s municipal 

water and sewer systems to the communities of Shop Hollow and Centerville, serving 56 homes. 

As of late 2012, these projects were ready to bid. 

 Completed detailed engineering design for a wastewater collection and treatment system serving 

42 homes in the community of Amigo. Partial funding had been obtained for the project as of late 

2012. 

 Completed project plans and a preliminary engineering report for a wastewater treatment and 

collection system serving 129 homes in the community of Crumpler. 

When completed, these projects will serve a total of 304 homes and one business, 29 more homes than 

projected in the ARC grant agreement. UGWA and WTCMC members acknowledge that CVI’s 

assistance has been critical in moving these projects forward. 

Other projects that CVI supported during the grant period are also progressing. These include 

wastewater treatment and collection systems serving a total of over 670 homes in the communities of 

Coalwood, Iaeger, and McDowell. CVI is also working with the UGWA to expand Wyoming County’s 

wastewater treatment plan to cover the entire county. 

In addition to improving the quality of life for area residents, these initiatives are beginning to yield 

significant economic impacts that will grow as more of them are completed. First, they are supporting 

tourism development. As streams that used to receive untreated sewage experience reductions in 

coliform bacteria levels, they are again becoming suitable for fishing. In Ashland, a few homes have been 

remodeled into tourist cabins for ATV visitors. And the reduction of odor from raw sewage has made 

communities with Coal Heritage Trail sites more appealing to visitors, potentially promoting the 

development of tourism-related businesses such as restaurants and artisan shops. 

Second, these initiatives are helping to strengthen the area’s housing market. Without sewage service, 

homes cannot legally be sold, contributing to housing disinvestment and abandonment. As homes are 

connected to new sewage systems, homeowners are now able to sell them legally, increasing their value, 

encouraging better upkeep, and enabling new residents to move into the communities as others leave. 

UGWA and WTCMC members observe that these improvements are contributing to a greater sense of 

pride and hope in residents who, having experienced the seemingly relentless decline of their 

communities, now see the potential for stabilization and even revitalization. 

To be sure, the projects in themselves, even when completed, will only make a small dent in the area’s 

wastewater problems. Scores of small, unincorporated communities throughout the area continue to lack 

treatment systems. However, CVI and local citizens groups view the projects as models that can increase 

awareness of the problem and demonstrate to PSDs, local governments, regulators and area residents that 

solutions are technically and financially feasible through the application of innovative technologies. They 

believe that a small number of successful projects will increase the knowledge and capacity of PSDs, 

working with local citizens, to apply new treatment technologies and generate more widespread efforts 

to extend wastewater services throughout the area. 
Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

128 Final Report

In the long run, improving wastewater treatment on a large scale can, along with other economic 

development investments, better position the area for economic diversification by providing modern 

infrastructure and improving the quality of life, important factors in business investment decisions. 

6.12.5 Lessons Learned 

As a whole, the project met or exceeded its milestones and is on course to achieve its goals for new 

wastewater treatment infrastructure, albeit at a slower pace than originally anticipated. CVI’s partner 

organizations, UGWA and WTCMC are highly satisfied with the assistance it provided. The project 

yielded a number of important lessons: 

 Grassroots citizen activism can play a key role in infrastructure development. ARC typically 

works with public agencies to support local infrastructure development. In this case, however, 

the public agencies responsible for infrastructure development needed prodding from an active 

local citizenry to begin to address a critical infrastructure problem. Community residents 

organized to establish the UGWA and WTCMC, then engaged with their public service districts 

and county governments to promote solutions. While these groups were not ultimately 

responsible for building and managing wastewater treatment systems, they played key roles in 

educating PSDs about new technologies, securing their involvement, determining project 

priorities, gaining public support, guiding project design, and obtaining funding. 

 Outside organizational and technical support can play a catalytic role in resource-poor 

communities. UGWA and WTCMC would have been much less effective without CVI’s 

support. With its extensive specialized knowledge of small-scale wastewater treatment, CVI 

played a critical role in community education, planning, systems design, funding, and 

construction oversight. CVI’s role was particularly important in supplementing the limited 

capacity of PSDs to undertake new projects using unfamiliar technologies. As such, ARC’s small 

investment in technical assistance resulted in millions of dollars of new infrastructure 

investment.

 New approaches need time to take hold. Several years of ground-laying work occurred prior to 

ARC’s two-year grant, and more time will be needed for the efforts it supported to come to 

fruition. The introduction of innovative technology to address wastewater treatment needs 

encountered institutional, funding, regulatory, and technical challenges that take time to 

address. Projects of this nature must be viewed over a relatively long timeframe to measure their 

results. 

6.12.6 Conclusion 

With ARC’s support, CVI’s work has helped to move several projects forward that have or will result in 

the provision of new wastewater treatment services in a number of small communities. The project met 

or exceeded all of the milestones established at the outset of the ARC grant and is considered a great 

success by CVI’s community partners. The project demonstrates the role citizen activism can play in 

promoting needed infrastructure investments, the high “value-added” of expert technical assistance, 

particularly in resource poor communities, and the need to apply a long-term timeframe when assessing 

the impacts of projects involving new approaches and innovative technologies. 
Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

Final Report 129

6.13 Water Valley Sewer and Water Improvements – Water Valley, MS 

The City of Water Valley received ARC funding to repair deteriorated sewer lines in a residential area 

and a part of its downtown, and to extend water and sewer service to a small residential area just outside 

the city limits. The amount of the ARC grant was $152,547, accounting for 25 percent of the total project 

cost of $602,547. Work on the project began in March 2005 and was completed in January 2007. 

6.13.1 Community Profile 

The City of Water Valley, situated in north central Mississippi, is the largest municipality in Yalobusha 

County, with a population of 3,392 in 2010, about one-quarter of the county’s total population of 12,678. 

The city lies approximately 20 miles southwest of the City of Oxford, home of the University of 

Mississippi. 

The city’s population declined by 7.8 percent between 2000 and 2010 compared to increases of 7.3 

percent statewide and 9.7 percent nationally. Despite this decline, city officials indicate that the city’s 

population has remained roughly stable during the past half century. 

Yalobusha County’s employment trends roughly paralleled state and national averages during the past 

decade. Between 2001 and 2011, total wage and salary employment declined by 1.2 percent, less than the 

statewide decline of 3.1 percent but greater than the national decline of 0.2 percent. However, the county 

has continued to suffer from relatively high unemployment. During the past decade, its unemployment 

rate exceeded state and national levels by averages of 0.8 and 1.7 percentage points respectively. In 2011, 

unemployment stood at 11.9 percent compared to 10.7 percent statewide and 8.9 percent nationally. 

Moreover, the county’s labor force participation rate, the number of workers employed or actively 

searching for work, was low – only 52.1 percent compared to 59.8 percent statewide and 65 percent 

nationally in 2010. While the county has a slightly lower percentage of working-age population than 

state and national averages, this is not the major factor in this disparity. 

The county’s industry structure has been shifting from manufacturing to services in recent years. It 

remains dominated by the manufacturing and retail sectors, which together accounted for about 54 

percent of total employment in 2011. However, both of these industries declined significantly between 

2001 and 2011, with these declines being largely offset by growth in other industries, notably health and 

social assistance, administrative services (primarily building services), and transportation and 

warehousing. 

The economic well-being of city residents is far below state and national averages. Median household 

income averaged $22,816 during the 2006-10 period, only 58.9 percent of the state average and 43.9 

percent of the national average. The poverty rate of 34.7 percent was 15.8 percentage points above the 

state average and 23.6 percentage points above the national average. 

The county was categorized by ARC as distressed in 2004, when the ARC grant was awarded, and 

continued to be categorized as such through 2012. Distressed counties rank among the worst 10 percent 

of counties in the nation. 

There are a few bright spots in the city’s economic picture. Its historic housing stock, constructed during 

a period of prosperity and growth from the early 1900s to the 1930s, has attracted University of 

Mississippi faculty and staff from nearby Oxford, as well as professionals from Oxford and beyond. These 

homes can be purchased at a fraction of the price of similar homes in Oxford. New home purchases and 

renovation have sparked somewhat of a revival in some of the city’s older neighborhoods. With the 

leadership of an active Main Streets organization, the city’s previously moribund downtown has come to 

life with new business investment and public realm improvements. Additionally, the BorgWarner 
Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

130 Final Report

transmission plant, the city’s largest employer, once thought to be in danger of closing, employed over 

400 workers at the end of 2012. 

6.13.2 Project Description 

The project had two components. The first was to repair or replace about one half mile of damaged sewer 

lines in one of the city’s older residential areas and in part of its downtown business district. Existing 

sewer lines were more than 50 years old, and some had collapsed. This had the effect of slowing 

wastewater flows, resulting in periodic backups of sewage into homes and businesses, particularly 

during periods of heavy rain when rainwater leaked into sewer lines, overwhelming the system’s flow 

capacity. The second component involved connecting 20 households located just outside the city limits 

to public water and sewer service, requiring the extension of water and sewer lines about 2,000 feet. In 

addition to the ARC funding of $152,547, the project received a $450,000 Small Cities Community 

Development Block Grant (CDBG) grant from the Mississippi Development Authority. 

6.13.3 Project Planning and Implementation 

By the early 2000s, Water Valley city officials were becoming acutely aware that the city’s municipallyowned water and sewer system, suffering from years of deferred maintenance, was in dire need of repair. 

The need to repair or replace many of the city’s older sewer lines was particularly acute. Many lines were 

cracked and some were near collapse. The condition of the lines, which slowed wastewater flows, was 

causing sewage backups into residences and businesses with increasing frequency. The system was 

particularly prone to backups during heavy rains, when rainwater runoff leaked into sewer pipes, further 

reducing wastewater flows. These backups were, at the least, unpleasant, and sometimes resulted in 

property damage. The sight of a Water and Sewer Department truck pumping blocked sewage from 

sewage cleanouts (capped pipes that provide access to sewer lines) was becoming increasingly common. 

However, the small city, did not have the fiscal capacity to make needed repairs with its own limited 

revenues. 

By 2002, city officials concluded that they had to begin addressing the problem before it reached a crisis 

stage. The Water and Sewer Department staff identified the system’s most problematic areas, and the 

city’s consulting engineer prepared plans and cost estimates for improvements. One of the city’s older 

residential areas and part of its downtown business district were identified as in most immediate need of 

repair. 

As its next step, the city embarked on a search for funding. City officials approached the North Central 

Mississippi Planning and Development District, which helped local communities in the region obtain 

and administer state and federal grants, to identify and prepare grant applications to appropriate funding 

sources. The P&DD identified ARC and the Mississippi Development Agency’s State Small Cities CDBG 

program as the most likely funding sources, prepared grant applications, and received funding from both 

agencies by March 2005. The city was ready to move forward with the project. 

According to city officials and P&DD staff, the planning process proceeded smoothly. Final designs were 

drawn up by the city’s consulting engineer. A public hearing was held, and citizens expressed broad 

support for the project. The project was put out to bid, a contractor was selected, and construction 

began in October 2005 with oversight by the consulting engineering. 

The project proceeded at a somewhat slower pace than initially anticipated. While not unexpected, the 

repair and replacement of old sewer lines involved certain complications. Some lines had to be rerouted, 

in some cases requiring easements to be negotiated with property owners. Others were difficult to reach 

because they were under buildings, and had to be repaired using a relatively new process involving 
Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

Final Report 131

forcing a waterproof sealant through the pipes to seal cracks. Lines were sometimes in close proximity to 

other infrastructure, requiring extra caution. Rain and flooding during the construction period also 

caused delays. In the end, while it was initially expected to be completed in one year, the project was 

completed in late January 2007, 22 months after it began. It was, however, completed within budget. 

The project was followed by a second phase, also funded by ARC and State Small Cities CDBG funds, 

focusing on other older residential areas and the rest of the central business district. As of late 2012, 

engineering plans for this second phase were being finalized. 

6.13.4 Economic and Community Impacts 

The project achieved its projected outcomes. It served 354 households and 15 businesses, helping to 

retain 100 jobs, mostly in the retail sector. The sewer system is functioning well in the area where repairs 

were made, and back-ups have been eliminated. It also extended new water and sewer service to 20 

households. 

The project has had two important impacts. First, it has strengthened the city’s residential real estate 

market. With reliable sewer service, homes in the area where the improvements were made are now more 

marketable. Economic development professionals note an acceleration of the already growing trend of 

University of Mississippi faculty and staff, and professionals from Oxford and even outside the region, 

purchasing homes in the area. This has generated more demand for local business services, increased 

incomes, and more residential property tax revenues for the city. 

The second important impact has been on the city’s downtown business district. As with residential 

properties, more reliable sewer service has created a more secure environment for commercial property 

and business investment. The improvements complement efforts to revitalize the previously declining 

downtown led by the city’s Main Streets Program. Main Streets has undertaken an aggressive agenda to 

promote additional investment in commercial properties, attract a diverse and interesting mix of 

businesses, expand downtown programming, improve the public realm, and market the district. Some of 

these efforts have received ARC funding. The result has been a downtown renaissance. According to the 

Main Streets executive director, the period between 2007 and 2012 saw the opening of 20 new businesses 

with 52 employees, including an “anchor” grocery store, restaurants, and art galleries, occupancy of 

75,000 square feet of previously vacant space, $1.8 million in private property investment, and the 

reduction of vacant properties from 18 to four. While the sewer project was not primarily responsible for 

these developments, it did create conditions under which they were more likely to occur. Moreover, 

without the completed and planned sewer improvements, sewage services would inevitably worsen, 

potentially reversing the gains the downtown has recently achieved. 

6.13.5 Lessons Learned 

Overall, the project met the expectations of city officials and local and regional economic development 

professionals. It successfully restored a section of the sewer system to good working order and 

established a more favorable environment for residential and business investment. A number of lessons 

can be drawn from the project: 

 Even when undertaken to address a specific, immediate problem, infrastructure projects 

can yield broader, longer-term economic benefits. The purpose of the project was primarily to 

maintain well-functioning sewer service to a defined set of residences and businesses. It 

addressed a public health issue and, indirectly, a business retention issue. But by ensuring 

adequate sewer service, it also enhanced the appeal of residential and commercial properties, 

promoted additional property investment, and supported downtown revitalization. These 
Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

132 Final Report

developments, in turn, helped to improve the community’s self image and optimism, perhaps 

laying the groundwork for additional community improvements. This suggests that communities 

should think creatively about how they can leverage the benefits of infrastructure improvements. 

 Maintain close, ongoing communications with key stakeholders. Those involved in the 

project credit good communications, both internal and external, with making the project 

proceed smoothly. This included: communications among those involved in planning and 

implementing the project, including city staff, the consulting engineer, and the P&DD staff; 

communication with the public to inform it about the project and enlist support; and 

communication with funders to keep them up-to-date, resolve any issues expeditiously, and keep 

funding disbursements on schedule. 

 Have a solid plan developed before approaching funders. Water Valley’s mayor partly credits 

the planning process undertaken before approaching funders with the city’s success in obtaining 

grant funding. The city defined a clear need, targeted the most problematic parts of the sewer 

system for improvements, and conducted engineering studies to develop a detailed construction 

plan and cost estimate. 

 Plan for ongoing maintenance and repair to avoid pushing infrastructure systems to the 

breaking point. While the project achieved its objective, the city waited to address its sewer 

problem until the system had already seriously deteriorated. By anticipating inevitable 

maintenance and repair needs earlier, the city would have been better positioned to keep the 

system in good repair by applying for funding support and perhaps establishing its own 

maintenance reserve before the system literally began to collapse. This would likely have reduced 

the ultimate costs and minimized community disruptions. 

6.13.6 Conclusion 

The Water Valley Water and Sewer Improvements project met its objectives of making sewer system 

improvements that ensured reliable service to 354 residences and 15 businesses, and of extending new 

water and sewer service to 20 residences. More broadly, it helped to generate additional residential and 

business property investment and supported downtown revitalization. Key factors in the project’s 

successful completion were sound planning and effective communications. At the same time, the city 

may have been able to avert the severe deterioration of the system and its attendant consequences 

through timelier forward planning for system maintenance and repair. 

6.14 Case Study Summary Findings and Recommendations 

The case studies revealed key performance metrics and other impacts generated by the ARC Investments. 

These findings led to a series of key lessons learned that can be found in Section 6.14.3. 

6.14.1 Key Performance Measures 

Each of the case study projects achieved one or more of ARC’s key performance measures: leveraging of 

financial or technical support; jobs; employability; and increasing or improving infrastructure services to 

residents. 
Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

Final Report 133

Jobs and Leveraged Investment 

Almost every project leveraged public investment from grantees or other federal and state agencies. 

Notable examples of leveraged private investment and job creation include the following: 

 The Barton Business Park Sewage Treatment Plant Project attracted a $12 million investment 

in a new plant by a wooden cabinet manufacturer, creating 364 jobs. 

 The Muscle Shoals Industrial Infrastructure project attracted a $60 million private investment 

in a new plant by an automotive supplier, creating 800 jobs. 

 The Dobson I-77 Infrastructure project attracted an $18.25 million private investment in a 

hotel/restaurant/retail complex, creating 66 jobs. 

 The Clinton I-75 Industrial Park Sewer Line project attracted a $63 million investment by an 

automotive supplier as well as additional investment by several other new industrial park 

occupants. A total of approximately 1,500 jobs were created. 

Employability 

The automotive supplier attracted by the Muscle Shoals Industrial Infrastructure project made a 

significant investment in a job training program at the local community college. In addition, at least three 

projects, the Big Caney Water project, the Owingsville/Bath County Industrial Park Wastewater 

Treatment Plant project, and the Racine Water Treatment Plant and Storage Facility project, 

supported the development of education and training facilities through the provision of water or 

wastewater treatment services. 

Infrastructure Services 

Several projects provided new or improved water or wastewater treatment services to local residents 

including the following: 

 The Big Caney Water project provided improved water service to 453 households, and other 

phases of the project served hundreds more households. Additional system capacity will also 

accommodate significant new residential development. 

 The Canaseraga Wastewater Collection and Treatment System project provided new service 

to 215 households, many of which had been faced with the cost of replacing failing septic 

systems. 

 The Water Valley Sewer and Water Improvements project repaired failing sewer lines serving 

354 households and extended new water and sewer service to an additional 20 households. 

6.14.2 Other Project Impacts 

Projects had a number of other impacts that have contributed or promise to contribute to strengthening 

the local economic environment. 

Increased Residential Property Values and Investment 

Communities with inadequate or compromised water or sewer service or lacking such service altogether 

are at a distinct economic disadvantage. Homes are more difficult or impossible to sell, lessening 
Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

134 Final Report

incentives for property investment and contributing to declines in the quality of the housing stock. 

Investment in new housing development is also deterred. When considering locations for their 

businesses, business owners are unlikely to select locations that do not provide suitable housing for their 

employees. While residential property investment and improvements in the residential real estate market 

are long-term processes, several communities where residential water and sewer projects have been 

completed have benefited or see the opportunity to benefit from improvements in the housing 

environment. 

After completion of the Water Valley Sewer and Water Improvements project, homes in the area 

where the improvements were made are now more marketable. Economic development professionals 

note an acceleration of the already growing trend of University of Mississippi faculty and staff, and 

professionals from Oxford and even outside the region, purchasing homes in the area. 

Completion of the Big Caney Water project has resulted in increased investment in existing housing as 

well as increased construction of new housing. County officials and economic development professionals 

believe that improved housing quality and availability is helping to stem the population outflow, 

convincing more young adults to remain in the area and attracting new residents such as retirees who 

earlier left the area to find work. 

After completion of the Racine Water Treatment Plant and Storage Facility project, a housing 

developer announced plans to build 15 new homes on vacant land, a substantial expansion of the 

residential stock for a community of 315 households. 

Enhanced Environmental Quality 

Two projects, the Wastewater Treatment for Small Communities project, the Canaseraga 

Wastewater Collection and Treatment project, stopped flows of raw sewage into waterways. Another, 

the Water Valley Sewer and Water Improvements project, stopped periodic sewage backups into 

residences and businesses. These projects not only improved environmental and health conditions, but 

generated additional economic benefits by making homes more marketable and supporting recreational 

tourism. 

Fiscal Benefits 

A number of projects had positive fiscal implications for their communities. These included: reducing 

costly water losses from leaky water systems, putting water system operators on a sounder financial 

footing, and reducing or eliminating the need for tax subsidies to keep the systems solvent; increasing 

property tax revenues from new and expanding businesses and rising residential real estate values; and 

the opportunity to sell increased water supplies to additional users. 

After completion of the Owingsville/Bath County Industrial Park Wastewater Treatment Plant, 

additional tax revenue generated by expansion of a food processing plant in the industrial park enabled 

the City of Owingsville to improve its downtown with a new streetscape program, which has 

contributed to the startup of several new downtown businesses. 

After completion of the Dobson I-77 Infrastructure project, occupancy tax revenue from the new hotel 

enabled the community to establish a Tourism Development Authority. Each year, the tax brings about 

$100,000 to the TDA, which has been used to beautify the downtown, build a new town website, 

advertise the town’s attractions, and promote local events.

Completion of the Big Caney Water improvements has drastically reduced water loss, resulting in 

significant cost savings for the Public Service Authority and reducing the possibility that the PSA will 
Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

Final Report 135

ever require a county subsidy. In addition, new home construction and increases in property values have 

generated new tax revenues for the county. 

The Canton Water System Improvements reduced water loss from nearly 65 percent to less than 20 

percent, enabling the Canton Borough Authority to sell water that had been previously lost through 

leakage to shale gas producers. The sale of the water has generated an additional $1 million in annual 

revenue over two years. 

Corporate Citizenship 

The Muscle Shoals Industrial Infrastructure and Clinton I-75 Industrial Park Sewer Line projects, by 

bringing major employers into their communities, have gained corporate “citizens” that are all too often 

missing from smaller communities. These firms have shown a commitment to their communities through 

their civic engagement and charitable activities. 

Positioning for Longer-term Economic Development 

Whether or not they resulted in significant immediate private investment and job creation, many of these 

projects have positioned their communities for longer-term economic development. While not 

measurable in the short-term, these impacts can yield significant economic benefits in the years to come. 

The Wastewater Treatment Solutions for Small Communities project has begun to have an impact on 

tourism development in southern West Virginia’s highly distressed coal counties. As streams that used to 

receive untreated sewage experience reductions in coliform bacteria levels, they are again becoming 

suitable for fishing. A few homes have been remodeled into tourist cabins for ATV visitors. The reduction 

of odor from raw sewage has made communities with Coal Heritage Trail sites more appealing to visitors, 

potentially promoting the development of tourism-related businesses such as restaurants and artisan 

shops. 

The Burson Center, in addition to its direct services to its tenants and other local entrepreneurs, has had 

a significant impact on the perception and reality of the community as a supportive entrepreneurial 

environment. Business and community leaders note that the center, with its wide range of business 

resources, has become the “go-to” place for budding entrepreneurs and has played a significant role in 

encouraging the growth of entrepreneurship within the region. This, in turn, has increased confidence 

within the community in its ability to shape its economic future. 

The completion of the Dobson I-77 Infrastructure project has improved the town’s self-image and 

demonstrated the potential of economic diversification. The town is now thought of as a tourism 

destination, and some have even termed it the “unofficial capital of the Yadkin Valley wine region.” This 

project has shown local residents how change can be beneficial and has helped to create a vision for the 

area’s future that residents can support. 

6.14.3 Key Lessons from the Case Studies 

The case studies yield important lessons that can be useful for other communities undertaking similar 

projects. 

Consider the Project in the Context of a Broader Strategic Vision 

While an infrastructure project is typically undertaken to meet an immediate community need, its 

impact can be magnified if it is tied to a broader community economic development strategy. 
Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

136 Final Report

The development of the Burson Center was not pursued in isolation, but rather as part of a broader 

community strategy that placed a strong emphasis on entrepreneurship development. The foundation of 

a comprehensive strategic plan with strong community support increased the likelihood that the project 

would gain support of key stakeholders and complement other efforts to promote local entrepreneurship.

Surry County leaders have begun to recognize that the county, the location of the Dobson I-77 

Infrastructure Project, has a competitive advantage in tourism and wine production. By focusing on 

winemaking, specialty agriculture, outdoor recreation, and heritage tourism, Surry County has chosen a 

path that is outward looking, but that would, by necessity, preserve or even enhance its rural, small town 

character. Because the Village at Shelton Vineyards, developed as a result of the project, was part of this 

broader strategy, the infrastructure extension is having an outsized impact, one that goes far beyond the 

businesses it directly serves.

Think and Plan Ahead 

The case studies demonstrate a number of ways that thinking and planning ahead can improve the 

chances for a successful project or avoid unanticipated challenges. 

First, communities can benefit by making foundational investments that enable them to move quickly 

when economic development opportunities arise. 

Muscle Shoals and the rest of the Shoals region responded to their economic development challenges 

proactively. They prepared a CEDS focusing on economic diversification, purchased industrial land, and 

prepared preliminary engineering plans for its development. As such, they were well positioned when a 

major industrial prospect, automotive supplier North American Lighting, imposed a short timeframe for 

the preparation of an industrial site for its new plant. Had the region not been well prepared, the NAL 

plant may well have located in another community.

Having already drawn up plans for a water and sewer extension, Surry County and Dobson town leaders 

waited for a good opportunity to come along before they went forward with the Dobson I-77 

Infrastructure project. In addition to the advantage of having a planned job-creating project to show 

when competing for grants, having some guaranteed new utility customers and an increased tax base 

ensured that at least some costs could be recovered. Also, having private sector partners as funders and 

allies was crucial to getting the project completed.

Second, project planning can benefit from research about similar projects and what approaches have 

been most successful. 

The Burson Center project participants engaged in a thorough planning process before proceeding with 

the project. This included engaging professional assistance to conduct a feasibility study, exploring 

models and best practices, and thoroughly considering such issues as size, design, financial structure, 

tenant services and resources, and operations. While observing other successful incubators, they took 

pains to customize the project to meet local needs and conditions. 

As part of the planning process for the Canton Water System Improvements project, different types of 

water main materials were evaluated to compare advantages and disadvantages. Polyethylene pipes were 

selected due to unique physical characteristics: low weight, corrosion resistance, high-impact resistance, 

and flexibility. Leaks were substantially reduced, increasing the efficiency of the system. 

Third, some of the case study communities delayed addressing their infrastructure problems until these 

problems had reached a crisis stage. These communities would have been better served by seeking 

assistance earlier, limiting environmental damage and possibly reducing the cost and complexity of their 

projects. 
Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

Final Report 137

In the Village of Canaseraga, residential septic systems were already failing and, with homeowners 

unable to afford the cost of repairs, untreated sewage was increasingly diverted into storm drains or the 

local river. This created significant environmental and health problems and also affected the village 

economy as property values dropped and recreational hunting and fishing declined. It was only when the 

local USDA Rural Development office director approached village officials after becoming aware of the 

problem that they began to explore a solution. 

While the Water Valley Water and Sewer Improvements project achieved its objective, the city 

waited to address its sewer problem until the system had already seriously deteriorated. By anticipating 

inevitable maintenance and repair needs earlier, the city would have been better positioned to keep the 

system in good repair by applying for funding support and perhaps establishing its own maintenance 

reserve before the system literally began to collapse. This would likely have reduced the ultimate costs 

and minimized community disruptions. 

Finally, infrastructure projects, like all construction projects, can encounter unexpected problems that 

can slow progress or increase costs. Thinking at the outset about what problems might occur and how 

they might be addressed can mitigate their impacts should they arise. 

Planning for the Barton Business Park did not anticipate that the proposed sewer outfall would lie on 

archaeological sites. The fact that the park was located near the Potomac River, with its many historical 

sites, made this a distinct possibility. Careful investigation of potential obstacles and risks may have 

avoided project delays and accounted for the additional cost of archeological data recovery. 

The cost of the Canton Water System Improvements project increased by $500,000 as a result of 

unexpected events and an estimating error by the engineering consultant. Although the engineering firm 

provided $100,000, the Canton Borough Authority had to make an additional loan to cover the overrun. It 

recommends that communities undertaking infrastructure projects establish a contingency fund or 

secure access to credit markets for additional loans to cover unexpected expenses. 

Right-size Your Project 

When designing projects, communities should think about how they relate to current and potential 

market demand. This could lead to downsizing a project or completing it in phases. 

The projected outcomes for job creation and private investment established for the Dobson I-77 

Infrastructure project were based solely on Phase I of the planned development. This ended up being a 

wise decision, because when the recession hit, development stalled, and subsequent phases have not been 

completed. In cases like this where there is a particular development associated with a project, 

construction timelines should be looked at carefully and developers should be clear about how 

circumstances might affect their plans. The Dobson project was successful, but if conservative 

projections had not been used, it may not have been viewed that way. 

The Barton Business Park project anticipated the creation of 800 to 1,000 jobs. Six years after the 

project’s completion, only one company had located at the business park and only 364 jobs had been 

created. Project planners must differentiate between the potential for site development and real interest 

in site development. A sound marketing strategy at the time of project completion may have helped drive 

industrial interest. 

On the other hand, a larger initial investment may save money in long run. 

The Powdersville Water District Water Storage Tank was designed with more capacity than was 

needed to meet immediate needs. By taking advantage of low construction costs during the recession, the 

Water District was able to build in the additional capacity that will very likely be needed in the not-too-
Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

138 Final Report

distant future at a reasonable cost. Waiting to expand the system until economic growth accelerated 

would likely have cost PWD customers more in the long run. 

The Owingsville/Bath County Industrial Park Wastewater Treatment Plant project was undertaken 

primarily for one large industrial user, but building in additional capacity enabled the city to serve 105 

new homes, a new agricultural education and marketing center, and the expansion of a nursing and 

rehabilitation facility, with additional capacity left for new businesses locating in the industrial park. 

Consider How to Fully Leverage Your Investment 

An infrastructure project can be a catalyst for development beyond its immediate objectives. 

Communities should think about how to maximize these ancillary benefits. 

The purpose of the Water Valley Sewer and Water Improvements project was primarily to maintain 

well-functioning sewer service to a defined set of residences and businesses. It addressed a public health 

issue and, indirectly, a business retention issue. But by ensuring adequate sewer service, it also enhanced 

the appeal of residential and commercial properties, promoted additional property investment, and 

supported downtown revitalization. These developments, in turn, helped to improve the community’s 

self image and optimism, perhaps laying the groundwork for additional community improvements. 

While the Big Caney Water project was primarily intended to improve the quality and reliability of 

water service to existing residential customers, it has also provided a foundation for economic 

diversification. It has already supported tourism development through improved water service to Breaks 

Interstate Park and will enable the development of more lodging facilities and second homes. The county 

is also better positioned for business recruitment since employers give careful consideration to quality of 

life for their employees, and the availability of good-quality housing is an important factor in that 

calculation. 

In addition to improving the quality of life for area residents, the Wastewater Treatment Solutions for 

Small Communities project in southern West Virginia’s coal country is helping to strengthen the area’s 

housing market. Without sewage service, homes cannot legally be sold, contributing to housing 

disinvestment and abandonment. As homes are connected to new sewage systems, homeowners are now 

able to sell them legally, increasing their value, encouraging better upkeep, and enabling new residents to 

move into the communities as others leave. Local residents observe that these improvements are 

contributing to a greater sense of pride and hope in residents who, having experienced the seemingly 

relentless decline of their communities, now see the potential for stabilization and even revitalization. 

The Canaseraga Wastewater Collection and Treatment System addressed environmental and health 

issues created by failing septic systems by connecting existing households and businesses to a new 

municipal sewer system. It also positioned the community for additional commercial, industrial and 

residential development, including capturing spin-offs from the development of the nearby Swain ski 

area through lodging and second home development. 

Develop Strong Working Relationships with Funders 

A key element of success is the ability to develop relationships with funders and manage those 

relationships over the long term. This has a number of dimensions. 

First, be able to demonstrate a compelling need and a credible plan. 

The mayor of Water Valley partly credits the planning process undertaken for the Water Valley Sewer 

and Water Improvements project before approaching funders with the city’s success in obtaining grant 
Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

Final Report 139

funding. The city defined a clear need, targeted the most problematic parts of the sewer system for 

improvements, and conducted engineering studies to develop a detailed construction plan and cost 

estimate. 

The plan to rebuild the Big Caney Water System and extend water service throughout the county was 

ambitious and costly, particularly for economically depressed Dickenson County. The Public Service 

Agency and its partners developed a long-term vision for the project, and were able to convey to funders 

its importance to the county’s future. At the same time, they recognized that the project could not be 

taken on all at once and would have to be broken into phases. This enabled funders to understand the 

project’s overall funding needs and at the same time see that it could be sequenced to meet their funding 

restrictions. 

Second, take care to effectively coordinate grants among multiple funders and to manage grants in ways that fully meet 

funder requirements. Having a clear, detailed understanding of the restrictions and parameters of the 

funding from all partners from the start of the project can allow for a smoother transaction process. 

Finding a first funder willing to commit money to a project is a chronic challenge. The Mayor of Racine 

invited all potential funders of the Racine Water Treatment Plant & Storage Facility to the village 

simultaneously to see the need and to discuss the collective resources they had available to invest. The 

community was then also able to demonstrate its own buy-in to the project. Racine raised nearly all of 

the funds it required to fund the water treatment plant and storage facility, including supplemental 

funds needed after a flood. 

The grant manager for the Owingsville/Bath County Industrial Park Wastewater Treatment Plant

project noted that one of the most valuable steps she took through the process was a small, simple one 

done from the start: tracking funding sources and uses. “Before we submitted the first application, we 

invited all funders to the table. We sat down around the table and brainstormed and talked about how 

we could best approach this,” she recalled. “When you’re trying to get something done quickly, you 

cannot afford to make mistakes. For example, if you get half-a-million in CDBG funding...it needs to be 

used in a particular way. You can’t use that for engineers, for instance. So we tracked the money. I 

designed a grid and tracked funding across components. It also saved time, and allowed people coming in 

who weren’t already familiar with the project to take over, if needed.” Funding requirements are often 

complicated, and become only more confusing when multiple funds are combined in one pool. Having 

simple processes from the start of a project to clearly depict which funds can be used for which parts of 

construction can save time and money. 

Funding for the City of Clinton’s I-77 Industrial Park Sewer Line project came from five different 

funding sources. Each funder had its own standard set of use restrictions and payment methods. Some 

funders only paid for certain expenses, while others required the grantee to absorb costs upfront and 

then get reimbursed. It ended up costing the project money to deal with the complexity and detail 

required. 

Develop Strong Collaborative Relationships and Effective Communication among All Key Local 

Actors 

Successful initiation and completion of projects requires the participation of many key actors. The case 

studies yield numerous examples of how collaborative relationships were developed and maintained 

among key actors to ensure alignment, needed resource commitments from various parties, and effective 

coordination. 

The leaders of the Burson Center project recognized early on that a wide variety of stakeholders would 

need to be engaged and relied upon for material support if the project were to succeed. Great pains were 
Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

140 Final Report

taken to reach out to city and county government, business and civic leadership, and higher education 

institutions to involve them in the process and gain their support. 

Those involved in the Water Valley Sewer and Water Improvements project credit good 

communications, both internal and external, with making the project proceed smoothly. This included: 

communications among those involved in planning and implementing the project, including city staff, the 

consulting engineer, and the Planning and Development District staff. 

In Dickenson County, the Public Service Authority’s collaborative approach extended back to its merger 

with the Big Caney Water Corporation, which was carefully negotiated to achieve a positive outcome for 

both parties. During the Big Caney Water project, the PSA worked closely with the county, 

municipalities, local development district, and state health officials to bring all ideas and perspectives to 

the table and distill them into a plan on which all stakeholders could agree. This collaborative approach 

continued through all phases of the project. 

Having competed for industrial prospects in the past, county and municipal officials involved in the 

Muscle Shoals Industrial Infrastructure project came to realize that in an era of global competition and 

international investment, demonstrating unity and collaboration is critical to attracting firms like North 

American Lighting. Extending the collaborative culture that had begun to gel in the early part of the 

decade, the communities shared financial burdens and together offered a welcoming and supportive 

environment for NAL that helped to persuade the firm that Muscle Shoals was the best location for its 

new plant. 

The Dobson I-77 Infrastructure project was designed through a process that ensured that the town and 

county governments, the developers, and area residents would all support it. The developers, the city, and 

the county all took part in the project’s planning and contributed significant funding to the project. This 

helped to ensure that everything went smoothly and that the final outcome would be to each partner’s 

liking. When a grant that the town had expected to receive did not come through, all the parties 

contributed money to fill the gap because they considered themselves equal partners. What could have 

been a major roadblock was just a speed bump because each stakeholder had engaged in the planning 

process and had something to gain from the final project. 

Tap into Outside Expertise and Resources 

Small communities often lack the expertise needed to plan and implement projects. Tapping into free or 

low-cost sources of technical assistance can help to ensure that a project is well designed and effectively 

managed. 

Citizens groups working on the Wastewater Treatment Solutions for Small Communities project 

would have been much less effective without the support of the Canaan Valley Institute. With its 

extensive specialized knowledge of small-scale wastewater treatment, CVI played a critical role in 

community education, planning, systems design, funding, and construction oversight. CVI’s role was 

particularly important in supplementing the limited capacity of Public Service Districts to undertake 

new projects using unfamiliar technologies. As such, ARC’s small investment in technical assistance 

resulted in millions of dollars of new infrastructure investment. 

As a small community with significant financial limitations, the Village of Racine did not have the 

resources to hire a consultant or engineer to develop a comprehensive plan to present to potential 

funders of Racine’s Water Treatment Plant & Storage Facility project. Instead, the village turned to 

the Rural Community Assistance Partnership (RCAP), a national nonprofit that provides technical 

assistance to rural communities. It provided Racine with the technical assistance and support its leaders 
Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

Final Report 141

needed to put together a viable and credible proposal. With RCAP’s assistance, Racine was able to 

obtain the funding it needed for a system that suited its needs. 

Be Prepared to Meet Business Timelines 

In the case of the Owingsville/Bath County Industrial Park Wastewater Treatment Plant project, the 

quick reactions of city and county officials, working together to draw up appropriate plans and find the 

appropriate financing to pay for the project, was critical to retaining one of the area’s largest employers. 

Without this timely action, the plant was in danger of relocating. Instead it has expanded, retaining 122 

jobs and adding over 100 new ones.

Take a Hands-on Approach during the Construction Phase 

The case study projects demonstrate the importance of paying close attention to the construction 

process and taking timely actions to minimize delays and disruptions. This has a number of dimensions. 

First, businesses constructing a new plant or expanding an existing plant can have inflexible timelines in order to meet 

production goals. Communities developing projects to accommodate new plants or expansions must 

organize themselves to complete projects on the same timeline. 

Local officials in Muscle Shoals organized to exercise close oversight of site preparation to ensure that it 

was completed according to North American Lighting’s tight schedule and to work closely with funding 

sources to ensure that funding delays did not slow the project.

Second, diligent oversight of the construction contractor is critical. 

The construction phase of the Canton Water System Improvements project proceeded smoothly due in 

large part to the diligent coordination between Canton Borough Authority and the project engineer. 

Daily inspections conducted by the project engineer ensured that the construction was carried out 

according to the plans and specifications. Canton Borough Authority also assisted the construction team 

in locating existing mains and service lines to reduce errors. 

For the Clinton I-75 Industrial Park Sewer Line project, the Utilities Board had a local wastewater 

specialist on staff who proved invaluable throughout the construction process. He consistently 

conducted quality checks on the materials and construction, and was able to make field adjustments as 

needed. 

Finally, it is important to keep residents and businesses informed about construction activities and help homeowners and 

businesses handle any service disruptions. 

The Big Caney Water project involved water line construction along roadways, and potential service 

disruptions. The Public Service Authority made extensive efforts to keep local leaders and the public 

informed about construction impacts and, as a result, encountered few complaints. 

The Canton Borough Authority recognized, in retrospect, the need to make a greater effort to keep 

downtown businesses affected by the construction of the Canton Water System Improvements project 

informed about the impacts on their operations and to help them mitigate these impacts. Businesses 

received several water advisories from CBA, but they were still not prepared for the project. 

Take a Long-term View 

Many grantees noted that, for small communities, undertaking even relatively straightforward 

infrastructure projects can pose difficult institutional, financial, and logistical challenges. Maintaining a 
Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

142 Final Report

strong commitment to the project throughout the process and responding to challenges with creativity 

and determination is often necessary to achieve a successful outcome. 

Several years of ground-laying work occurred prior to ARC’s grant for the Wastewater Treatment 

Solutions for Small Communities project, and more time will be needed for the efforts it supported to 

come to fruition. The introduction of innovative technology to address wastewater treatment needs 

encountered institutional, funding, regulatory, and technical challenges that take time to address. 

Projects of this nature must be viewed over a relatively long timeframe to measure their results. 

The Canaseraga Wastewater Collection & Treatment System project took over six years from initial 

conception to completion. It took two years for the village to obtain its first financial commitment from 

USDA Rural Development. Once the first funder committed, successive financing became more easily 

available. By the time the project came to fruition, leaders had achieved a high level of local buy-in, 

developed a sound, well thought out plan, and honed an effective case for why the project was necessary. 

It was all accomplished by having a process through which stakeholders learned from failure and did not 

get discouraged in the pursuit of their goals. 

Those involved in the Burson Center project recognized that developing and growing a local 

entrepreneurial base is a long-term process. They had the patience to take the time needed to ensure that 

the incubator was well-designed and professionally managed and, while they are rigorous in measuring 

the progress of incubator tenants, recognize that the larger economic changes they are seeking to 

promote will take more than a few years to materialize.

The director of the Public Service Authority managing the Big Caney Water project acknowledged that 

the technical and financial challenges presented by the project sometimes appeared insurmountable, but 

the strong commitment of the PSA and its partners would eventually yield solutions. He advised, “Never 

go into a project and throw up your arms and say it can’t be done. There are always bumps, but stay 

focused, develop good relationships, and keep strong people around you.” At the same time, he noted the 

importance of adjusting plans to align with funding limitations, stating, “You look at your funding 

sources, you look at your priority list, and you have to cater the project to the funding.”
Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

Final Report 143

7 FINDINGS AND RECOMMENDATIONS 

The Appalachian Regional Commission has four main goals, as laid out in the 2011-2016 Strategic Plan: 

1. Increase job opportunities and per capita income in Appalachia to reach parity with the nation. 

2. Strengthen the capacity of the people of Appalachia to compete in the global economy. 

3. Develop and improve Appalachia’s infrastructure to make the Region economically competitive.

4. Build the Appalachian Development Highway System to reduce Appalachia’s isolation.

The performance of the ARC’s infrastructure investment program is closely related to these goals, and the 

intent of this program evaluation was to assess the performance and economic impacts of ARC 

infrastructure investments between 2004 and 2010. In addition, the evaluation was to shed light on ways 

that the ARC can enhance its ability to document and report program impacts. 

7.1 Key Findings 

The team’s infrastructure program evaluation work examined the impact of a particular investment on 

income, employment opportunities, job creation and retention, as well as the number of households and 

businesses served by the project. The evaluation considered both quantitative and qualitative outcomes 

of the investments in order to gauge the overall performance of the projects. 

Water and sewer projects were the most frequent uses of grants, as well as the most heavily funded 

project types, based on survey results and the ARCnet database. These projects also generated the most 

benefit in terms of jobs created or retained, businesses served, and households served, according to the 

survey. Statistically, every million dollars spent on water or sewer projects generated approximately 30

new jobs between 2004 and 2010. 

Based on the survey responses and case study findings, grant recipients seemed to underestimate the job 

and business impacts that the investment would have on their area. In many cases, the anticipated 

outcome per dollar spent was actually less than the actual outcome per dollar spent – a benefit to the 

ARC. Qualitatively, survey and case study respondents thought that the funding they received 

contributed to attracting new businesses and jobs to the community, as well as improving the efficiency 

of business operations. According to respondents, approximately 35 percent of the projects would not 

have occurred without the contributions from the ARC, indicating the importance of the program 

toward achieving the Commission’s goals.

Overall, the evaluation found mixed results in terms of outcomes and outputs, though there are many 

factors contributing to these results. The varied feedback and success of the projects, as identified 

through the online survey results and case study evaluations, has led the team to create a series of 

recommendations for future program funding prioritization. This list of recommendations is not meant 

to be a total assessment of the program or its priorities, but rather is intended to support future success 

while achieving the goals of the ARC. 

The team observed several important “lessons learned” through the questionnaire and case study process 

that may help the Commission and future grant recipients best leverage ARC funding to achieve 

maximum economic development outcomes: 

 Consider the project in the context of a broader strategic vision. 

 Think and plan ahead. 

 Right-size your project, but know that sometimes a larger initial investment may save money in 

the long run. 
Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

144 Final Report

 Consider how to fully leverage your investment and develop strong working relationships with 

funders. 

 Develop strong collaborative relationships and effective communications among all key local 

actors. 

 Tap into outside expertise and resources. 

 Be prepared to meet business timelines and take a hands-on approach during the construction 

phase. 

 Take a long-term view. 

7.2 Recommendations 

The recommendations provided below are intended to both improve the quality of the data utilized in 

ARC program evaluation and also the data collection process itself. 

Require tracking of outputs and outcomes as a stipulation of receiving the grant. Several 

respondents noted that they were only required to keep records for five years and thus did not have any 

information on the project in question. Others simply did not have the information because they did not 

track the outputs or outcomes. This led to a lower than desirable response rate to the questionnaire and 

the possibility of skewed results. Given the high priority of tangible improvements to job opportunities 

and other competitiveness measures, it would be in the best interest of the ARC to require tracking 

simply for reporting and analysis purposes. This does not mean that a lack of the anticipated success in a 

particular case would be penalized, but would simply allow for better future prioritization of funding 

investments. One suggestion to achieve this is to develop a letter directed to the Local Development 

Districts indicating the importance of collecting this information.

Consider new types of infrastructure investment to increase competitiveness. Water and sewer 

infrastructure projects accounted for the largest share of ARC spending during the evaluation period, and 

these projects are essential in providing basic services to underserved areas. As the region’s most basic 

needs are met, it is important to invest in additional infrastructure. For example, a strong fiber optic and 

broadband infrastructure could help increase Appalachia’s competiveness with other regions of the 

country.

Clarification of Project Categories. There may be some disconnect in the classification of projects for 

funding purposes and the project type as viewed by the grant recipients. When comparing the project 

type from the ARCnet database to the project type as identified by the survey respondent, there were 

multiple cases of discrepancies between the two. For example, in one case a project was classified as an 

access road in the ARCnet database, though it is actually an airport runway. Clarity on project type will 

aid in tracking performance metrics in the future.

Collection of Additional Information Related to Funding Sources. It is not always the case that ARC 

funds are the only sources of capital for specific infrastructure projects. Better understanding of all the 

funding required to bring a project to fruition, as well as the sources of these funds, would provide useful 

insight to ARC. It would be interesting to know, for example, whether greater success in terms of job 

retention and creation occurs when there are many parties involved in funding the project. A better 

understanding of the leveraging power of ARC investments, as well as the non-ARC incentives that are 

being provided to support specific projects, may be informative to ARC as they consider future 

investment. 
Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

Final Report 145

APPENDICES 

Appendix A: Appalachian Regional Commission Program Evaluation of Infrastructure 

and Public Works Questionnaire 

Appendix B: Sample Letter to Grant Recipients 

Appendix C: Full Database Distribution by Project Type and State 

Appendix D: Survey Response Project Types and Responses 

Appendix E: Outlier Detection and Removal 

Appendix F: Regression Analysis Predictive Ability 
Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

146 Final Report

APPENDIX A: APPALACHIAN REGIONAL COMMISSION 

PROGRAM EVALUATION OF INFRASTRUCTURE AND PUBLIC 

WORKS QUESTIONNAIRE 

Appalachian Regional Commission 

Program Evaluation of Infrastructure and Public Works Projects Questionnaire 

April 2012 

This short questionnaire should not take more than 20 minutes to complete. All your responses will be 

kept confidential. 

The results of this questionnaire will help the Appalachian Regional Commission fulfill its mission to 

promote economic development in the region, and assist current and future grantees with improving 

project performance measurement. Thank you very much for participating. 

SECTION A: Background Information 

1. The answers you give in this survey will be held confidential. In order to track responses and 

enable us to follow-up with you, we ask that you provide your contact information.

 Name of person completing this questionnaire 

 Organization 

 Address 

 City 

 State 

 Zip 

 Phone 

 Email 

 Website 

2. What was the ARC project title? 

3. What was the ARC project ID (for example, AL-12345)? 

4. In what month and year was the ARC-funded portion of this project closed out? (Indicate N/A if 

the project is still open)

5. What type of infrastructure project was it? (Choose one of the following) 

Water system, Sewer system, Access road, Industrial site development, Telecommunications, Gas line, Intermodal 

facility, Rail, Airport, Highway 

6. This project primarily benefited:

 Households 

 Businesses 

 Downtown District 

 Public Services 

 Other (please specify) 
Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

Final Report 147

7. Where was the project primarily implemented? 

City: 

State: 

County: 

LDD (Local Development District): 

Congressional District(s): 

If the project covered multiple jurisdictions, list them here: 

Section B: Project Outputs 

What specific results (outputs) were actually achieved by this project? These questions attempt to 

capture the characteristics, capacity and direct outputs of the infrastructure project. Depending 

on your project timeframe, include cumulative outputs up to 3 years after project closeout. 

In the Output column, please provide a numerical estimate. If you did not track information for a 

specific output, please enter “DNT” in the corresponding box in the Output column (e.g., you do 

not have information about the number of households served). If an output is not applicable to your 

project, please enter “NA” in the corresponding box in the Output column (e.g., your project did 

not provide services to households). 

Use the definitions provided below to determine how a given output should be expressed. 

8. Group Outputs: Output 

Households served

Businesses served

Organizations served

Communities served

9a. Infrastructure Outputs: Water system

Linear feet of pipe 

System capacity (millions of gallons or millions of gallons per day – indicate which) 

Previous capacity in MG or MGD (if applicable) 

Percent of water supply losses (difference between “production” from plant and “consumption” 

by users 

Number of water main failures per year 

9b. Infrastructure Outputs: Sewer system

 Linear feet of sewer pipe 

 System capacity (millions of gallons or millions of gallons per day) 

 Previous capacity (MG or MGD if applicable) 

 Percent of sewer system overflows (volumes of untreated water released into streams) 

 Number of sewer main failures per year 

9c. Infrastructure Outputs: Access Road

 Miles of access road 

 Number of business sites served 

 Annual average daily automobile traffic (AADT) 
Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

148 Final Report

 Annual average daily truck traffic (AADTT) 

9d. Infrastructure Outputs: Industrial Site Development

 Number of business sites served 

 Square feet of developed commercial or industrial space 

 Acreage improved 

9e. Infrastructure Outputs: Telecommunications

 Number of telecommunications sites developed 

 Number of businesses sites developed 

 Linear feet of cable/fiber 

 Speed (mega-bytes per second) 

 Giga-bytes 

9f. Infrastructure Outputs: Gas Line 

 Linear feet of gas line 

 Volume of gas (cubic feet or therms – specify) 

9g. Infrastructure Outputs: Intermodal Facility

 Acreage improved 

 Number of trucks loaded/unloaded (annual) 

 Facility capacity for throughput – annual carloads 

 Facility capacity for throughput – annual containers 

 Facility capacity for throughput – annual tons 

9h. Infrastructure Outputs: Rail

 Mileage of track 

 Carloads served (annual) 

 Containers served (annual) 

 Tonnage (annual) 

9i. Infrastructure Outputs: Airport

 Number of planes annually 

 Number of passengers annually 

 Number of cargo shipments annually 

9j. Infrastructure Outputs: Highway

 Mileage of highway constructed 

 Number of communities served 

 Annual average daily automobile traffic (AADT) 

 Annual average daily truck traffic (AADTT) 

10. List any other outputs that were measured or evaluated: ___________________________ 

11. Please use this box to provide any additional information, including: data sources; estimation 

methods; whether or not outputs vary over time; or anything else that may be relevant. 
Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

Final Report 149

DEFINITIONS: 

Households Served: The cumulative total number of households that are served by your ARC project, 

from project inception until now. 

Businesses Served: The cumulative total number of businesses that are served by your ARC project, 

from project inception until now. 

Organizations Served: The cumulative total number of organizations that are served by your ARC 

project, from project inception until now. 

Communities Served: The cumulative total number of communities that are served by your ARC 

project, from inception until now. 

Section C: Project Outcomes

Please provide the following information about final project achievements and results. 

What specific results (outcomes) were actually achieved by this project? These questions attempt 

to capture the direct outcomes of the infrastructure project. Depending on your project 

timeframe, include cumulative outcomes up to 3 years after project closeout. 

In the Outcomes column, please provide a numerical estimate. If you did not track information 

for a specific outcome, please enter “DNT” in the corresponding box in the Outcomes Column 

(e.g., you do not have information about the number of jobs created). If an outcome measure is not 

applicable to your project, please enter “NA” in the corresponding box in the Outcomes column 

(e.g., your project did not create any jobs). 

In the Year(s) column, please indicate the number of years for which the outcome measure was 

tracked. 

Use the definitions provided below to determine how a given outcome should be expressed. 

12. Group Outcomes: Outcomes Year(s) 

Households improved

Businesses improved

Organizations improved

Communities improved

13. Economic Outcomes: 

Jobs created

Jobs retained

Businesses created

Businesses retained

Leveraged private investment

Revenues increased (non-export, $)

Revenues increased (export, $)

Costs reduced ($)
Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

150 Final Report

14. In what industries were the jobs created or retained? Please quantify number for each industry. 

 Agriculture 

 Mining 

 Construction 

 Manufacturing 

 Transportation Utilities 

 Wholesale 

 Retail 

 Finance, Insurance or Real Estate 

 Services 

 Other 

14. List any other outcomes that were measured or evaluated: ___________________________ 

16. Please provide relevant information about how the outcome measures were calculated such as data 

sources used or key assumptions. 

DEFINITIONS:

Households improved: the cumulative total number of households that directly benefited from the ARC 

project. 

Businesses improved: the cumulative total number of businesses that directly benefited from the ARC 

project 

Organizations improved: the cumulative total number of organizations that directly benefited from the 

ARC project. 

Communities improved: the cumulative total number of communities that directly benefited from the 

ARC project. 

Jobs created: the total number of (1) direct hires made as a result of the project’s operation (e.g. teachers, 

public safety, information service, etc.); and (2) direct private sector jobs created following the delivery of 

ARC-funded services or project completion. This does NOT include construction jobs to build the ARC 

funded project. In the case of part-time jobs, please convert these to full-time equivalent and round up to 

report whole numbers. 

Jobs retained: the total number of jobs that were retained because of an ARC investment that was 

needed to keep the business and jobs in continued operation in the area. 

Businesses created: the total number of businesses that located in the region as a direct result of the 

ARC-supported project. 

Businesses retained: the total number of existing businesses that remained in the region due to the ARCsupported infrastructure project. 

Leveraged private investment: the total dollar amount of private sector financial commitments and 

investment that were not part of the project funding, but followed as a result of the completion of your 

ARC project. 

Revenues increased (non-export): the total dollar amount of any increase in non-export (domestic) sales 

that occurred among businesses improved as a result of your ARC project. 

Revenues increased (export): the total dollar amount of any increase in export (international) sales that 

occurred among businesses improved as a result of your ARC project. 

Costs reduced: the total dollar amount of any cost reductions (savings) that occurred among 

organizations and businesses improved as a result of your ARC project. 
Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

Final Report 151

Section D: Long-Range Impacts

Using the textual scale defined below, how would you rate the long-range impacts of your project on the 

economy, the environment, and the quality of life in your community? 

None: Project had little to no impact on trend

Slight: Project impact was not large enough to reverse or stabilize trend 

Moderate: Project impact contributed to the stabilization or reversal of trend 

High: Project impact was responsible for significant improvement in trend 

17. Economic Measures None Slight Moderate High 

Attracting new residents or stabilizing the area’s population 

Attracting new jobs or increasing employment at existing businesses 

Creating new sources of income or increasing income for local residents 

Increasing local business sales or the value of business assets (such as equipment, real estate) 

Increasing the value of household assets (such as homes, land, farms) for local residents 

Increasing the value of community assets (such as community buildings, schools, infrastructure, parks) 

Other economic measures (please specify) 

18. Competitiveness Measures None Slight Moderate High

Improving the stability and sustainability of the local economy 

Improving the efficiency of business operations or public services 

Improving the productivity of students, employees, businesses, land, or other assets 

Improving the skill level of the workforce 

Increasing the viability of local businesses 

Improving access to markets for local products, businesses, artisans, and entrepreneurs 

Reducing the cost of doing business 

Other competitiveness measures (please specify) 

Section E: Other Questions 

19. Aside from the direct jobs and investment associated with the project, were there any other 

positive or negative economic impacts on the community? If so, please explain and quantify if 

possible. 

20. Did the project help attract any additional government or philanthropic funding? Please 

describe. 

21. To what extent are the estimated outcomes attributable to the ARC intervention and funding? 

Entirely, mostly, somewhat, slightly, or too difficult to determine. 

22. What do you think would have happened to the project if ARC funding had not been available? 

 Would have been completed with other funds in approximately the same time period 

Would have been delayed for up to a year 

Would have been delayed for a year or more 

Would have been completed on a smaller scale 

Would not have been undertaken 

23. Please describe any private investment not directly related to the project (i.e., not included in 

your response to Question 12) that occurred as a result of the project. 

24. Please provide any additional comments on the project in terms of accomplishments, challenges, 

or other relevant information to help with the program evaluation.
Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

152 Final Report

APPENDIX B: SAMPLE LETTER TO GRANT RECIPIENTS 

[This Page Intentionally Blank] 
695 Atlantic Avenue, 2FL, Boston, MA 02111 

Telephone: (617) 357-7763  Fax: (617) 357-7759  http://www.hdrinc.com 

Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

April 25, 2012 

<<Name>> 

<<Organization>> 

<<Street Address>> 

<<City>>, <<State>> <<Zip>> 

Dear <<Name>>, 

I am contacting you on behalf of the Appalachian Regional Commission (ARC). The ARC has contracted with HDR 

Decision Economics, an independent research firm, to conduct an external evaluation of its Infrastructure and Public 

Works projects. The purpose of the evaluation is to collect information on the outcomes and long-term impact of 

infrastructure projects so that ARC may improve future project performance. 

HDR is conducting an online questionnaire of all ARC Infrastructure and Public Works projects funded between 

2004 and 2010. Participation in the survey is encouraged but voluntary. Your responses will help provide valuable 

information that will assist ARC in making decisions about future program efforts to improve infrastructure and 

public works initiatives. We anticipate that the questionnaire will take approximately 20 minutes to complete for 

each project for which you received ARC funding. 

<<Project Name>> 

<<Project ID>> 

The questionnaire can be completed at: https://www.surveymonkey.com/s/ARCInfrastructureEvaluation

The questionnaire for a give project must be completed on one computer from one location. It is best to complete the 

questionnaire in one sitting. Please also note the following: 

 The due date for the questionnaire is May 18, 2012

 Please complete a questionnaire for each of the projects listed above. 

All components of this program evaluation have been reviewed and approved by Kostas Skordas of the 

Appalachian Regional Commission. If you have any questions, please feel free to contact him or Marissa 

Witkowski (marissa.witkowski@hdrinc.com or 1-617-357-7763). 

On behalf of the ARC, we thank you in advance for your cooperation. 

Best regards, 

Marissa Witkowski 

HDR Survey Manager for ARC
695 Atlantic Avenue, 2FL, Boston, MA 02111 

Telephone: (617) 357-7763  Fax: (617) 357-7759  http://www.hdrinc.com 

154 

APPENDIX C: FULL DATABASE DISTRIBUTION BY PROJECT 

TYPE AND STATE 

AL GA KY MD MS NC NY OH PA SC TN VA WV Total

Access Road 1 12 2 19 23 2 2 20 30 1 0 4 0 116

Airport 0 0 0 0 0 0 0 1 0 0 0 0 0 1

Gas Line 1 0 1 0 1 0 0 0 0 0 1 0 0 4

Highway 0 0 0 0 0 0 1 0 0 0 0 0 0 1

Industrial Site Development 12 9 4 4 25 2 1 2 16 2 6 1 2 86

Intermodal 0 0 1 0 0 0 0 0 2 0 0 0 0 3

Rail 0 0 0 0 1 0 1 0 0 0 1 0 0 3

Sewer System 19 17 24 8 27 14 12 37 11 11 15 6 16 217

Telecommunications 15 3 2 6 4 2 10 6 5 1 2 5 4 65

Water System 31 26 48 7 39 31 8 26 10 17 40 10 22 315

Total 79 67 82 44 120 51 35 92 74 32 65 26 44 811
Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

Final Report 155

APPENDIX D: SURVEY RESPONSE PROJECT TYPES AND 

RESPONSES 

Access Road Count

# with Actual # Value Miles of Road 20

# with DNT/NA 6

# Blank 4

# with Actual # Value Businesses Served 20

# with DNT/NA 5

# Blank 5

# with Actual # Value AADT 10

# with DNT/NA 13

# Blank 7

# with Actual # Value AADTT 11

# with DNT/NA 13

# Blank 6

# Responses (Total) 30

Airport Count

# with Actual # Value Planes 1

# with DNT/NA 0

# Blank 0

# with Actual # Value Passengers 1

# with DNT/NA 0

# Blank 0

# with Actual # Value Cargo Shipments 0

# with DNT/NA 0

# Blank 1

# Responses (Total) 1
156 Final Report

Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

Gas Line Count

# with Actual # Value Linear Feet 0

# with DNT/NA 0

# Blank 1

# with Actual # Value Volume 0

# with DNT/NA 0

# Blank 1

# Responses (Total) 1

Highway Count

# with Actual # Value Miles of Road 3

# with DNT/NA 0

# Blank 0

# with Actual # Value AADT 2

# with DNT/NA 1

# Blank 0

# with Actual # Value AADTT 2

# with DNT/NA 1

# Blank 0

# with Actual # Value Communities Served 3

# with DNT/NA 0

# Blank 0

# Responses (Total) 3

Industrial Site Development Count

# with Actual # Value Businesses Served 11

# with DNT/NA 3

# Blank 3

# with Actual # Value Sq. Ft. Development 7

# with DNT/NA 8

# Blank 2

# with Actual # Value Acreage 6

# with DNT/NA 10

# Blank 1

# Responses (Total) 17
Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

Final Report 157

Intermodal Facility Count

# with Actual # Value Acreage 1

# with DNT/NA 0

# Blank 1

# with Actual # Value Trucks Loaded 0

# with DNT/NA 1

# Blank 1

# with Actual # Value Capacity Carloads 1

# with DNT/NA 0

# Blank 1

# with Actual # Value Capacity Containers 0

# with DNT/NA 1

# Blank 1

# with Actual # Value Capacity Tons 0

# with DNT/NA 1

# Blank 1

# Responses (Total) 2

Rail Count

# with Actual # Value Capacity Carloads 1

# with DNT/NA 1

# Blank 0

# with Actual # Value Capacity Containers 1

# with DNT/NA 1

# Blank 0

# with Actual # Value Capacity Tons 1

# with DNT/NA 1

# Blank 0

# with Actual # Value Miles of Track 2

# with DNT/NA 0

# Blank 0

# Responses (Total) 2
158 Final Report

Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

Sewer System Count

# with Actual # Value Linear Feet 28

# with DNT/NA 17

# Blank 7

# with Actual # Value Capacity 17

# with DNT/NA 22

# Blank 13

# with Actual # Value Previous Capacity 9

# with DNT/NA 29

# Blank 14

# with Actual # Value Water Supply Loss 0

# with DNT/NA 0

# Blank 52

# with Actual # Value Failures per Year 8

# with DNT/NA 30

# Blank 14

# Responses (Total) 52

Telecommunications Count

# with Actual # Value Sites Developed 13

# with DNT/NA 5

# Blank 1

# with Actual # Value Businesses Served 9

# with DNT/NA 7

# Blank 3

# with Actual # Value Linear Feet 8

# with DNT/NA 8

# Blank 3

# with Actual # Value Speed 6

# with DNT/NA 8

# Blank 5

# with Actual # Value Gigabytes 5

# with DNT/NA 10

# Blank 4

# Responses (Total) 19
Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

Final Report 159

Water System Count

# with Actual # Value Linear Feet 47

# with DNT/NA 17

# Blank 6

# with Actual # Value Capacity 33

# with DNT/NA 23

# Blank 14

# with Actual # Value Previous Capacity 21

# with DNT/NA 30

# Blank 19

# with Actual # Value Water Supply Loss 21

# with DNT/NA 32

# Blank 17

# with Actual # Value Failures per Year 18

# with DNT/NA 35

# Blank 17

# Responses (Total) 70
160 Final Report

Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

APPENDIX E: OUTLIER DETECTION AND REMOVAL 

Outliers removed from the group of access road and industrial site development projects16 were identified 

through the following multi-step process, which was primarily, though not always, followed in this 

order: 

1. A scatterplot was generated for the dependent variable compared to each of the explanatory 

variables in the regression equations to identify, at a cursory glance, any projects that should be 

analyzed with greater suspicion in subsequent steps. 

2. Various types of residuals were generated and plotted using a histogram in order to assess both 

their unstandardized and standardized deviations from the mean. Residuals generated and 

analyzed included standardized residuals, deleted residuals, studentized residuals, and 

studentized deleted residuals. The reason for comparing the distribution of various types of 

residuals is because deleted residuals, in particular, can vary greatly from other residuals if 

certain projects are exhibiting unusually high leverage, or pull, in the regression. 

3. In order to look more closely at a project’s influence on the regression,

a. differences between the various residuals generated in the previous step were calculated 

for each residual; 

b. leverage values were generated; and 

c. Cook’s D values were generated.

Standard thresholds17 were then used to identify projects with excessive influence that should be 

flagged as possible outliers. 

4. Residual values were plotted against predicted values in order to confirm that there was no 

relationship (correlation). The mean of residuals was also calculated in order to confirm that it 

equaled zero. Projects exhibiting a non-random distribution or skewing the average of residuals 

were flagged as potential outliers. 

5. Finally, Q-Q and P-P plots were generated in order to inspect for deviations from normality 

among the residuals. The reason for generating both plots is because P-P plots are generally 

better for assessing deviations from the middle of the normal distribution and Q-Q plots are 

generally better for assessing deviations from the tails of the normal distribution. 

The decision to label a project as an outlier did not rest on the determination of one step alone, but rather 

a combination of steps that collectively heightened the level of suspicion for the influence of a particular 

project on the final regression model and its ability to predict variance in job creation or intermediate 

outcomes. Care was taken to preserve as many projects as possible as to both preserve the degrees of 

freedom in the equation and to create a model that best represents the full set of projects in the ARCnet 

database. A replication of Table 12, the set of regression equations presented in the text, is provided 

below, this time including outliers except for water/sewer projects where no outliers were identified. 

 

16 Note that no outlying water/sewer projects were identified. 

17 Leverage values exceeding 

 

, where k is the number of explanatory variables and n is the number of projects, 

were flagged as potential outliers. Cook’s D values exceeding 

 

 where n is the number of projects were flagged as 

potential outliers. 
Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

Final Report 161

Explanatory Variable

Coefficient by Project Type

Access Road Industrial Site Water/Sewer

Total Spending ($1 million) 272.4 83.1 51.9**

Density (10 persons/square mile) 11.03*** -.81 .51

Density*Total Spending ($1 million) -.167 .271* .049**

Distress (10 units) 2.95 6.79*** 5.50***

Distress*Total Spending ($1 million) -1.69 -.636 -.310**

Percent Explained R

2 = .410 R2 = .405 R2 = .486

Model Significance (F-score) *** *** ***

N 86 72 161

Note: Linear regression through the origin: Dependent (predicted) variable is “jobs created.” The constant term was suppressed 

so that there would be no jobs predicted if no spending occurred. 

*Significant at 90% confidence level; **Significant at 95% confidence level; ***Significant at 99% confidence level 
162 Final Report

Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

APPENDIX F: REGRESSION ANALYSIS PREDICTIVE ABILITY 

To better illustrate the predictive ability of the regression equations, an analysis of residuals was 

performed using results from both the full set of ARCnet projects and the subset of those projects that 

had survey responses. Residuals represent the difference between job creation values observed in both 

datasets and the values estimated by the regression equations. The results, shown in Figure 15

through 17, reflect the level of difference (horizontal axis, labeled at the bottom) and frequency of 

occurrence (vertical axis, labeled at the left). Largely consistent among all project types, these results 

indicate that estimated values are very close to observed values for the great majority of cases, though 

there was a small group for which the regressions consistently under-estimated job creation. This is 

represented by a dotted circle in each graph. There was no corresponding group for which the 

regressions over-estimated job creation. And while apparent in the case of both predicted and actual job 

creation values, the under-estimation was more common among the full set of ARCnet projects than 

among the survey responses. 

The presence of this phenomenon, occurring for every project type, indicates that some of these projects 

must have had non-observed factors causing them to generate significantly larger-than-expected job 

impacts. The most logical explanation, which is supported by some case studies, is that some of the 

ARC-funded infrastructure projects were bundled with other (non-infrastructure) economic 

development actions (e.g., grants, loans, tax abatements, job training, etc.) to create a comprehensive 

package. Any such non-infrastructure actions would not be captured in either the ARCnet dataset or the 

survey dataset, yet they would tend to increase job impacts beyond what would otherwise be expected 

from the single project alone. This bundling can involve multiple grantors at multiple levels of 

government, each collecting grantee data separately. In the absence of coordination, impacts reported by 

projects receiving significant amounts of support from multiple agencies could thus appear larger than 

expected. 

Figure 15: Access Road Projects – Difference between Estimated and Observed Job Creation 

 Predicted Jobs Actual Jobs 

N=16

Expected 

distribution of error 

Cases for which job 

creation was substantially 

underestimated 

N=76

difference
Program Evaluation of the Appalachian Regional Commission’s Infrastructure & Public Works Projects

Final Report 163

Figure 16: Water/Sewer Projects – Difference between Estimated and Observed Job Creation 

 Predicted Jobs Actual Jobs 

Figure 17: Industrial Site Projects – Difference between Estimated and Observed Job Creation 

 Predicted Jobs Actual Jobs 

If these special cases are removed from the analysis of the ARCnet dataset, the explanatory precision of 

the regression estimates improves drastically. With the secondary cluster removed, regressions on the 

ARCnet dataset – which provides more observations and much higher significance – estimate job 

impacts within ten percent of the actual impact for over three-quarters of the projects. This finding 

indicates the potential value of predictive tools, but also the need for further research to improve the 

process for evaluating economic development programs. 

Expected distribution 

of job creation

Cases for which job 

creation was substantially 

underestimated 

N=161

Expected 

distribution of error

Cases for which job 

creation was substantially 

underestimated 

difference

difference

N=84

N=61 N=34