Evaluation of the ARC Entrepreneurship Initiative

1

Evaluation of the Early Stages of the Appalachian Regional Commission’s Entrepreneurship Initiative

A Report to the Appalachian Regional Commission

March 2001

Revised December 2001

Regional Technology Strategies, Inc. 

205 Lloyd Street, Suite 210

Carrboro, North Carolina 27510

919 933-6699 Fax 919 933-6688

Info@rtsinc.org

www.rtsinc.org
Evaluation of the ARC Entrepreneurship Initiative

2
Evaluation of the ARC Entrepreneurship Initiative

3

Table of Contents

 Section Pa

ge

Acknowledgments iv

Executive Summary vii

I. Introduction 1

II. Methodology 3

A. Review of Literature 4

B. Information from Project Files 8

C. Sample Selection 8

D. Survey Design 9

E. Client Survey 10

F. Site Visits 11

III. Findings and Analysis 12

A. Caveats 13

B Characteristics of Projects 13

C. Degrees of Satisfaction 19

D. Economic Outcomes and Impacts 19

E. Program Outcomes: What Really Matters? 23

F. Obstacles and Problems 25

IV. Recommendations 27

V. Appendices 33

A. Entrepreneurship Project Profiles (Evaluation Sample) 33

B. Project Survey Responses 57

C. Survey Instruments 68

D. Detailed Literature Review and Bibliography 84
Evaluation of the ARC Entrepreneurship Initiative

4

Acknowledgments

This report represents the diligent work of RTS staff and consultants on the research 

team. Pat Dusenbury, Creative Strategies, Inc., conducted the two site visits, 

interviewed clients, and helped draft conclusions and recommendations. Phil Shapira, 

Georgia Tech, helped design the methodology, conducted statistical analysis of the 

survey data, and provided valuable insights throughout the study. Sarah Butzen and 

Sarah Heinemeier also assisted with the literature review. Ann Guissinger and Sarah 

Heinemeier conducted the bulk of the client interviews. Jonathan Morgan and Eric 

Forman were instrumental in getting survey responses from the projects, with 

assistance from Sarah Butzen and Heather McMahon who helped compile the data. 

Jonathan also co-authored and helped edit the final report. Special thanks also to Sue 

Soltis for her keen proofreading of the final document and managing the contractual 

process. 

We are especially grateful to ARC staff directors Greg Bischak and Ray Daffner, who 

provided critical information, advice, and guidance throughout the process, and the 

members of the Evaluation Advisory Committee for critiquing the report and for devoting 

their time and energy to ensuring that the project was a success. 

Finally, we wish to thank all of the many project staff directors and staff and the clients 

who so carefully completed the survey forms and answered our questions. 

Stuart Rosenfeld, Project Director

March 2001
Evaluation of the ARC Entrepreneurship Initiative

5

Entrepreneurship Initiative

Evaluation Advisory Committee

Todd Christensen – ARC State Program Manager

Virginia Department of Housing and Community Development

Richmond, VA

Brian Dabson, President

Corporation for Enterprise Development

Washington, DC

Jerry Davis, Executive Director

Shoals Entrepreneurship Center

Florence, AL 

Jeff Dukes, Special Projects Officer

Mississippi Dept. of Community and Economic Development

Jackson, MS 

Rick Larson, National Director

REAL Enterprises

Durham, NC

Karen Mocker, Sr. Advisor Community Affairs

Federal Reserve Bank of Cleveland

Cincinnati, OH 

Joy Padgett, Director

Governor's Office of Appalachia

Columbus, Ohio 

Don Rychnowski

Southern Tier West Regional Planning and Development Board

Center for Regional Excellence 

Salamanca, NY 
Evaluation of the ARC Entrepreneurship Initiative

6
Evaluation of the ARC Entrepreneurship Initiative

7

Executive Summary

I. Introduction

In 1997 the Appalachian Regional Commission (ARC) launched an $17.6 million effort 

to bolster the entrepreneurial infrastructure of Appalachia. The primary goal of this 

Entrepreneurship Initiative is to promote the creation and development of locally owned, 

high value-added firms that will increase local wealth and provide employment 

opportunities for local residents. It focuses on five areas:

ð Entrepreneurial education and training

ð Entrepreneurial networks and clusters

ð Technology transfer

ð Access to capital and financial assistance

ð Technical and managerial assistance

The importance of an entrepreneurial approach to economic development in the 

Appalachian region cannot be overstated. Such an approach helps strengthen and 

diversify the region’s economic base through a strategy of “building from within” and 

stimulating growth of indigenous industries. It builds upon Appalachia’s unique 

strengths by nurturing homegrown firms, encouraging innovation and risk-taking, and 

fostering an environment conducive to new business formation. While there is evidence 

of notable entrepreneurial activity in the region, the phenomenon is neither widespread 

enough nor at a sufficient scale to have substantial and sustainable regional impacts. 

II. Research Approach

The Appalachian Regional Commission commissioned an external evaluation of the 

Entrepreneurship Initiative grants to (a) examine its funded projects and their respective 

impacts in sufficient depth to verify results, (b) identify outcomes and impacts not 

captured by standardized data, (c) develop other category-specific performance criteria, 

and (d) recommend actions to improve the effectiveness of continuing and future 

initiatives. The ARC awarded Regional Technology Strategies, Inc. (RTS) the contract 

to undertake the assessment and analysis based on a sample of 23 - 25 projects.

To collect the necessary data, the research team (1) conducted a careful review of 

information and documentation supplied to the ARC by grant recipients, (2) conducted a 

survey of the sample of projects supplemented by selected telephone interviews with 

project staff, (3) conducted telephone interviews with randomly selected clients and 

partner organizations from among names submitted by projects, and (4) visited two 

project sites for a more in-depth perspective.

III. Key Early Stage Findings

It is very important to note that this was an initial evaluation effort based on an early set 

of projects. Most of the projects were short term and, when the evaluation was 
Evaluation of the ARC Entrepreneurship Initiative

8

conducted, had been in operation for an average of just 18 to 21 months, and 

entrepreneurial outcomes generally require time for gestation and development. 

Further, the sample of projects evaluated was biased toward an early program stage 

mix of activities that favored education and training and networks. Later stage projects 

included a higher proportion of projects such as capital access and technical

assistance, which are more likely to produce quicker results. Thus, one might expect 

increasing evidence of economic outcomes for the Initiative over time. The challenge 

will be to shift projects towards activities such as capital access and technology transfer 

that are more often associated with growth plans while continuing to develop the selfemployment potential of the underemployed and unemployed. 

These caveats should be kept in mind when interpreting results, given the long termnature of entrepreneurial outcomes and impacts. However, the findings in this report 

provide an indication of preliminary program performance and outcomes that generally 

validate the ARC target outcomes and it provides meaningful insights that might aid in 

improving future implementation and outcomes of the ARC Entrepreneurship Initiative. 

A. Characteristics of Projects:

ð The awards were relatively small. The median ARC grant in the sample was under 

$60,000. Half of the project teams had staffs of fewer than three people, even 

during their peak period of effort, and only two reported five or more at peak.

• The grant recipient organizations also tended to be small. Thirteen of the 

organizations reported annual organizational budgets of less than $1 million. About 

half of the grants were made to non-profit economic development agencies, and the 

other half were divided among government agencies, regional planning agencies, 

community organizations, universities, and community colleges. 

ð Forty percent of the evaluated programs specifically targeted disadvantaged 

populations, while 35 percent targeted youth. The majority of sampled projects 

(57 percent) were located in counties designated as distressed by ARC.

• The internal evaluation and monitoring systems used by the projects tend to lack 

specific outcome measures. Although many grantees claimed to have conducted 

internal assessments, few could document tangible and measurable economic 

results. While only seven of 18 business assistance projects and three of 20 

education and training projects lacked any formal evaluation, 43 percent of all 

projects cited “monitoring outcomes” as a problem.

• All grantee organizations used ARC funding to leverage or match other funds. ARC 

funds were half the total budgets of the projects in aggregate. Of the remaining 

funds, nearly a quarter were “other,” a fifth were state or county, and only one tenth 

of one percent were from fees for services. 

• Most projects were multi-purpose. Only two had a single purpose as defined by 

ARC. This reflects project developers’ sensitivity to clients’ preference for “one stop 

shops,” and intermediaries able to provide a range of real and brokered services. 
Evaluation of the ARC Entrepreneurship Initiative

9

B. Project Activities

• The most common major activities of the projects are: (1) one-on-one assistance to 

new firms (74 percent); (2) education and training of adults (65 percent); (3) one-onone assistance for existing firms (48 percent); (4) business networks among new 

firms (48 percent); (5) custom software development (48 percent); and (6) business 

seminars (44 percent).

• Half of client firms interviewed received assistance with gaining access to capital 

and 47 percent received managerial and technical assistance. Forty-two percent 

reported having received entrepreneurial training.

• Among programs that made loans, reported amounts were quite small. More than 

half were less than $25,000. Although many of the loans went to micro-enterprises 

in which small amounts can have a large impact, these amounts are quite small for 

the technology-based businesses likely to generate growth and impact an economy.

C. Project Outcomes and Impacts

The evaluation uses a variety of measures to assess outcomes and impacts on 

individuals and the local economy. The first set of measures assesses relative degrees 

of satisfaction of clients with the intervention. The second important set evaluates the 

economic results of the intervention on the client such as new enterprises started, new 

markets, or new products lines. The third captures the more general impacts on the 

economy in terms of jobs created by the new enterprises, growth potential of the 

enterprises, and potential sustainability of the program. 

Many of these outcomes were reported to the ARC in the grantees final report. Results 

reported in this evaluation may differ—in some instances, substantially—because a 

different time frame has been used for evaluation (i.e., not limited by the grant closing 

date), the measure is defined slightly differently, or programs that have been ended are 

integrated with other related programs that makes it impossible to attribute outcomes a 

specific grant. Taking these caveats into account, some of the following outcomes were 

reported. 

Clients of the programs generally expressed high levels of satisfaction with the support 

they received. Of those interviewed, 70 percent expressed high satisfaction with 

services they received, 17 percent said they were satisfied, and only two were 

dissatisfied. Forty- three percent said the assistance or advice they received exceeded 

their expectations, 47 percent said it met them, while 10 percent still had unmet needs.

Three-quarters of projects reported businesses developed new products, 55 percent 

indicated that firms upgraded technologies or management methods, and half reported 

starting new businesses.

About 52 percent of the sample projects reported creating jobs in existing firms and 39 

percent reported saving jobs that would have otherwise been lost. Adults created 214 
Evaluation of the ARC Entrepreneurship Initiative

10

new firms—33 new firms with1 and 181 without employees. Based on those projects 

that were able to report hard numbers in the survey, 356 new jobs were created—54 in 

new firms, 121 in existing firms, and 181 jobs through self-employment. Another 85 

jobs were saved from extinction in existing firms. In addition, surveyed projects 

reported 46 new businesses created by youth or students as class projects, some of 

which could become self-sustaining businesses after graduation. 

It is too early to determine impacts on the economy, and thus they can only be deduced 

from the aspirations of the clients. The objective of the majority of the entrepreneurs 

was to generate a livable income for an individual or family but the objective of a large 

minority was growth. It is clear that a significant number of the assisted businesses in 

fact have the potential and desire to grow and have an impact on the economy. Of the 

clients interviewed with businesses, 36 percent expected to “grow,” 55 percent wanted 

stable “lifestyle” businesses, and 4 percent were attempting to reverse a decline. 

D. Obstacles and Problems

• Difficulty in attracting business customers was noted by many of the grantees. 

Getting existing businesses to use the available services was most often mentioned 

as a problem, with ten projects citing it—nine as minor and one as major problem. 

Students were more readily attracted to programs.

• Monitoring and evaluation was also an issue, with five projects designating it a major 

problem and five a minor problem. The difficulty that the evaluation project staff 

experienced in obtaining data about project outcomes supports the perception that 

this is a major problem.

IV. Summary Observations

The ARC Entrepreneurship Initiative supported programs that produced numerous 

individual successes (people who have improved income and opportunity) but fewer 

that could demonstrate significant impacts on their local economies in the short time 

frame. Most funded programs supported very small businesses and start-ups—

predominantly in low-growth sectors and serving local markets.

The ARC grant itself clearly matters to the grantees’ communities. Responses to the 

project survey indicate that ARC funding was essential. Half of the projects reported 

that they would not have gone forward without it and it was important to the timing of 

most others. Only two would have secured other funding and moved ahead without 

delay. 

Sustainability is crucial to the success of the initiatives. The majority (14) of the grants 

analyzed funded new programs, and all but one of the existing programs had previously 

received some form of government or foundation support. In this context, 

sustainability—or the lack of it—becomes an important outcome. The entrepreneurial 

 

1

 Not all projects reporting the creation of firms with employees specified the actual number of employees. 
Evaluation of the ARC Entrepreneurship Initiative

11

initiatives are more likely to have lasting impacts if they continue beyond the ARC grant 

period. While some projects are short-term and designed to address an immediate 

issue, sustainability builds local capacity for entrepreneurial support. 

Diversification of the funding base supporting these projects is the key to sustainability. 

The 18 programs that continued beyond the end of their ARC support did so by drawing 

upon multiple funding sources. The largest number—almost three-fourths—reported 

receiving state funds. The large state role in follow-up funding suggests that states may 

be using ARC grants as start-up funds or to give promising ideas a test run. Only seven 

of the continuing programs included fee income from clients among sources of 

continuation funding. Client fees provided less than one percent of program budgets 

during the ARC grant period. The small role played by client funding is reinforced by 

the results of the client survey. Those who reported paying for services were 

predominantly students paying tuition. Only a few paid nominal fees for services. Thus, 

it appears that very few entrepreneurial support programs are self-sustaining based on 

fees for services, and thus require continuing support after the initial grant.

The size of the organizations awarded grants is important. It affects staff support 

capacity and long-term sustainability and has implications for the ability to manage not 

only project activities but also federal grant funds and reporting requirements. The 

capacity of small organizations to manage federal grants is critical to success, yet often 

limited.

The size of awards may have some influence on outcomes. Larger grants are 

associated with slightly higher outcomes, as measured by new businesses started, 

although the differences are too small for statistical significance.

The inability to attract business customers is a common problem. It may reflect lack of 

sufficient attention to, or analysis of, market demand in the design and proposal stage. 

Proposals for projects that deliver services ought to be able to demonstrate core 

demand or the projects may be solutions in search of problems. 

Use of specific, outcome-based indicators is lacking in internal monitoring and 

assessment systems. Overall, the projects do not appear to be effectively tracking 

performance as a management tool to provide feedback to local staff so they can make 

adjustments, solve problems, and work toward continuous improvement. 

Projects have useful community impacts. About one-half of projects reported improved 

entrepreneurship education and training opportunities and business assistance services 

in the community after the ARC project ended (Table 8). Seventeen percent of the 

projects report an increase in the availability of funding for new business start-ups even 

though only a small number of projects give capital access a priority among their 

objectives.

V. Recommendations

Based on the above findings and observations the research team recommends the 

following.
Evaluation of the ARC Entrepreneurship Initiative

12

1. Move the Entrepreneurship Initiative to Scale

The current ARC Entrepreneurship Initiative has established useful project models and 

assisted individual entrepreneurs and businesses. It would be worthwhile to continue 

the effort at the current scale. Reducing or eliminating a program that has begun to 

achieve some degree of success and demonstrate potential, and which depends on 

patience and long-term support to realize its value, would send the wrong message to 

communities that may have few other options. However, to have a substantial impact 

on the region’s economy, the program needs to be expanded to a more significant 

scale. Any expanded investment needs to be made with a multi-year time horizon 

(since these efforts require sustained commitment) and with consideration to the 

changes and improvements suggested in our subsequent recommendations.

2. Provide more technical assistance

Funded programs must build local capacity if they are to be sustainable. ARC is a 

wholesaler, but for the system to work, ARC must build the capacity of the retailers that 

deliver the services. Many grantees would benefit greatly from additional external 

support. Small organizations in rural areas—especially those trying new and innovative 

activities—have few places to turn to for advice and counsel. Three possible ways to 

introduce support and supplement local resources are: 

ð to build a technical assistance budget into all grants and designate either a cohort of 

consultants or lead (existing) agency/institutions for all new projects. 

ð to form learning networks comprised of grantees with similar or complementary 

goals to help share experiences and learn from each other. 

ð conduct more staff training for project directors, requiring all its principal 

investigators to assemble at least annually for budget and management workshops, 

a project fair, and presentations.

3. Take a market-oriented approach

The limited resources available and the nature of the ARC Entrepreneurship Initiative—

an intervention to address market failures—suggest that its operation should be as 

market-oriented as possible. One of the problems reported by projects was difficulty in 

attracting clients to their services. While grant applicants are required to demonstrate a 

general need for proposed services a more thorough analysis of the target population 

and the willingness of potential clients to use services will ensure better utilization. ARC 

could provide a small grant to finance the market research and commit to a larger 

project grant only when and if the market research reveals sufficient demand. 

4. Structure and support evaluation systems

The performance measurement process for the Entrepreneurship Initiative should offer 

ARC an opportunity not only to respond to its GRPA responsibilities but also help its 

grantees improve project management. For ARC to guide grantees in this process, it 
Evaluation of the ARC Entrepreneurship Initiative

13

must have its own activity and outcome measures aligned into a meaningful system. 

Thus, the first step is for ARC to assess and revise its performance measures for the 

Entrepreneurship Initiative so that they can effectively serve as the organizing context 

for funded projects’ performance measurement. The research undertaken in this project 

provides only a starting point for this endeavor. Grantees need standard guidelines, 

technical assistance and additional support to conduct valid evaluations of outcomes 

that are related to proposed goals. Nine of the projects noted monitoring progress and 

outcomes as a problem—five as a major problem. Few project staffs have the 

necessary resources or competencies to follow the progress of their clients or students 

and adequately measure the outcomes. ARC could encourage more effective 

management and reporting by allocating resources to cover additional costs of a 

performance measurement system. Additional resources needed would vary with the 

type and scale of the evaluation, but in general this would only require about three to 

five percent of grant funds. To help grantees meet performance measurement needs, 

ARC could provide more specific guidelines and require that proposals specify how data

will be collected and used to measure project outcomes. 

5. Build regional technical assistance capabilities

These capabilities would each be targeted at some aspect of the market for 

entrepreneurial efforts (e.g., micro-enterprises, youth entrepreneurship, cooperatives, 

and technology or market development) or an industry cluster if a critical mass of a 

particular sector exists. These capabilities would not have to be developed at new 

organizations (although they could be) but more likely would operate in existing stable, 

high quality organizations or institutions. Each would be funded to further develop its 

expertise, provide technical assistance, store knowledge and information, provide 

contacts and broker relationships, conduct training sessions, and perhaps be 

responsible for conducting evaluations. Organizations in the ARC region with greater 

levels of capacity could be designated as mentors to provide technical assistance to 

less experienced groups. Those places that implemented REAL, for example, have 

access to a national support center and network that allows them to access information 

and get advice.
Evaluation of the ARC Entrepreneurship Initiative

14

6. Organize projects among dimensions other than function 

The functions that are currently used overlap in most projects—as well they should, 

because small businesses’ needs are invariably interrelated—and few programs are 

“pure” in the sense that they have a single purpose. Perhaps segregating programs 

according to those that build capabilities in organizations, those that aim to build 

capacities or change cultures in communities, and those that support program 

operations is a more useful taxonomy for understanding outcomes and successes. 

7. Make fewer small and short-term grants

Many of the grants are too small and too short to have any significant impact. Make 

fewer, longer, larger grants with sufficient resources and enough time to overcome 

unforeseen obstacles; pay high enough salaries to keep staff and build internal 

capacity; and buy technical assistance as needed. A longer grant period would give 

new programs time to get up to speed. Reducing uncertainty about year two funding 

would also address the issue of staff instability that impedes local capacity building.

8. Seek opportunities to highly leverage grants

Leveraging will result in greater scale of impacts of ARC funds and increase chances for 

success. However, the more ARC funds are leveraged, the more fungible the monies 

become, and the more difficult it is to distinguish their effects from those of 

complementary or supplementary funds. Examples are Ohio’s ACENet, New York’s 

Ceramics Corridor project and West Virginia’s Revolving Micro-Loan and Rural 

Outreach Program. In these programs, it is very difficult, if not impossible, to separate 

the outcomes associated with the ARC dollars from those associated with other dollars. 

Higher leveraging may mean accepting contributory rather than exclusive outcomes. 

9. Set realistic goals for sustainability

Government funding agencies often view grants that support proprietary initiatives and 

private businesses as temporary and assume that once the value of the service is 

realized and accepted, the firms or individuals ought to pay market price and the service 

should ultimately be self-sufficient. Experience of the projects and throughout the world 

has proven that self-sufficiency is rarely achieved in any program that either has 

catalyzing change as its goal or that operates in a weak economy and targets lowincome and marginal businesses. The need for post-grant sustainability turns grantees’ 

attention to raising funds for continuity. Short grant periods also make it more difficult to 

attract good project staff. To the credit of the grantees, most have been able to acquire 

funding to continue, but mostly from other government agencies, as described above. 

10. Acknowledge the risks inherent in the Initiative

An entrepreneurial development program is by definition a risky undertaking. The risks 

are compounded by the lack of economic prosperity in the Appalachian Region. Finding 

innovative and successful approaches, particularly in distressed areas and with lowincome populations, requires patience and acceptance of the potentially higher risk of 
Evaluation of the ARC Entrepreneurship Initiative

15

failure involved. Projects that fail to meet their goals can provide valuable lessons for 

more successful efforts in the future. There are tensions between helping 

disadvantaged individuals to support themselves through entrepreneurship and 

emphasizing new business creation that adds new jobs. The former goal speaks 

generally to low-skill and less-educated individuals whose businesses produce goods 

and services that are consumed locally and likely will never have large payrolls. The 

latter shifts the focus to manufacturing and high technology industries that may 

ultimately employ large numbers of workers and that usually sell their product outside 

the local economic region. Desire for quick and sizable job growth in traded sectors 

directs activity to larger employers, which may not be an appropriate target for an 

entrepreneurship program. Clearly, entrepreneurial support does not comprise an 

economic development program. Rather it is one strategy in what should be a broader 

effort.

11. Replicate successes

Now that the ARC region has a history of innovations and a good sense of what has 

proven successful, it may be fruitful to begin to replicate good practices. Many of the 

surveyed projects are replicable models—and provide considerable opportunities for 

further improvements and innovation. The Commission could build on what works or 

shows promise, at other sites as well as additional phases at the same place. Some of 

the grants to introduce REAL Enterprises represent a good example of replication in 

some places. Dissemination of information about best practices or good examples 

should not discourage innovation but rather encourage innovative improvements or 

adopting practices to different circumstances. This could be done through mentoring 

grants in which both the adopter and model program were given funds. 

12. Conduct a follow-up, later stage evaluation

Given the limitations of early stage evaluations, we urge the ARC to conduct another 

evaluation of mature open (i.e., in late stages of funding) and closed projects in order to 

learn more about findings in this report that were tentative due to the timing of the 

evaluation and to expand the data base and improve the reliability of the findings. 
Evaluation of the ARC Entrepreneurship Initiative

16

I. Introduction

Economic transition, exacerbated by the increased pace of globalization and 

technological advance, has continued to result in job losses in much of the Appalachian 

region. New sources of business activity are needed to diversify a regional economy 

that has been overly dependent on footloose manufacturing plants and vulnerable 

industries like mining and tobacco. 

One way to strengthen and diversify the region’s economic base is to pursue a strategy 

of “building from within” and bolstering the growth of indigenous companies. Such an 

approach promises a more stable Appalachian economy by nurturing homegrown firms, 

encouraging innovation and risk-taking, and fostering an environment conducive to new 

business formation. However, while there is evidence of entrepreneurial activity in 

Appalachia, the phenomenon is neither widespread enough nor at a sufficient scale to 

have substantial and sustainable regional impacts. In addition, research has found 

deficiencies in the region’s entrepreneurial support infrastructure. 

In response to these challenges the Appalachian Regional Commission (ARC), in 1997, 

launched a $17.6 million effort aimed at bolstering the entrepreneurial infrastructure of 

Appalachia. The ARC recognizes that Appalachia’s future economic growth and 

prosperity will be enhanced by a population that is better equipped with skills to start 

and manage new businesses and by structures that can support businesses in their 

start-up and early development stages. 

Entrepreneurship has many definitions. It can be distinguished from the concepts of 

small business creation or self-employment. According to one definition, entrepreneurs 

are “individuals who blend innovation with sound business practices to commercialize 

new products and services that result in high-growth firms.”2 From this perspective, 

entrepreneurial firms are innovators with high-growth potential. Entrepreneurial 

companies generally start out very small. But not all small businesses are necessarily 

entrepreneurial, if one accepts that growth must be a goal of entrepreneurship. In rural, 

distressed areas this distinction may not be quite as critical. In these Appalachian 

regions, small businesses of all types are needed—those with high-growth potential and 

also those formed for life-style purposes or self-sufficiency that primarily serve local 

needs. The ARC Entrepreneurial Initiative takes a broad view of entrepreneurship by 

promoting the creation of both life-style and growth-oriented firms. 

The primary goal of the ARC Entrepreneurship Initiative is to promote the creation and 

development of locally owned, value-added firms that will increase local wealth and 

provide employment opportunities for local residents. The ARC focuses its 

entrepreneurial development efforts in five general areas:

• Entrepreneurial education and training

• Entrepreneurial networks and clusters

• Technology transfer

 

2

 Jay Kayne, State Entrepreneurship Policies and Programs, Kauffman Center for Entrepreneurial 

Leadership, Kansas City, MO, November 1999.
Evaluation of the ARC Entrepreneurship Initiative

17

• Access to capital and financial assistance

• Technical and managerial assistance

Through November 2000, ARC had invested over $17.6 million in the Entrepreneurship 

Initiative, funding 169 educational, business assistance and capacity-building projects. 

Organizations that received grants reported leveraging $13.9 million from other funding 

sources. The 169 projects funded were distributed across all thirteen ARC member 

states. Of these projects, 83 supported the provision of entrepreneurial education and 

training; 86 provided technical and managerial assistance to firms; 48 assisted with 

enhancing access to capital and financial assistance; 53 were intended to support the 

formation of networks; and 14 focused on technology transfer.3 

The first 50 projects that were completed reported the creation of 249 new businesses 

and the creation or retention of 587 jobs. The 119 on-going programs are projected to 

create or retain 3,586 jobs in the region. 

Even though each grantee is required to measure and report on one or more of a set of 

standard “hard” outcome measures, the types of projects were quite disparate. They 

range from capital financing to new business formation to introducing an entrepreneurial 

culture through the public schools. Thus, the Appalachian Regional Commission asked 

an external evaluator to (a) examine its funded projects and their respective impacts in 

sufficient depth to verify the results; (b) identify outcomes that may not be captured by 

the standardized data collected; (c) compare project results to national and regional 

outcomes for comparable projects; (d) suggest performance criteria and measures that 

might better assess project and program success; and (e) recommend actions that may 

improve other entrepreneurial activities in the ARC region and influence the direction of 

future ARC initiatives.

In Fall 2000, ARC awarded Regional Technology Strategies, Inc. (RTS) a contract to 

assess the achieved and potential impacts of the Entrepreneurship Initiative grants 

based on a sample of 23-25 projects and to suggest strategies that might improve their 

success rate and increase the value they add to their economies.

The Initiative awarded grants to projects in the five areas mentioned: (1) entrepreneurial 

education and training, (2) entrepreneurial networks and clusters, (3) technology 

transfer, (4) business and management assistance, and (5) access to capital and 

financial assistance. Most projects, however, included services in more than one 

category. Recognizing the wide scope of programs funded under the Initiative, the 

research team paid considerable attention to identifying the appropriate selection 

criteria for the sample of projects to be evaluated. These criteria capture individual 

project results while at the same time provide appropriate information for cross-case 

analysis so that summative results may be obtained.

The methodology, described below, was designed to accomplish three objectives:

• establish, benchmark, and verify evaluation criteria appropriate for the range of 

entrepreneurship projects sponsored by ARC. 

 

3

 Total exceeds 169 because most projects have multiple objectives. 
Evaluation of the ARC Entrepreneurship Initiative

18

• develop an assessment of performance and results of selected projects, using 

the evaluation criteria 

• draw conclusions and provide information to guide policy, programmatic, and 

reporting procedures for ongoing and future ARC entrepreneurial initiatives

II. Methodology

The evaluation is based on the following information and intelligence:

• analyses of existing national, international, regional, state, and local evaluations; 

• information submitted by grantees to and compiled by the ARC and selection of 

sample of projects to be evaluated; 

• information submitted by project managers in response to surveys, requests for 

documentation, and lists of clients, customers, and/or stakeholders; 

• surveys of selected clients, customers, and/or stakeholders; and 

• visits to two project sites. 

The nature and timing of the grants influenced the evaluation methodology and the 

results. First, the projects funded vary not only by project types but also by approaches 

and by targeted client/customer populations. For example, some projects are aimed at 

expanding or sustaining existing entrepreneurial micro-enterprises, some target youth in 

an attempt to influence their attitudes and future aspirations—but not necessarily their 

immediate actions. Some attempt to disseminate information or alter a community’s 

“culture,” while others work to create new companies. Thus, it is evident that these very 

diverse project goals require different evaluation perspectives and measures. Second, 

projects also vary in their geographic and demographic emphasis, i.e., regional versus 

local economic impacts, serving broad population groups versus targeting specific 

segments of the population. Third, the sample of projects evaluated was in fact biased 

toward an early program stage mix of activities that favored education and training and 

networks. Subsequent projects funded have included a higher proportion of projects 

such as capital access and technical assistance, which are more likely to produce 

quicker outcomes. As a result, one might expect increasing evidence of economic 

outcomes for the program over time. Fourth, the projects evaluated vary in their time to 

fruition; some are multi-year and, in some instances, multi-phase and some are single 

events. Entrepreneurial outcomes generally require considerable time and patience to 

allow for gestation and development. Finally, some of the projects (generally the newer 

ones) altered their work plans during the course of the projects, which could affect their 

outcomes. 

The research team addressed these evaluation design issues by (a) considering 

different stakeholder objectives in the survey instrument design and interview protocol 

and (b) relating project assessments back to the initial and subsequent iterations of 

ARC objectives. RTS identified findings related to performance and outcomes 

appropriate for particular project types and, through cross-case analysis, identified 

areas of commonality and variation. 
Evaluation of the ARC Entrepreneurship Initiative

19

Using a cross-case analysis, the research team attempted to discern elements of 

similarity and difference, compare evaluation metrics and outcomes between and 

across project types, and, where appropriate, compare outcomes and results from ARC 

projects to those reported in similar national, state, and local program evaluations. 

Because of the absence of any control group of communities and limited sample size, 

the statistical tools used to analyze the cross-case results are straightforward. 

Nonetheless, we believe we have gathered useful information concerning the types of 

outcomes most often achieved and the types of programs most often successful in 

meeting their objectives. 

A. Review of Literature

The assessment began with a review of other project evaluations at the international, 

national, state, and local levels that were similar or comparable to those sponsored by 

the ARC. These were classified according to the same five categories used in the ARC 

initiative—access to capital and financial assistance, business and management 

assistance, entrepreneurial education and training, entrepreneurial networks and 

clusters, and technology transfer. The review produced the results summarized in 

Table 1 and described in more detail in Appendix D. It became the starting point for 

developing the methodology. 

The literature revealed several appropriate metrics useful in evaluating 

entrepreneurship programs. For example, the metrics used to evaluate seed and 

venture capital programs typically are a mixture of activity and accomplishment 

measures. Internal activity measures include deal flow, debt/equity ratio, number of 

deals exited, leveraging of private funds, and internal rate of return. Meanwhile, 

accomplishment measures include tax revenue generated, success of funded 

businesses, and whether the target population or area was served. With the exception 

of tax revenue, the metrics mostly focus on microeconomic, programmatic impacts 

rather than macroeconomic changes, highlighting the difficulty of detecting large-scale 

impacts even of high-investment programs.

As one would expect, the procedure for evaluating micro-finance programs is 

substantially different. The most common metrics used to evaluate such programs 

include participant income, rates of entry into self-employment, length of selfemployment spells, length of unemployment spells, the degree to which the program 

reaches its target population, and a variety of qualitative variables that fall under the 

category of “empowerment.” Like those used to assess the performance of venture 

capital programs, these metrics are largely micro- rather than macro-level. As the name 

indicates, the relatively small scale of micro-finance programs suggests that it is difficult 

to connect them with macroeconomic change.

The literature review examined assessments of diverse programs providing 

management and technical assistance to entrepreneurs. These programs range from 

micro-enterprise programs, such as those who participated in the Aspen Institute SelfEmployment Learning Project, to a national network of Small Business Development 
Evaluation of the ARC Entrepreneurship Initiative

20

Centers, which is a partnership between the Small Business Administration and local 

institutions of higher education.

Micro-enterprise programs target potential or existing entrepreneurs and focus 

performance measurement on indicators such as the number of new business start-ups, 

survival rates, and expansions. Small business assistance programs measure their 

success in the number of new jobs created and the wages paid new workers at assisted 

firms. The examples of performance indicators are not mutually exclusive; rather the 

difference is a matter of emphasis. Both micro-enterprise and small business 

assistance programs consider customer satisfaction, which can be measured with a 

customer survey and/or indicated by a willingness to pay for services. All programs 

face resource constraints, and thus performance indicators include efficiency measures. 

Funding agencies and program administrators track not only the number of jobs created 

but also the cost per job created. 

Business incubators are often the vehicles through which assistance is delivered to 

start-up companies. An incubator can take the form of a multi-tenant building that 

provides affordable office space for new firms. Incubator firms typically share critical 

office services and equipment in order to reduce overhead costs. In addition to office 

services, incubator programs may provide affordable rent, managerial and technical 

assistance, financial assistance, and increased opportunities for interaction with firms 

both inside the incubator and within the larger local economy. A recent national study of 

50 incubator programs used measures such as sales growth, firm survival rates among 

incubator graduates, number of new technologies, patents/trademarks developed, and 

jobs created in the community to evaluate incubator outcomes.4 

The benefits resulting from management and technical assistance to small businesses 

are commonly defined at three levels: (a) the individual assisted, (b) the firm (if there is 

one) assisted, and (c) the community. While the program goal is to help an individual 

start and grow a business, that is not the only potential positive outcome. There are 

benefits derived from a person learning that he/she lacks an entrepreneurial aptitude, 

personality, or skills and/or has a weak business idea and thus avoids a debacle. For 

some, entrepreneurial training may convey entrepreneurial skills that help them obtain 

better jobs or advance in their current jobs. A comprehensive performance 

measurement system goes well beyond counting business start-ups. It tracks the 

economic situation of program clients (or a sample of clients) for a period after they 

have received services.

Desired outcomes at the firm level include increased profitability and growth as 

measured by sales and employment. Desired outcomes at the community level also 

are measured in terms of job growth. Additional community benefits might include a 

more diverse economic base, greater wealth, and increased corporate involvement in 

non-profit, volunteer activities. 

 

4

 Hugh Sherman and David Chappell, Methodological challenges in evaluating business incubator 

outcomes, Economic Development Quarterly, Vol. 12, No. 4, Nov. 1998, pp. 313-321.
Evaluation of the ARC Entrepreneurship Initiative

21

Entrepreneurship education provides skills that are not ordinarily taught in general 

curricula, while also giving students an opportunity to experience the practical effects of 

the skills being acquired. While there is a long tradition of business concepts being 

taught in higher education, entrepreneurial programs are now expanding into primary

and secondary school as well. Studies of these programs demonstrate that 

entrepreneurship is a topic of interest among students of all ages.

There are numerous issues that make it challenging to evaluate the effectiveness of 

entrepreneurial education.5 These include (1) lack of consensus on a definition of 

entrepreneurship and what should be included in entrepreneurial education, (2) rapid 

change in programs over time, (3) many entrepreneurship education programs are in 

their infancy, and (4) lack of consensus on how to measure success. Moreover, there 

are multiple factors, in addition to education, that might motivate someone to start a 

business. Previous studies of entrepreneurial education programs have examined a 

variety of performance measures including number of business start-ups, firm survival 

rates, satisfaction levels among graduates and their contributions to the local economy.6 

Most evaluations of networks include qualitative factors. Among the most often used 

metrics are firms’ perceptions of the usefulness and impact of network membership, 

changing attitudes about inter-firm collaboration, firms’ level of trust and cooperation 

with other firms, level of commitment to the network, and the competitive environment of 

member firms. Evaluations focus on these characteristics because they are some of 

the most significant for determining a company’s willingness to participate in and 

commit to joining a network. Other qualitative characteristics include impact on supply 

chain relationships, firms’ assessment of services offered by the network sponsor, and 

firms’ opinions of the hindrance and success factors of business networks.

Evaluations of networks also include quantitative characteristics, especially those 

evaluations that seek to construct a typology. Quantitative variables most commonly 

measured include differences in employment levels, revenues, and profitability; number 

of shared activities undertaken by member firms; and characteristics of member firms 

such as size, sales, and markets. There does not, however, appear to be a set of 

benchmarks determining a standard for the level of quantitative change that would mark 

success or failure of a network. Most evaluators, therefore, include qualitative as well 

as quantitative measures.

Some of the intermediate outcome measures for technology transfer programs include 

adoption of new technologies, level of new investments, development of new products, 

and increased productivity and value-added. Longer-term economic outcomes may be 

 

5

 Nancy Upton, et al., Have we made a difference? An examination of career activity of entrepreneurship 

majors since 1981, Baylor University, http://hsb.baylor.edu/html/cel/ifb/research/babpap.htm.

6

 Z. Block and S. Stump, Entrepreneurship education research: Experience and challenge, 1992. In D.L. 

Sexton and J. Kasarda (eds.), The State of the Art in Entrepreneurship, Boston: PWS-Kent Publishing. 
Evaluation of the ARC Entrepreneurship Initiative

22

measured in terms of increased profitability and size of firms, greater degree of 

innovation among firms, and increased firm-level employment.
Evaluation of the ARC Entrepreneurship Initiative

23

Table 1

Sample Performance Metrics for Assessing Entrepreneurial Programs

Benefits To: Process Outputs Intermediate Outputs Economic Outcomes

INDIVIDUALS 

Education & 

Training

Attendance at classes, seminars, 

and meetings

Retention rates

“Graduations”

Information and skills acquired

Improved productivity of individual

Customer satisfaction 

Employment/self employment

Increased income

Increased no. of business starts

FIRMS 

Management & 

Technical 

Assistance 

Number/hours of one-on-one 

counseling sessions

Market information provided

Business plans completed

Loan applications completed

Access to office space and equip.

Loans obtained

Expanded sales/revenues

Business procedures modified

Increased productivity and/or value added

Customer satisfaction

Reduction in overhead costs 

More profitable and/or larger firms

More competitive firms

Increased employment at firms

FIRMS

Technology 

Transfer 

Number/hours of one-on-one 

counseling sessions

Information provided about 

applicable technologies

Classes and seminars

Introductions and contacts

Adoption of new technology

New investment

Development of new product

Increased productivity, value added

Customer satisfaction

More profitable and/or larger firms

More competitive firms

Increased employment at firms

Expanded sales/revenue

Increased market share

FIRMS

Access to 

Capital

Loan applications prepared and 

received

Amount of funds dispersed in 

loans/number of loans

New investment in firms

Increased productivity, value added

Payback rate

Customer satisfaction

Increased access to venture/equity capital

More profitable and/or larger firms

More competitive firms

Increased employment at firms

Faster regional growth

INDUSTRY 

Networks and 

Clusters

Attendance at network events

Association membership

Multi-firm partnerships

Increased productivity and/or value added 

for industry

More profitable and/or larger industry, More 

competitive industry

Increased employment in the industry

COMMUNITIES Community meetings

Local press coverage on 

successful local entrepreneurs

Attention from local leadership

Increased employment 

Reduced under and unemployment

Community satisfaction

Increased tax revenues 

Changes in local bank assets lending (proxy 

for wealth and access to capital)

Increased household and per capita 

incomes

Changes in participation rates in free and 

reduced school lunch program

More diverse economic base

Source: RTS literature review (see Appendix D)
Evaluation of the ARC Entrepreneurship Initiative

24

B. Information from Project Files

The evaluation team catalogued and organized the information about the 

entrepreneurship projects that was available in ARC’s files. This included the original 

proposals, staff summaries of projects, correspondence between projects and ARC, and 

interim and final reports. Some core data about the project, such as location, grantee 

organizations, project managers, project budgets, time lines, and reported outcomes, 

were entered into a database. With respect to outcomes, the ARC has a set of process 

and economic measures by which it holds projects accountable. These include—on the 

process side—numbers of students, trainees, participants, and businesses served and 

new telecommunications sites and—on the outcome side—numbers of jobs retained 

and created and of new businesses created. This information was used to (a) select the 

sample of projects to be evaluated and (b) establish baseline conditions against which 

to measure outcomes. 

C. Sample Selection

The intent of the evaluation was to select a sample of between 23 and 25 grants 

stratified roughly (a) in proportion to the numbers of grants in which each of the five 

categories of activities is included in projects, (b) by concentrations by state, and (c) by 

size of grant. The ultimate selection process, however, was limited by the length of time 

since or to completion of the project. Since economic outcomes of most development 

initiatives—particularly those that rely on business startups and expansions—take 

considerable time to demonstrate measurable results, projects that had not been 

operating long enough had to be eliminated. The original intent was to include projects 

that are either closed or beyond a milestone that was expected to produce results. The 

process also eliminated grants that were considered too small in scale to have 

measurable impacts—generally a single event, such as a conference. 

When the evaluation began, 133 projects had been funded7. The grants that had been 

completed or were near completion (i.e., two or more years of operational experience) 

totaled 48, which was the “potential” sample base of projects. Of the 48, 25 were 

already closed, and 23 were nearly closed. After screening out those projects 

undertaking one-time events such as conferences or which were very small, the actual 

sample was 24 projects. These 24 grants did in fact roughly match the proportional mix 

of types (Table 2). Thus, the sample includes the largest number of projects focusing 

on entrepreneurial education and training and the next largest on technical and 

managerial assistance and fewer on technology transfer. But 16 of the 24 targeted 

more than one category. The six grants with a single focus were all entrepreneurial 

education and training.

 

7

 The number of funded projects increased to 169 during the period from the start of the evaluation to the 

completion of the final report. 
Evaluation of the ARC Entrepreneurship Initiative

25

Table 2

Number and percentage of projects by category

Category All Grants Sample

Number Percent Number Percent

Entrepreneurial education and training 83 58% 21 88%

Entrepreneurial networks 53 37% 9 38%

Technology transfer 14 10% 3 13%

Technology and management assistance 86 61% 15 63%

Access to capital and financial assistance 48 34% 7 29%

The assessment also roughly matched the mix of grants that had been made to date 

(Table 2), which resulted in sampling all ARC states except North and South Carolina, 

which together received only seven grants. 

Table 3

Number and percentages of grants by state, as of September 2000

State All Grants Sample

Number Percent Number Percent

Alabama 14 9.9% 1 4.2%

Georgia 5 3.5% 1 4.2%

Kentucky 12 8.5% 3 12.5%

Maryland 8 5.6% 1 4.2%

Mississippi 6 4.2% 2 8.3%

New York 14 9.9% 4 16.7%

North Carolina 3 2.1% 0 0%

Ohio 27 19.0% 3 12.5%

Pennsylvania 7 4.9% 2 8.3%

South Carolina 4 2.8% 0 0%

Tennessee 6 4.2% 1 4.2%

Virginia 20 14.1% 3 12.5%

West Virginia 16 11.3% 3 12.5%

D. Survey Design

The next step was to design a survey instrument for project managers that would 

supplement the existing reported economic outcome data with fields that reflected the 

outcomes and evaluation metrics and methods in the project proposals. Although the 

goal was to identify and measure economic outcomes, some of the projects in the 

portfolio emphasized other outcomes—particularly those that sponsored education 

activities at grade school level and therefore have only very long-term consequences. 
Evaluation of the ARC Entrepreneurship Initiative

26

Informed by the reviews of project documentation, the literature review, and discussions 

with ARC staff, the questionnaire in Appendix C was mailed to all project managers. It 

included a cover letter from ARC’s executive director to lend legitimacy to the endeavor 

and increase the likelihood of response. The survey instrument requested names of 

least six clients served and/or stakeholders plus additional information describing 

outcomes of the project (e.g., progress reports, survey results, newsletters, and case 

studies). 

This survey protocol included the following elements: 

• confirmation of project activities, timing, and status; 

• funding, staff time, and other inputs invested in the project that supplemented ARC 

resources; 

• intermediate steps undertaken by the project, e.g. workshops held, number of 

technical assistance visits made, number of companies assisted in an incubator; 

• methods and metrics used to monitor project activities; availability of written 

progress and final reports; methods used to obtain information, e.g. surveys, followup meetings, case studies, focus group or advisory group feedback; outside 

evaluations; 

• outcomes and results from the project, as tracked through formal evaluative 

procedures and also understood through informal feedback and experience; 

• comparison of achieved outcomes after project activities with anticipated outcomes 

at proposal stage before project was undertaken; 

• reporting (i.e., how are results from project activities communicated to stakeholders, 

in addition to ARC, and feedback obtained from this); and 

• learning (i.e., what has been learned as a result of the project and its outcomes, how 

has this fed back into ongoing or future project activities, and what metrics might be 

more appropriate or realistic in future activities of this kind).

E. Client Survey

Two clients were selected at random from those submitted for a telephone interview. In 

some instances all of the clients/participants were underage students so individuals 

from partner organizations were interviewed instead. These interviews were used to 

further probe participants’ experiences with the programs, resources invested, and 

outcomes achieved. This process was designed not only to verify what the program 

managers reported but also to surface results and outcomes that programs may not 

have been fully appreciated. The protocol for the client survey is included in Appendix 

C. 
Evaluation of the ARC Entrepreneurship Initiative

27

F. Site Visits

After the information and interviewing process was completed, two projects were 

selected for site visits to collect more in-depth information. The first site was the North 

Georgia Technical Institute Entrepreneurship Demonstration Project serving a sixcounty mountainous, scenic, and essentially rural part of northern Georgia. The host 

institution was North Georgia Technical College (NGTC). The catalyst for this project 

was the college president, who believed there was an unmet need for entrepreneurship 

that could help grow the travel and tourism industry. A study by the Georgia 

Department of Technical and Adult Education that found 23 percent of the individuals 

graduating from a technical institute between 1987 and 1997 started their own business 

helped document the need. The project goal was for at least 15 students to complete 

three elective entrepreneurial education courses (all offered at three sites) that could be 

merged into a certificate program and to create a Small Business Resource Center on 

the college’s Clarkesville campus that provides no-cost technical and managerial 

assistance. In year two, the project would also work with local high schools to provide 

entrepreneurial education to their students, and expand the local network of education 

and development agencies supporting entrepreneurship. 

When the grant expired, 35 students were enrolled in entrepreneurship courses, and 

therefore the State funded it through June 2000 to allow them to complete the program. 

Another full year of state funding will support the entrepreneurial course offerings 

through June 2001. Currently, 70 students are taking entrepreneurial education 

courses with five enrolled in the certificate program. Course enrollments did not meet 

initial targets, new business creation exceeded projections. Enrollment in year one 

totaled 60 students in 5 offerings, and the number of participating individuals was 43, 

close to the projected number. Three new businesses were started by program clients. 

In year two, total enrollment reached 88 (projected to be 110)—primarily students taking 

an entrepreneurial course as an elective within another course of study and not seeking 

certification. Ten new businesses were started, almost 50 percent over the projection. 

Staff turnover, insufficient partner involvement, and lack of management capabilities 

have proved barriers to meeting all of the project’s goals, such as offering courses at 

local high schools and increasing enrollments in the certificate program (which may 

have been an unrealistic aim). Yet the project appears to have helped institutionalize 

entrepreneurial courses throughout the state’s college system. 

The second site was the Levi Strauss Employees Entrepreneurial Training in Knoxville, 

Tennessee. This project was funded to help ease the impact on the community after 

Levi-Strauss announced plans to close its Knoxville plant in 1998. After discovering that 

a large number of workers about to be laid off—most of whom were African-American, 

female, and heads of households—already had a small business on the side or were 

interested in starting one, the Council for Adult and Experiential Learning started a 

support program. ARC awarded a grant to Economic Ventures, Inc. (a non-profit 

subsidiary of the Knoxville Community Development Corporation) and the Pellissippi 

State Small Business Development Center to further assist these displaced workers 

who were interested in entrepreneurship. EVI organized peer lending groups and 

training classes, and helped package loans. But the separation package from the Levi 

Strauss Foundation included $6,000 grants to displaced workers who completed a 
Evaluation of the ARC Entrepreneurship Initiative

28

business plan, which reduced interest in a peer lending program that began with a loan 

of only $500. The project augmented existing activities already underway, such as 

seminars on how to complete the Business Start-Up worksheets. The program 

established an office on-site at the Levi plant where workers received technical 

assistance both one-on-one and in group settings. The SBDC staffed the office with 

one specialist for 20 hours a week during the initial nine months. EVI offered seminars 

and individual counseling in credit education. 

The results of this initiative are well documented and quite impressive. Overall, service 

delivery reached over 90 percent of those displaced Levi Strauss workers who were 

interested in entrepreneurship. Of the 2,300 workers displaced by the plant closing, 945 

indicated interest in entrepreneurial assistance, and 880 received assistance from the 

project. The 17 peer lending groups formed involved a total of 85 new companies. Of 

the 880 individuals receiving services, 102 are full-time entrepreneurs, and all these 

businesses are still in operation. The other 778 participants have chosen to work fulltime and run a small business on the side to supplement their income and perhaps to 

develop a second career after retirement. The major limitation on regional outcomes is 

that almost all businesses created serve a local market and are not likely to make a 

large contribution to the economic base of the community. But they do provide a living 

to their owners, and the enhanced availability of services improves the quality of life for 

residents. 

III. Findings and Analysis

The projects evaluated represent about one-fourth of the awards that were made in the 

initial stages of the Initiative. However, because the grants had to have been closed or 

near closed for the evaluation to be meaningful, the eligible projects (on the basis of 

time since inception) totaled only 48 projects. Of that 48, only 24 projects were 

selected for the study after further screening to ensure a representative sample that was 

fairly proportional to the universe of grants made. Three-fourths of the surveyed 

projects were closed, and the remaining fourth were near closure. Of the projects 

visited, one was closed and one in the process of closure. The characteristics of the 

projects evaluated appear to be approximately representative of all the awards made to 

date in terms of the distribution of aims, scale, location, and sector. (See Tables 2 and 

3). Of the 24 projects selected for assessment, 23 responded to the survey 

questionnaire. Further details of the survey results are contained in Appendix B. 

A. Caveats

This analysis must begin with several caveats. First, there may be a bias toward 

positive results. Much of the data collected was self-reported and thus may be subject 

to some level of subconscious embellishment. Further, because the clients interviewed 

were selected from among a larger number provided by the grantees, there may be a 

bias towards satisfied and/or successful clients. 

Many of the grants were made to existing organizations with business support missions, 

and the ARC grant funded activities that either continue or supplement much more 
Evaluation of the ARC Entrepreneurship Initiative

29

substantial related activities. As a result, it may have been difficult for the grantee to 

assign specific outcomes to activities carried out with the ARC grant funds. Some of the 

seemingly larger-than-life outcomes may have been due to aggregation of outcomes 

with those from other related programs or grants. 

Two other factors complicate the assessment. The first is that nearly half of the 

programs surveyed reported altered goals between the award and our assessment. 

The second is that most of the projects were relatively young and when the evaluation 

was conducted had been in operation for only 18 to 21 months. Too little time has 

elapsed from inception to evaluation to measure ultimate outcomes. With the 

evaluation occurring close to project completion, it is likely that costs are underemphasized and benefits overemphasized (Shapira and Youtie, 1998). For example, 

many small firms will require additional support, and many are likely to go out of 

business within a year or two of establishment. On the other hand, there will also be a 

few significant successes, but these will not be apparent until more time has elapsed. 

B. Characteristics of Projects

Scale and Capacity: The awards made under the ARC Entrepreneurial Initiative have 

been on average relatively small. For the group of surveyed projects, the median grant 

was just under $60,000. In aggregate, the ARC grants provided just over half the 

funding for the evaluated projects. Clearly this effort is concentrating on relatively smallscale efforts. Half of the project teams had staffs of fewer than three people, even 

during their peak period of effort. Only two projects reported a staff of five or more 

people at peak.

About half of the grants were made to non-profit economic development agencies, and 

the other half were divided (no category with more than two awards) among government 

agencies, regional planning agencies, community organizations, universities, and 

colleges. The grant recipient organizations also tended to be small. Thirteen of the 

organizations reported total organizational budgets of less than $1 million. 

The size of the organizations that were awarded the grants is of interest because it may 

be associated with staff support capacity and long-term sustainability and has 

implications for the ability to manage not only project activities but also federal grant 

funds and reporting requirements. The capacity of small organizations to manage 

federal grants is critical to success, yet often limited. The project visits suggest that 

prior experience with federal grants is more important than the size of the organization. 
Evaluation of the ARC Entrepreneurship Initiative

30

Figure 1

Distribution of Size of Overall Organizational Budgets

4.3

8.7

43.5

26.1

17.4

0

10

20

30

40

50

Percent of 

projects

<$50K $50 -

249K

$250 -

$1M

$1M -

$10M

>$10M

Figure 2

Distribution of ARC Project Budgets

47.8

30.5

13

8.7

0

10

20

30

40

50

Percent of 

projects

<$50+ $50 - $100K $100 -

$200K

>$200K

Figure 3

Distribution of Peak Project Staff Size

56.5

34.8

8.7

0

10

20

30

40

50

60

Percent of 

projects

1 or 2 3 or 4 More than 4

Source: Survey of ARC Entrepreneurship Projects, 2000. (N=23.)
Evaluation of the ARC Entrepreneurship Initiative

31

Target areas/populations: Most surveyed projects (70 percent) targeted multi-county 

regions, but only three spanned more than one local development district and only one 

crossed a state border. The small target areas are consistent with the small project size 

and generally represent either the grantee organization’s service area or the portion of 

that service area that lies within the ARC region. Forty percent of the evaluated 

programs specifically targeted disadvantaged populations, while 35 percent targeted 

youth. The most specific program targeted workers displaced by the closing of the LeviStrauss plant in Knoxville.

The majority of sampled projects (57 percent) were located in counties designated as 

distressed by ARC (Table 4). Thirty-five percent of projects surveyed were in counties 

designated as non-metropolitan by ARC and 61 percent were in both metro and nonmetro counties. 

Table 4

Distribution of Projects by ARC Area Designation

ARC County 

Designation

ARC 

Score

Surveyed 

Projects

Area 

Designation

Number of 

Projects

Percent of 

Projects

Distressed 1.00 2

1.50 7

1.75 4

Distressed 

(all or in part)

13 57%

Transitional 2.00 9

2.50 1

Transitional 10 43%

Competitive 3.00 - - - -

Attainment 4.00 - - - -

Total - 23 23 100%

Note: ARC score values between the ARC county designations are multi-county estimates provided 

by ARC staff. Area designations to either “distressed” or “transitional” are made by RTS.

Internal evaluations: The internal evaluation and monitoring systems used by the 

projects tend to lack specific outcome measures. Although many grantees claimed to 

have conducted internal assessments, few could document any tangible and 

measurable economic results. This weakness was confirmed in the survey by the large 

number of grantees (43 percent) that named “monitoring outcomes” as a problem. The 

most common types of assessments used were based on degrees of learner or 

customer satisfaction at the conclusion of an intervention (or program) as measured by 

survey instruments. Only two education and training projects and three technical 

assistance project conducted evaluations at some period of time after completion. This 

is consistent with large-scale evaluations of programs such as Small Business 

Development Centers, which have been found to have difficulty with self-assessment 

due to imprecise performance metrics and inconsistent data collection.8 

 

8

 Johnson, S.L. and Holly M. Mudd. Performance Measurement in the SBDC Program, Inspection Report 

No. 98-09-01. Washington, D.C.: U.S. Small Business Administration Office of Inspector General, 1998.
Evaluation of the ARC Entrepreneurship Initiative

32

Resources: ARC funds comprised half the total budgets of the projects, in aggregate. 

Of the remaining funds, nearly a quarter were “other,” a fifth were state or county, and 

only one tenth of one percent were from fees for services. The distribution of leveraging 

was:

Table 5

Relationship of Percent Match to Scale

Grantees

Percent Number

Percent Match

42% 8 25% or less

26% 5 26% to 100%

32% 6 More than 100%

Somewhat surprisingly, the largest matches were from the smaller organizations. Of 

those reporting more than a 100 percent match, two-thirds had organizational budgets 

of less than $1 million. All grantee organizations used ARC funding to leverage or 

match other funds.

Range of goals: The ARC Entrepreneurship Initiative was designed to ensure that 

Appalachian communities have the means to start, manage, and expand local 

businesses. These include access to capital and financial assistance; entrepreneurial 

skills and knowledge (education and training); networking and sector development 

(social capital); technology transfer; and technical and managerial assistance. All 

funded projects embraced at least one of these categories. 

 

Most projects are multi-purpose. Only two projects in the sample had only a single 

purpose as defined by ARC. Both were education providers. This reflects project 

developers’ sensitivity to the clients’ preference for “one stop shops,” and intermediaries 

that are able to provide a range of real and brokered services. Most needs of small 

businesses are multi-dimensional. For example, acquiring new technology requires 

new skills and often capital and new management systems. Still, a common complaint 

of a client was the inability to acquire a necessary but related service from the grantee, 

such as access to capital or technical materials. 

There was significant emphasis among grantees on using networks. But most actual 

activities worked with existing networks and introduced new members, and did not form 

new networks (only 5 percent). It is evident, however, that the value of collective action 

and cooperative efforts is becoming widely accepted and embedded in the conventional 

wisdom of regional economic development. This finding is reinforced in current network 

literature.9

 

9 Cf. Kingsley, Gordon. Establishing the Building Blocks: A Follow-up Evaluation of USNet’s Foundation 

Forum on Inter-Firm Collaboration. USNet Evaluation Working Paper No. 9504. Chapel Hill, N.C.: 

Regional Technology Strategies, Inc., 1995, and Fulop, Liz, and Jo Kelly. Survey of Industry Network 

Initiatives in New South Wales. Final Report prepared for the Strengthening Local Economic Capacity 

Project, an initiative of the Commonwealth Department of Housing and Development. Kingswood, 

N.S.W., Australia: University of Western Sydney Nepean, 1995.

Source: Survey of ARC Entrepreneurship Projects, 2000. (N=23.)
Evaluation of the ARC Entrepreneurship Initiative

33

Frequency of categories of activity: The most common major activities of the 

projects are: (a) one-on-one assistance for new firms (74 percent); (b) education and 

training or adults (65 percent); (c) one-on-one assistance for existing firms (48 percent); 

(d) business networks of new firms (48 percent); (e) custom software development (48 

percent); and (f) business seminars (44 percent). (See Table 6.) Technology related 

activities, such as technology transfer or technology opportunities research, were quite 

infrequent and in fact were cited as a major emphasis by only four of the 23 projects 

surveyed. 

Table 6

Frequency of Major and Minor Activities Reported 

by ARC Entrepreneurship Projects

ACTIVITY Major 

Emphasis

Minor 

Emphasis

Percent

Major

% Major 

or Minor

One-on-one assistance-new start-ups 17 3 74 87

Education and training of adults 15 2 65 74

One-on-one assistance-existing firms 11 8 48 83

Business networks-new start-ups 11 9 48 87

Custom software development 11 1 48 52

Business seminars 10 3 44 56

Curriculum development 8 4 39 52

Business opportunities research & marketing 9 7 39 70

Business networks-existing businesses 9 8 39 74

Training of staff and other trainers 8 9 35 74

Local and regional needs assessment 8 7 35 65

Loan or equity fund 8 3 35 48

Referral systems to other service providers 7 8 30 65

Community conferences, meetings 6 10 26 70

Education and training of students 5 4 22 39

Technology opportunities research 4 10 17 61

New physical facilities developed 4 4 17 35

New economic development strategies 4 11 17 65

Government procurement assistance 2 6 9 35

Distance learning 2 1 9 13

Web site 2 8 9 44

Business internships 1 5 4 26

The ARC funded projects appear to be well connected to their communities and to 

broader economic development efforts. Nearly all (96 percent) of the project managers 

surveyed reported that they met with other community leaders to discuss and plan the 

project. Eighty-three percent referred clients to other sources of assistance, which 

clients recognized and appreciated. The most common services delivered were adult 

education (83 percent), individual assistance to adults starting new business (70 

percent), assistance to networks (52 percent), and assistance to existing firms both with 

and without employees (52 percent). 
Evaluation of the ARC Entrepreneurship Initiative

34

Figure 4

Percent Frequency of Services Provided to Participants

Reported by ARC Entrepreneurship Projects

0% 20% 40% 60% 80% 100% 120%

E&T-Youth

Direct financing

Assist-firms with employees

Assist-firms w/o employees

Assist-networks

Assist-adult entrepreneurs

E&T-Adults

Referrals

Mtgs w/community leaders

Many services were free, and where clients did pay a fee for their service it was 

nominal—less than $100 and often as low as $10. Therefore, it is highly unlikely that 

these programs will become self-sustainable based on income. 

Scale of intervention: The intervention or services received by a client can range from 

a simple one-time response to a question to intensive support extending over a period 

of months. Although impacts are not necessarily related to time spent together, the 

duration of the interaction is one indicator of the importance of the intervention to the 

outcome. 

The reported intensity of interaction was greatest for youth education (where the project 

usually had a captive audience) and for assistance to entrepreneurs with no employees 

starting a new business. It appears that total class time was included in time for youth 

entrepreneurship programs, whereas the intensity of interaction was much lower for 

adult education, where more than half reported less than eight hours of contact, and for 

assistance to firms with employees.

The data on intensity of the interventions, as indicated by hours of contact, is quite 

skewed. For example, one project at the University of Alabama is responsible for 89 

percent of the businesses with employees receiving more than 30 hours of assistance 

and 71 percent of adults receiving more than 309 hours of assistance in starting new 

businesses and one in New York is responsible for 55 percent of the mid-range 

assistance (8 to 30 hours) in the latter category. 

In the programs that make loans, the reported amounts are quite small. More than half 

of those loans made were less than $25,000. Although many of the loans went to 

micro-enterprises in which small amounts can have a significant impact, these amounts 
Evaluation of the ARC Entrepreneurship Initiative

35

generally are too small for the kind of technology-based businesses that are likely to 

generate growth and impact an economy. 

The distribution of both activities and outcomes are highly skewed and, with respect to 

most, one or two programs produce a very large proportion of the total outcome. This is 

not surprising since the scale and regional scope of the projects are quite varied, but it 

does distort many of the aggregated statistics10. For example:

• Two projects account for 79 percent of all adults trained, one under Team 

Pennsylvania and one the Lighthouse Program in West Virginia, both covering large 

geographic multi-county areas.

• Two projects account for 86 percent of all youth educated, one in Kentucky (Forward 

in the Fifth) and one under Team Pennsylvania (REAL), again both responsible for 

large multi-county regions. 

• Two projects, the Levi Strauss project and the ACCORD project, account for 78 

percent of all new firms started without employees.

C. Degrees of Satisfaction

Clients served generally expressed high levels of satisfaction with the funded projects. 

Of those interviewed, 70 percent expressed high satisfaction with the services they 

received, 17 percent said they were satisfied, and only two were dissatisfied. Fortythree percent said the assistance or advice they received exceeded their expectations, 

47 percent said it met them, while 10 percent still had unmet needs. While these 

responses are overwhelmingly favorable, it is important to emphasize that the lists of 

clients from which those surveyed were drawn were provided by the projects 

themselves.

Another dimension of satisfaction is that of the grantees with ARC’s technical support. 

Two-thirds of the project managers surveyed found ARC technical support moderately 

or very useful, but seven responded that they did not receive any. All but three said 

ARC provided opportunities to learn about the experiences of other similar programs. 

D. Economic Outcomes and Impacts

The conclusions about outcomes and impacts were synthesized from four sources of 

information: (1) surveys from project managers; (2) personal conversations with project 

managers; (3) surveys from clients of programs; and (4) data submitted to the ARC in 

progress and final reports. Additional and more detailed information was obtained from 

the two site visits. While it is not possible to strictly compare the outcomes reported to 

the study with those reported as ARC targets because the time frames were different, 

i.e., some projects were not yet closed and some well beyond their closing date, the 

aggregate results are sufficiently close to validate the ARC data. 

 

10 Where appropriate, we report median results, which are more typical for most projects, rather than the 

average, which can be skewed by a small number of outliers.
Evaluation of the ARC Entrepreneurship Initiative

36

1. Documented outcomes

The Entrepreneurship Initiative supported programs that produced numerous individual 

successes (people who have improved income and opportunity) but few projects were 

able to demonstrate significant impacts on their local economies. About one-half of the 

surveyed projects reported that they had assisted business start-ups by adults, while 

one-half reported business start-ups by youth. Similarly, just over half (52 percent) of 

the surveyed projects reported that they had helped to create jobs in existing 

businesses and 39 percent reported saving jobs that would have otherwise been lost. 

Adults created 214 new firms—33 new with employees11 and 181 without employees. 

Based on projects that were able to report hard numbers in the survey, 356 new jobs 

were created—54 in new firms, 121 in existing firms, 181 jobs through self-employment. 

Another 85 jobs were saved from extinction in existing firms. In addition, surveyed 

projects reported 46 new businesses created by youth or students as class projects, 

some of which could become self-sustaining businesses after graduation.

Although such activities helped many individuals, the relatively small scale of the 

projects and the nature of the businesses assisted limit broader impacts. Most of the 

funded programs supported very small enterprises and start-ups, predominantly in nontraded sectors12 and serving local markets, or youth business projects. Those programs 

that produced enterprises with the greatest capacity to expand the economy are highly 

leveraged operations, with ARC funds an important—but not sole—component. 

These results are not unusual or unexpected. Other studies have found that, while 

micro-enterprises have economic development potential, the difficulties of selfemployment limit their impacts.13 Moreover, it is apparent that an increasing proportion 

of services are becoming part of the traded sector. This is particularly true of 

information and knowledge based firms. Service firms generally require a smaller initial 

capital investment than manufacturing industries, which traditionally have dominated the 

traded sectors. Service firms also have the potential to export and create a local 

multiplier that increases the impact of the revenue they directly bring into the local 

economy. However, the evaluation team found that most of the service businesses 

assisted by surveyed projects were locally serving. 

Furthermore, only a subset of assisted businesses had an orientation towards growth. 

Of the clients with businesses interviewed: 

• 36 percent planned to “grow” larger,

• 55 percent wanted a stable “lifestyle” business, and 

• 4 percent were attempting to reverse a decline. 

 

11 Not all projects reporting the creation of firms with employees specified the actual number of 

employees. 

12 Non-traded industries produce goods and services that are consumed locally. Since these sectors 

serve the local economy, they do not bring in “new money,” rather they re-circulate existing dollars. Thus, 

there is no net addition to local income.

13 Schreiner, Mark. Lessons for Microenterprise Programs from the Unemployment Insurance SelfEmployment Demonstration. St. Louis, Mo.: Washington University in St. Louis Center for Social 

Development, 1999. 
Evaluation of the ARC Entrepreneurship Initiative

37

While most businesses started by high school students show some income, they are not 

true sustainable enterprises. Instead, they are more akin to the “supervised 

occupational experience” in secondary agricultural education, in which students are 

expected to carry out business activities that demonstrate profit and measure their 

productivity. For example, the 29 “businesses started” through ACENet’s project are 

still school based projects and not independent and sustainable enterprises and, in fact, 

many of the student entrepreneurs will very likely go on to higher education after 

graduation. The ultimate success of the project will be determined by the number of 

students that return to the region to start businesses. 

Many of the projects that reported specific activities or outcomes were unable to 

quantify or estimate the numbers to support them. Those that offered education were 

usually able to provide enrollment data, but quantitative measures of technical 

assistance were less frequently kept. For example, 12 projects stated that they 

provided assistance to businesses with employees but only nine were able to estimate 

even the number of firms served; 12 projects provided assistance to businesses without 

employees, but only eight were able to estimate the number of companies. North 

Georgia Technical College, for example, reported new business start-ups but due to 

staff turnover and poor or non-existent records could not document them.

Although the value of the intervention is not related only to time together, one might 

suspect that learning is related to the time-based intensity of the experience. The data 

show the following.

• Time spent on the education of youth—an in-school and thus captive audience—

was greater than time educating adults. Slightly more than 95 percent of youth 

received more than 30 hours of education, while only 17.9 percent of adults received 

more than 30 hours. More than half the adults had less than eight hours of learning. 

• Less time was spent assisting adult entrepreneurs or existing businesses with

employees than in assisting entrepreneurs without employees. Almost 39 percent of 

the businesses with employees and 31 percent of adults received less than eight 

hours of help, but only eight percent of businesses with no employees received less 

than eight hours. It is possible that the assistance to the micro-enterprises is more 

critical to survival and thus they request more support.

2. Views from Clients of Programs

Information about the people served by the various projects comes from telephone 

interviews and project surveys. The client firms interviewed varied significantly in terms 

of the goods and services they provide. Table 7 shows the variety of firm types 

included in this analysis. They range from high-tech manufacturing and computer 

services to home-based day care. 

More than half (55 percent) of the business owners started their firms for stability and to 

support their current lifestyle while only 36 percent were interested in growing. Most 

spent a great deal of time with the program—which may be why their names were 

submitted to RTS by program staff. Eight percent of clients reported having greater 

than 20 interactions with program staff.
Evaluation of the ARC Entrepreneurship Initiative

38

The highest percentage of firms interviewed—50 percent—received assistance with 

gaining access to capital followed by 47 percent that received managerial and technical 

assistance. Forty-two percent reported having received entrepreneurial training.

Table 7

Client Survey Respondents by Type of Firm

Manufacturing Business Services Personal Services Retail/Wholesale

Analog switches Gene cloning Child day care (2) Bed and breakfast

Metal fabrication Computer services 

and repair (3)

Hair salon (2) Building products

Industrial equipment E-commerce, web 

design

Embroidery Fly fishing shop

Electronics Glass installation 

and repair

Natural personal 

care products

Fine grain, high 

purity ceramics

Carpet installation Restaurant (2)

Hand-painted 

ceramics

Insulation services Book store

Furniture parts Door repair Vitamins and herbs

Nanostructure 

optical devices

Medical case mgmt.

Source: RTS survey of ARC Project clients

Additional information about the people served by the projects can be gleaned from the 

“successes” cited in the survey responses. Each project was asked to provide 

examples of particularly successful results, and all but a few complied. Twenty-eight 

businesses were identified as potential clients for the survey. Based on the somewhat 

sketchy information about successful entrepreneurs or companies contained in the 

content of the brief description, we know that at least:

• 18 percent (5) were disadvantaged entrepreneurs and 18 percent (5) were youth

• 32 percent (9) were new enterprises, and a third of those (3) were value added 

firms and the rest local services

• 21 percent (6) were expansions of existing small firms through new markets or 

products, 11 percent (3) were youth operating small but income generating 

“projects,” and 11 percent (3) resulted in better jobs in existing firms 

• 21 percent (6) received loans and seven percent (2) received grants14

This small sample supports the findings of other entrepreneurial support programs; that 

is, most new businesses are local service providers. For example, the Economic 

Ventures Incorporated effort to help workers in Knoxville displaced by the closing of the 

Levi-Strauss plant reported that its clients started 102 new businesses. Almost all of 

those new businesses provide services to a local market. One in five provides child 

care, one in six produces crafts for sale, and one in eight provides sewing services. 

 

14 Percentages need not total 100 due to multiple responses.
Evaluation of the ARC Entrepreneurship Initiative

39

While a more diverse client base may provide a more diverse mix of new businesses, 

the Levi Strauss experience illustrates the preponderance of “non-traded” products from 

new businesses resulting from entrepreneurial programs targeting disadvantaged 

individuals—in this case, low-skill females.

Some aspects of the program’s impact from the client’s perspective were not readily 

quantifiable. For example, a few clients stated that they had become more self-reliant. 

In the words of one client: “the program was great. Without it we wouldn’t have 

anything—we would still be on welfare.” There were other instances where starting a 

business enabled an individual to purchase needed medication and care for an elderly 

relatively at home. 

E. Program Outcomes: What Really Matters?

The grant itself clearly matters. Responses to the project survey indicate that ARC 

funding was essential to carrying out the initiative. Half of the projects reported they 

would not have gone forward at all without the ARC grant, and it was important to the 

timing of nearly all others. Only two grantees believe they could have secured other 

funding and moved their projects ahead without any delay. 

The size of the project may have some influence on outcomes. Larger ARC grants are 

associated with slightly higher outcomes, as measured by numbers of new businesses 

started, although the numbers are too small for statistical significance. 

Sustainability is crucial to the success of entrepreneurial initiatives. The majority (14) of 

the grants analyzed funded new programs, and all but one of the existing programs had 

previously received some form of government or foundation support. In this context, 

sustainability—or lack of it—becomes an important outcome. The ARC Entrepreneurial 

Initiative is more likely to have a lasting impact if the funded activity continues beyond 

the grant period. While some projects are short-term and designed to address an 

immediate issue, sustainability builds local capacity for entrepreneurial support.

Diversification of the funding base supporting these projects is the key to sustainability. 

The 18 programs that continued beyond the end of their ARC grant did so by drawing 

upon multiple funding sources. The largest number of respondents—almost threefourths—reported that they received additional funding from state agencies. The large 

state role in follow-up funding suggests that states may be using ARC grants as start-up 

funds or to pilot test promising ideas. This was the case with one of the programs to 

visited—the Entrepreneurial Education Program at North Georgia Technical College. 

Non-ARC federal support was a distant second as a source of continuation funding. 

Eight of the 18 continuing programs received federal funds from other agencies. This 

group included the other project site visited, Knoxville (TN) Economic Ventures 

Incorporated (EVI). It has partnered with an Empowerment Zone to support a 

continuing and expanding program. EVI also received local government funding before 

and during the ARC grant and continues to be funded by the city. ACENet, with the 

help of ARC funds, was able to secure a direct congressional appropriation about 20 
Evaluation of the ARC Entrepreneurship Initiative

40

times the size of its ARC grant. This will allow the project to complete its complex and 

high cost entrepreneurial curriculum development process. 

Fees for services are negligible sources of revenue. Only seven of the continuing 

programs included fee income from clients among their sources of continuation funding. 

As noted previously, client fees provided less that one percent of program budgets 

during the ARC grant period. The small role played by client funding is reinforced 

further by the client surveys. Clients who reported paying for services were 

predominantly students who paid school tuition or to enroll in a workshop, and only a 

few others paid very nominal fees for services. This finding is consistent with nearly all 

government programs that provide technical assistance or offer training to very small 

firms and entrepreneurs, particularly in poor regions. Very few entrepreneurial support 

programs become self-sustaining. Therefore, an issue for ARC to consider as it initiates 

new programs is how they will be sustained and what entity might step up to assume 

responsibility for supporting it after ARC funding ends. 

The experiences gained from ARC-funded projects expanded the capabilities and 

capacities of grantee organizations. Over half of the organizations reported a greater 

willingness to seek outside funding for project support, and almost half reported a 

greater willingness to invest their own resources in entrepreneurial assistance projects. 

About half reported significant increases in entrepreneurial education training 

opportunities (52 percent) and businesses assistance services (48 percent) available in 

their community. Table 8 shows the broad range of positive impacts.

Transitional versus distressed location only modestly influences outcomes. An analysis 

that tested for differences between projects based on their ARC area designation 

(distressed versus transitional county status) revealed only a few modest differences.15 

More than expected, sampled distressed area projects built on earlier funded projects, 

while sampled projects in transitional areas tended to be new. Sampled projects in 

distressed areas were more likely to have been started in 1998 or later. The typical 

(median) project duration was shorter in distressed areas than in transitional areas. 

Sampled projects in distressed areas typically had a larger peak staff than those in 

transitional areas. Comparing the median values for sampled projects reporting 

detailed budgets, projects in distressed areas tended to have larger total budgets than 

those in transitional areas, although the typical ARC contribution was the same in both 

types or areas. Sampled projects in distressed areas were less likely than expected to 

undertake curriculum development and loan or equity funding. Finally, projects in 

distressed areas were as likely as those in transitional areas to report new businesses 

started by adults and youth, but less likely to report jobs created or saved in existing 

firms. 

Projects have useful community impacts. About one-half of projects reported improved 

entrepreneurship education and training opportunities and business assistance services 

in the community after the ARC project ended (Table 8). Seventeen percent of the 

projects report an increase in the availability of funding for new business start-ups even 

 

15 Care in interpretation is needed due to small sample sizes.
Evaluation of the ARC Entrepreneurship Initiative

41

though only a small number of projects give capital access a priority among their 

objectives. 

Table 8

Project Impacts

Percent of projects 

reporting

For the following factors, please compare the position before the start 

of the ARC project with the position after the end of the project. 

Significant 

increase

No 

change

Host organization’s willingness to seek outside state and federal 

funding for entrepreneurial assistance projects after the completion of 

the ARC project

56.5 4.3

Entrepreneurship education and training opportunities available in the 

community after the completion of the ARC project 

52.2 8.7

Business assistance services available in the community after the 

completion of the ARC project 

47.8 21.7

Host organization’s willingness to invest its own resources in 

entrepreneurial assistance projects after the completion of the ARC 

project

47.8 8.7

Host organization’s willingness to seek local community funding for 

entrepreneurial assistance projects after the completion of the ARC 

project

43.5 8.7

Your knowledge of the capabilities of other organizations within the 

community who provide entrepreneurial training or business 

assistance

39.1 8.7

Your knowledge of the capabilities of other organizations outside your 

local community who provide entrepreneurial training or business 

assistance

34.8 91.3

Amount of funding available for new business start-ups after the 

completion of the ARC project

17.4 43.5

Number of business network groups in operation in the community 

after the completion of the ARC project

8.7 47.8

Source: RTS Survey of ARC Entrepreneurship Projects (N=23)

F. Obstacles and problems

The inability to attract clients for business assistance services was one of the most 

frequently noted problems among grantees. While only one project cited difficulty in 

attracting existing client firms as a major problem, nine others indicated that it was a 

minor problem. This may reflect lack of sufficient attention to, or inadequate analysis of, 

market demand in the design and proposal stage. Proposals for projects that deliver 

services must include some evidence a reasonable base of demand. Otherwise the 

project may be a solution in search of a problem. The scale of the problem cited, 

however, suggests that either the initial projections are optimistic or marketing efforts 

are inadequate. Most clients interviewed reported that they learned of the program 

informally through person-to-person rather than any media-based channels. Of the 

clients interviewed, half learned about the program through referrals, 17 percent by 

word of mouth, and 27 percent through advertisements. 
Evaluation of the ARC Entrepreneurship Initiative

42

The other most commonly named problem was monitoring and evaluation. Five 

grantees named this a major problem and five a minor problem. The difficulty that the 

evaluation project staff experienced in obtaining data about project outcomes supports 

the perception that this is a major problem. As noted previously, rigorous internal 

assessments were rare. 

The problems with monitoring and assessment have broad implications. Overall, the 

program does not appear to be taking advantage of the fact that tracking performance is 

more than a means for funding agencies and others to judge the success of a project or 

program. Tracking performance is also a management tool to provide feedback to 

managers so that they can make adjustments to correct problem areas and work for 

continuous improvement. While there were many problems with monitoring and 

evaluation, there was at least one notable exception. Economic Ventures Inc. (EVI), 

one of two sites visited, has harnessed the power of performance management to build 

program quality. The EVI intake process gathers relevant information about clients’ 

situations, compiles it into a database and updates it annually. As a result, EVI knows 

which clients are improving or not improving their businesses. Analysis of these data 

combined with survey results of client satisfaction guides program modifications. The 

result is a successful program that is responsive to clients’ needs. Not coincidentally, it 

also is market oriented and offers services for which there is demonstrated demand. 

Difficulties with monitoring and assessment at the individual project level are pervasive 

enough to suggest that it is systemic. For ARC and other federal agencies, the 

Government Performance Reporting Act (GPRA) drives the outcome evaluation 

process, and to respond to GPRA reporting requirements ARC must gather and compile 

relevant performance information from its funded projects. This system is both new and 

complex, and thus must be viewed as a work in progress. 

The current organization of projects within the Entrepreneurship Initiative does not 

support a comprehensive performance measurement system. Research into evaluation 

of other projects with similar goals produced a range of metrics and methodologies, 

described in the methodology section. The lowest level of metrics focuses on activity 

counts, and feeds into intermediate outcomes, which assess the success of different 

strategies and finally into outcome measures, which indicate whether or not the 

programs are achieving their goals. It was difficult to align any but the lowest level 

metrics, the activity counts, with the current system of organizing ARC projects, which 

focuses on functions such as technical assistance, and capital access rather than 

outcomes.

The ability to attract capable staff (two respondents named it a major problem and 

three, a minor problem) and develop or get access to technical materials also were 

problems cited in multiple sites (one named it a major problem and six, minor). Some 

projects noted problems finding and keeping staff. One responded to a survey with the 

comment that “finding quality staff often takes time in rural communities.” Both 

evaluation and attracting and retaining staff were problems in the North Georgia 

Technical College project. The two problems exacerbated each other as staff turnover 

erased any institutional memory that might have countered the lack of written 

information about clients at the business assistance center.
Evaluation of the ARC Entrepreneurship Initiative

43

IV. Recommendations

The Entrepreneurship Initiative to date represents a modest but potentially significant 

success. It is unlikely that the program alone will transform localities, but it clearly 

provides useful services and has tangible benefits for many parts of the Appalachian 

region. It is premature to detect any significant regional economic impacts. Most of the 

projects were short term and, when the evaluation was conducted, had been in 

operation for an average of only 18 to 21 months. However, these early findings 

provide an indication of preliminary program performance and outcomes as well as 

meaningful insights that can aid in improving implementation of ARC’s Entrepreneurship 

Initiative.

It is clear that the ARC grant funds are needed to support the local enterprise 

development efforts in the grantees’ communities. Responses to the project survey 

indicate that ARC funding was essential to half of the projects that reported they would 

not have gone forward without it and important to the timing of most others. 

A synthesis of the findings from this evaluation reveals a few core themes with regard to 

the implementation and performance of the Entrepreneurship Initiative. There is a need 

to make the projects more sustainable at the local level. An entrepreneurial 

development program must be at a sufficient scale in terms of dollars and time invested 

in order to have lasting impacts. The local entrepreneurial initiatives are more likely to 

bring about the desired changes in the regional economy if they continue beyond the 

ARC grant period. While some projects are short-term and designed to address an 

immediate problem, sustainability builds capacity at the local level for ongoing 

entrepreneurial support.

Another major theme is the need for a better performance measurement system in 

order to assess more consistently the outcomes of the Entrepreneurship Initiative. This 

will be critical as the program continues over time and its actual affects on the region’s 

economy become more readily measurable. 

Finally, with regard to the underlying rationale of the Initiative, it is apparent that 

entrepreneurship is conceived of in the broadest sense. The funded projects cover a 

wide range of activities including everything from education and training of youth to 

technical assistance for existing firms. The types of businesses served include some 

with high-growth potential, but many small enterprises or sole proprietors in business for 

lifestyle reasons. Such an approach can be justified but it must be realized that the 

outcomes and impacts will vary depending on the nature of the activity and type of client 

served. This has very real implications for program design, implementation, and 

performance measurement. With these core themes in mind, drawn from the evaluation 

findings, the research team recommends the following. 

1. Move the entrepreneurship initiative to scale

The current ARC Entrepreneurship Initiative has established useful project models and 

assisted individual entrepreneurs and businesses. It would be worthwhile to continue 

the effort at the current scale. Reducing or eliminating a program that has begun to 
Evaluation of the ARC Entrepreneurship Initiative

44

achieve some degree of success and demonstrate potential, and which depends on 

patience and long-term support to realize its value, would send the wrong message to 

communities that may have few other options. However, to have a substantial impact 

on the region’s economy, the program needs to be expanded to a more significant 

scale. Any expanded investment needs to be made with a multi-year time horizon 

(since these efforts require sustained commitment) and with consideration to the 

changes and improvements suggested in our subsequent recommendations.

2. Provide more technical assistance

Funded programs must be able to develop greater local capacity if they are to be 

sustainable. ARC’s primary role is as a wholesaler of services, but for the system to 

work ARC should help build greater capacity among the retailers who deliver the 

services. Many organizations that were awarded grants would benefit greatly from 

additional external support and assistance to maximize their chances for success. 

Small organizations in rural areas—especially those attempting new and innovative 

activities—have few places to turn for advice and counsel. The two sites visited 

emphasized that need. Assistance is particularly important when a key member of an 

already small project team is replaced. Three ways to introduce technical assistance 

and supplement local resources are as follows. 

• Build a technical assistance budget into all grants and have either a cohort of 

consultants or lead (existing) agency/institutions for all new projects. ARC has 

provided some of this, and in fact half of the grantees found it very helpful. But ARC 

is not in a position to provide on-going advice and support from its staff for all of its 

grantees. 

• Organize grantees with similar or complementary goals into learning networks during 

the process, with facilitators, so that they can share experiences and learn from 

each other. The networks would meet several times a year and have their own web 

sites for on-going dialogue. 

• Conduct more training for project directors and their staff, as the National Science 

Foundation (NSF) does in its Advanced Technological Education (ATE) Program. 

NSF requires that all of its ATE program principal investigators assemble once a 

year for budget and management workshops, a project fair, and presentations.

3. Take a market-oriented approach

The limited resources available and nature of the ARC Entrepreneurship Initiative—

which is an intervention to address market failures—suggest that its operation should be 

as market oriented as possible. One of the most common problems reported in the 

surveys was difficulty in attracting clients to use project services. Currently, proposals 

for funding are required to present information demonstrating a general extant demand 

for the proposed service. However, a more detailed analysis of the target population 

and the willingness of potential clients to use services as well as identification of 

potential strategies to generate demand will lead to greater utilization. If no such 

information is available, ARC could provide a small grant to finance the needed market 
Evaluation of the ARC Entrepreneurship Initiative

45

research and commit to a larger project grant only if and when the market research 

reveals such a demand. The availability of a small research grant is important to 

keeping this requirement from stifling innovative solutions. The market research also 

provides an opportunity for gathering information about potential clients’ preferences for 

mode of service delivery; i.e. on-line, in class, in a one-on-one consultation. 

Similarly, if implementation of a project or program requires participation by other 

entities than the grant applicant, the proposal should include letters from 

representatives of those organizations documenting interest and support. A second 

aspect of market-oriented operation is the increased use of performance information, 

addressed in the next recommendation. 

4. Structure and support evaluation systems

The performance measurement process for the Entrepreneurship Initiative should offer 

ARC an opportunity not only to respond to GRPA responsibilities but also to help 

grantees improve project management. For ARC to guide its grantees in this process, 

the agency must have its own set of activity and outcomes measures that is aligned in a 

meaningful system. Thus, the first step is for ARC to assess and revise its performance 

measures to effectively serve as the organizing context for funded projects’ 

performance measurement. 

Grantees should be given standard guidelines, technical assistance and the support to 

conduct valid evaluations of outcomes that are related to their proposed goals. Nine of 

the projects noted monitoring progress and outcomes as a problem, and five cited it as 

a major problem. This is not surprising since the outcomes specified are difficult and 

expensive to ascertain because they require gathering information at some interval after 

the intervention and, in some instances, are not directly related to goals or activities. 

Some of the benchmark evaluations reported this as a finding.16 The outcomes that the 

ARC suggests do indeed reflect desired impacts, however, it is very difficult to 

accurately capture the incremental impact of a small-scale intervention (perhaps only a 

few hours per contact) on an outcome. 

Few project staffs have the necessary resources or competencies to follow the progress 

of their clients or students after they end their formal connection to the project and 

adequately measure outcomes. ARC could encourage more effective program 

management and reporting by allocating resources to cover the additional costs of 

implementing a performance measurement system that may be based on sampling or 

other evidence about outcomes. The additional resources that are required vary with 

the type and scale of the evaluation, but it may be appropriate to allocate about three to 

five percent of grant funds to sustain monitoring and evaluation efforts. 

Grantees are likely to need more than funding to implement performance measurement 

effectively. ARC should provide more specific guidelines and require that proposals 

specify data to be collected and used to measure outcomes. Requiring performance 

 

16 David Barkley, Deborah Markley, and Julia Sass Rubin. Public Involvement in Venture Capital Funds: 

Lessons from Three Program Alternatives. Columbia, Mo.: Rural Policy Research Institute, November, 

1999. <www.rupri.org.pubs/archive/reports/p99-9/index.html>. 
Evaluation of the ARC Entrepreneurship Initiative

46

measures at the start can avoid subsequent assessment problems, and project 

managers can use the performance data to guide mid-course adjustments where results 

are not meeting expectations. In addition to better ongoing program measurement and 

tracking, the ARC should also sponsor independent evaluations and technical 

assistance from outside organizations. 

Another issue is aggregating projects with very different goals. For example, the 

Initiative treats education as a single category but education of adults and of youth are 

quite different, have different delivery systems, and require special outcome measures. 

The former ought to show hard outcomes sought, the latter produces soft outcomes, 

such as entrepreneurial aspirations, which are difficult to convert to jobs or wealth in the 

short term. The research undertaken in this project provides a starting point for the 

design of improved measures. Once accomplished, ARC will be better positioned to 

provide needed support to its grantees in this arena. 

5. Build regional technical assistance capabilities

Resource and technical assistance centers for programs can help to solve many of the 

implementation problems and fortify staff expertise and experience. For example, 

seven projects noted lack of ability to develop or obtain technical materials as a major 

problem. Support capabilities could be targeted at some aspect of the market for 

entrepreneurial efforts (e.g., micro-enterprises, youth entrepreneurship, cooperatives, 

and technology or market development). Perhaps an industry cluster could be a focus 

where a critical mass of initiatives in a particular sector exists, e.g., wood products, 

apparel, or information technologies. The National Science Foundation, for example, 

has created Centers that focus on clusters and technologies at many universities and 

community colleges. The capabilities would not necessarily have to be developed at 

new organizations (although they could be), but more likely would operate in existing 

stable, high quality organizations or institutions. Each would be funded to further 

develop its expertise, provide technical assistance, store knowledge and information, 

provide contacts and broker relationships, conduct training sessions, and perhaps be 

responsible for conducting evaluations. 

Organizations in the ARC region with high levels of expertise could be designated as 

mentor organizations to provide technical assistance to less experienced groups. 

Organizations such as REAL serve these purposes for certain kinds of projects that can 

be a repository of information and expertise, help with evaluations, etc. Those places 

that implemented REAL, for example, have access to a national support center and 

network that allows them to access information and get advice.

6. Organize projects among dimensions other than function 

The functions that are currently used overlap in most projects—as well they should, 

because small businesses’ needs are invariably interrelated—and few programs are 

“pure” in the sense that they have a single purpose. Therefore the existing taxonomy or 

grants is less useful than it might be. Perhaps segregating programs according to those 

that (a) build capabilities in organizations, (b) aim to build capacities or change cultures 
Evaluation of the ARC Entrepreneurship Initiative

47

in communities, and (c) support program operations is a more useful classification 

scheme for understanding outcomes and successes. 

7. Make fewer small and short-term grants

Many of the grants are too small and for too short a period of time to produce a 

significant economic impact. Making perhaps fewer of the longer and larger grants now 

being awarded would give grantees sufficient resources and time to recruit and retain 

good staff, build greater internal capacity, and buy needed technical assistance. A 

longer grant period would also allow more time to overcome unforeseen obstacles and 

perhaps to produce outcomes at a scale that can make a difference in a local economy. 

The Entrepreneurial Program at North Georgia Technical college, for example, suffered 

from high staff turnover and an inability to offer more than temporary employment 

limited by the term of the grant. One project manager noted that “Funding 

entrepreneurship programs for one year is unrealistic...The curriculum has taken 

several years to complete and is still being improved.” Reducing uncertainty about year 

two funding would help address the issue of staff instability that impedes local capacity 

building. A longer grant period would give new programs time to get up to speed. 

8. Seek opportunities to highly leverage grants

Leveraging will result in greater scale of impacts of ARC funds and increase chances for 

success. However, the more ARC funds are leveraged, the more fungible the monies 

become, and the more difficult it is to distinguish their effects from those of 

complementary or supplementary funds. Examples are Ohio’s ACENet, New York’s 

Ceramics Corridor project and the West Virginia’s Revolving Micro-Loan and Rural 

Outreach Program, where it is very difficult, if not impossible, to separate the outcomes 

associated with the ARC dollars from those associated with other dollars. Higher 

leveraging may mean accepting contributory rather than exclusive outcomes. 

9. Set realistic goals for sustainability

Government funding agencies often view grants that support proprietary initiatives and 

private businesses as temporary and assume that once the value of the service is 

realized and accepted, the firms or individuals ought to pay market price and the service 

should ultimately be self-sufficient. Experience of the projects and throughout the world 

has proven that self-sufficiency is rarely achieved in any program that either has 

catalyzing change as its goal or that operates in a weak economy and targets low 

income and marginal businesses. Only seven of the projects recorded any fee income. 

Among the clients interviewed a sizable number paid a nominal fee, but much less than 

the true cost of the program. Instead, the expectation of sustainability after a relatively 

short period of funding turns grantees’ attention to raising funds to meet continuation 

expenses. Short grant periods also make it more difficult to attract good project staff. 

To the credit of the grantees, most have been able to acquire funding to continue, but 

mostly from other government agencies, as described above. 
Evaluation of the ARC Entrepreneurship Initiative

48

10. Acknowledge the risks inherent in making entrepreneurial investments in 

distressed regions

Most of the projects are operating with marginal budgets and under difficult conditions, 

i.e., targeting low income and low education populations and depressed areas. Finding 

innovative and successful approaches under these circumstances requires patience 

and acceptance of the potentially higher risk of failure involved. Projects that fail to 

meet their goals can provide valuable lessons for more successful efforts in the future. 

The population served also may demand greater risk taking. There is tension between 

goals to help disadvantaged individuals support themselves through entrepreneurship 

and an emphasis on new business creation to add jobs. The former goal speaks 

generally to low-skill and less-educated individuals whose businesses are “non-traded” 

service providers and likely will never have large payrolls. The latter shifts the focus to 

manufacturing and high technology industries that may ultimately employ large numbers 

of workers and that usually sell their product outside the local economic region. Desire 

for quick and large results that show job growth in traded sectors directs activity to 

larger employers, which may not be an appropriate target for an entrepreneurship 

program. Clearly, entrepreneurship support does not comprise an economic 

development program. Rather it is one strategy in what should be a broader effort. 

11. Replicate successful initiatives

Now that the ARC region has a history of innovations and a good sense of what has 

proven successful, it may be fruitful to begin to replicate good practices. Many of the 

surveyed projects have proven their value and are replicable models—and provide 

considerable opportunities for adaptation to other circumstances that may lead to further 

improvements and innovations and new lessons. 

The Commission could build on what works or shows promise at other sites in addition 

to additional phases or program expansions at the same places. Some of the grants to 

introduce REAL Enterprises are a good example of this occurring in some places. 

Dissemination of information about best practices or good examples should not 

discourage innovation but rather encourage innovative improvements or adopting 

practices to different circumstances. This could be done through mentoring grants in 

which both the adopter and model program were given funds. The U.S. Department of 

Education’s Fund for the Improvement of Post-Secondary Education (FIPSE) has such 

a program in place. 

12. Conduct a follow-up, later stage evaluation

Given the limitations of early stage evaluations, we urge the ARC to conduct another 

evaluation of mature open (i.e., in late stages of funding) and closed projects in order to 

learn more about findings in this report that were tentative due to the timing of the 

evaluation and to expand the data base and improve the reliability of the findings. 
Evaluation of the ARC Entrepreneurship Initiative

49

Appendix A

Entrepreneurship Project Profiles (Evaluation Sample)

AL-12931: ASBDC Entrepreneurial Education and Training Program

The University of Alabama’s Small Business Development Consortium (ASBDC), which 

includes universities in eleven states, created a new entrepreneurial training program to target 

both minority and women small business owners in 37 counties throughout the state. 

ARC $ Total $ Organizational 

Budget

FTE

Staff

Length in 

Months

Months since 

Completion

$43,000 $76,760 $1-10 Million 2.5 21 +6

Project goals are quite general: to build institutional capacity for entrepreneurial training and 

development and support initiatives to assist entrepreneurs. 

ARC Metrics Target Actual

(ARC Final Report)

Actual

(RTS Survey)

# of participants 130 Grant not 

completed

193

# of jobs created∗ Grant not completed 13 Grant not 

completed

Measurable Achievements 

• New businesses started, sales increased, jobs saved and created; however, no relevant 

numbers are currently available, and no exceptional success stories were mentioned.

• As result of this program, more entrepreneurial education is available in the community.

Clients’ Reactions: The first client, a small businessman (door repair) wanted to shift into cattle 

management. He took fifteen weeks of classes with nine others (paid $75). He is very 

enthusiastic about the value of the program – he reports that he learned how to prepare a 

business plan, do cash flow analysis, manage risk, giving him increased confidence in his 

entrepreneurial abilities. As a result of going through the program, he has decided to delay his 

business startup. 

The second client received technical assistance and technology transfer. Through the program, 

he learned how to prepare a business plan, and is working on building and licensing casting and 

extruding equipment. He plans to open his company soon. He has customers and four patents 

already, and attributes his accelerated progress to his participation in the program.

Since this program is relatively small in scale and part of a very large SBA-funded program with 

education and training part of its mission, it is difficult to pinpoint the value added by ARC funds.

 

∗

 Job creation numbers do not include jobs generated through self-employment. Self-employment is 

captured in the business creation data.
Evaluation of the ARC Entrepreneurship Initiative

50

GA-12788: Entrepreneurial Education Program

This project funded the creation of an entrepreneurial training certification program at North 

Georgia Technical College, which serves six counties in Northeast Georgia. The project 

operated with ARC support for two years and is ongoing, using state funds. 

ARC $ Total $ Organizational 

Budget

FTE

Staff

Length in 

Months

Months since 

Completion

$96,019 $132,719 Unspecified 1.5 28 +8

Project Goals are to (1) develop a curriculum for an entrepreneurial education certificate 

program, (2) deliver entrepreneurial education courses to about 50 students, (3) establish a 

small business resource center on campus, (4) offer courses via Internet, and (5) provide 

entrepreneurial education to high school students. The first two goals were for year one, 

increasing the number of students served in year two. The third and fourth goals were added 

for year two. The fifth goal of serving high school students was for both years.

ARC Metrics Target Actual

(ARC Final Report)

Actual

(RTS Survey)

# of participants 158 148 150

# of new businesses created 7 10 Not reported

Measurable Achievements 

• A three-course curriculum was developed and offered on campus and via the Internet

• A total of about 150 individuals took at least one course

• Courses are also being offered statewide through the on-line Georgia Virtual University

Notes: The project proposal included follow-up and assessment, but the only evaluation done 

was an end-of-course survey, and those surveys were not saved nor was a summary of the 

information preserved. Project reports stated that clients and students started new businesses. 

However, the current manager was not able to document this due to changes in project staff 

and failure by previous staff to maintain records of clients assisted at the entrepreneurial 

assistance center. Thus it is difficult to specify measurable achievements. The impression 

gained by the on-site researcher is that there are positive outcomes, but no documentation to 

substantiate them. 

Clients’ Reactions: Project staff provided the name of one student and no business center 

clients. That student, who took the course via Internet, reported satisfaction with the 

entrepreneurial training he received. In June, he leased a server and started a computer repair 

business with sales between $1,000 – $2,000 per month. He plans to begin selling web sites as 

well.
Evaluation of the ARC Entrepreneurship Initiative

51

KY-12110: Women’s Entrepreneurial Training Network & Mentoring Program (WINGS) 

This project serves two of the nation’s poorest Congressional districts, Kentucky’s Fifth and 

Sixth. Both have high rates of poverty and few job opportunities. WINGS, a non-profit 

economic development organization that has received ARC funding since 1995, targets poor, 

Appalachian women through mobile training units. 

ARC $ Total $ Organizational 

Budget

FTE

Staff

Length in 

Months

Months since 

Completion

$50,000 $180,000 $50-250,000 3 10 +11

Project Goals are to (1) improve business practices, (2) create networking opportunities, (3) 

support leadership development, and (4) share lessons. One original goal, youth outreach, was 

deferred for a year. These goals varied slightly from proposed goals, which highlighted new 

women-owned startups and increased sales. 

ARC Metrics Target Actual

(ARC Final Report)

Actual

(RTS Survey)

# of participants 90 105 241

# of new businesses created 25 6 6

Measurable Achievements 

• 6 new single employee businesses formed (catering, Montessori school, greenhouse, 

bookstore, B&B, and nursery) 

• 8 businesses upgraded

• 5 developed new products

• 12 companies increased sales as a result of assistance

Notes: The project reached its goals of providing entrepreneurial training, including to the first 

class of African Americans and to women starting new businesses. Among its 

accomplishments are the start of the area’s first Internet-based business and a scholarship fund 

that allows entrepreneurs to attend trade shows. The latter resulted in two members identifying 

and marketing new products. The program’s internal assessment was confined to student 

evaluations.

Clients’ Reactions: One female entrepreneur, on disability, operates an embroidery business 

with two part time employees. As a result of the program, she has invested $30,000 in her 

business, expanded into new products and markets, and increased sales from $2,000 in the 

past year to $15,000 in the current year. She was satisfied with the program with the training 

and networking opportunities she received, though disappointed that she did not receive a loan. 

She pointed out lack of capital access as a weakness of the program.

The other client co-owns a health food store, and was very satisfied with the business training 

she received. As a result of the program, she started the business with a $50,000 investment, 

and is now generating sales of $50,000 per year. She affirmed the value of an entrepreneurship 

program aimed at women. 
Evaluation of the ARC Entrepreneurship Initiative

52

CO-13027 (KY): Kentucky Wood Products Training Program 

This project provided training for new employees of Kentucky Wood Manufacturing, Inc in 

Harlan County, which is in the highly distressed southern Kentucky coal region. The county had 

13 percent unemployment and almost one-third of residents with incomes below the poverty 

line. The Kentucky Wood Products Competitiveness Corporation is a non-profit, quasigovernmental organization established by legislation to promote the wood products industry and 

economic development in Kentucky.

ARC $ Total $ Organizational 

Budget

FTE

Staff

Length in 

Months

Months since 

Completion

$70,000 $630,650 $250,000-$999,000 12 12 +4

Project Goals are to: (1) train 54 new employees and provide marketing support for a new 

company-—Kentucky Wood Manufacturing, Inc.; (2) improve the competitiveness of the 

upholstered furniture industry in the Harlan County region through technology transfer, 

entrepreneurial development, and job creation; and (3) establish a program resulting in 100 

additional people employed at a living wage within two years and at least one new start-up 

subsidiary company.

ARC Metrics Target Actual 

(ARC Final Report)

Actual 

(RTS Survey)

# of participants 54 22 28

# of jobs created∗ 54 16 16

# of new businesses created 1 1 1

Measurable Achievements 

• 24 adults and one youth received training

• One new business with employees 

• One existing business has developed new products and upgraded technologies

• Four or five business have increased out-of-state sales

• Leveraged project funds with Welfare-to-Work dollars, giving 20 people new careers

Note: While exploring alternative sales outlets for products, participants offered a solution to a 

chair company, using their product, which eliminated processes and resulted in savings. Project 

efforts to change framestock processes are being adopted by industry and will result in savings 

and higher efficiency. The grantee measures its success in terms of improved productivity of 

assisted firms.

Clients’ Reactions: The two owners were very satisfied with the production and management 

training they received, but did not provide details about their businesses.

 

∗

 Job creation numbers do not include jobs generated through self-employment. Self-employment is 

captured in the business creation data.
Evaluation of the ARC Entrepreneurship Initiative

53

KY-13280: Forward in the Fifth Student Entrepreneurship Center

This project provides services to schools and communities in 31 Kentucky counties to help 

prepare students for successful careers. Through the development of student-run lending and 

credit union groups, it teaches youth entrepreneurship and workforce skills that will enhance 

their employability after graduation.

ARC $ Total $ Organizational 

Budget (s)

Project 

FTE Staff

Length in 

Months

Months since 

Completion

$50,000 $300,000 $250K-$1M 3 12 +11

Project Goals are (1) enhanced skills training for participating youth, (2) professional 

development for teachers and affiliate organizers, (3) introduction of entrepreneurial culture into 

region’s middle and high schools, and (4) community involvement through mentoring, 

instruction, and fairs. Specific objectives are that (1) 450 students participate in the Student 

Leadership Initiative Program (SLIP) with 80% (360 students) continuing in SLIP; (2) 210 

students participating in Educational Designs Generating Excellence (EDGE) Program—108 

upon entering high school; and (3) 126 EDGE students starting a business, 105 entering the 

workforce after graduation, and 63 pursuing post-secondary education.

ARC Metrics Target Actual

(ARC Final Report)

Actual

(RTS Survey)

# of participants 660 250 750

# of new businesses created 126 12 20 (by youth)

Measurable Achievements 

• 170 eighth grade students from nine schools participated in SLIP

• 18 students attended a summer leadership program designed for rising ninth graders

• 3 schools participated in a high school entrepreneurship education program (EDGE).

• 80 students participated in EDGE

• 20 student businesses were formed, including businesses selling gifts, make-up, jewelry, 

and candles

Notes: ARC funding has ended, but the project is continuing, reflected in the increased 

outcomes reported.

Client/Partner Reactions: Because this program’s clients were under eighteen years old, no 

client surveys were conducted. However, two partner organizations were contacted for their 

perspectives on the project. Both of the partner organizations were educational institutions. 

Both expect to continue working with the program over the next year and indicated very positive 

impacts on their organization as a result of their involvement with the project. Both partner 

groups had very positive overall opinion of the project.
Evaluation of the ARC Entrepreneurship Initiative

54

MD-13005: Garrett County Micro Business Partnership

This project supports micro-businesses in Garret County and targets individuals with limited 

business and management experience. The grantee is a non-profit community action agency.

ARC $ Total $ Organizational 

Budget 

Project 

FTE Staff

Length in 

Months

Months since 

Completion

$50,000 $285,000 $1M - $10M 1 12 +20

Project Goal was to enhance the micro-business climate through education and with public 

relations aimed at more traditional community and economic development efforts. Specific 

objectives include creating a program that would (1) assist 10 entrepreneurs in the first year, 

and 30 per year thereafter, (2) provide 5 loans up to $15,000 each to entrepreneurs, (3) train 20 

participants in a 100 hour program, (4) provide at least 20 hours of individual counseling to 30 

participants, and (5) create 30 new jobs through start-up or expansion.

ARC Metrics Target Actual

(ARC Final Report)

Actual

(RTS Survey)

# of participants 30 45 40

# of new businesses created 5 8 10

# of jobs created∗ 30 14 3

# of jobs retained Not reported 4 16

Measurable Achievements 

• Created 8 new businesses and created or saved 14 jobs in existing businesses

• Added to sustainability of 19 other businesses

• 3 businesses have developed new products and 6 have upgraded technologies

• 45 participants received assistance in finding new markets

• 30 micro-business owners participated in training

Notes: Internal evaluations are conducted using end-of-course student evaluations for education 

and training, and using end of project evaluations (some well after project completion) for other 

activities. Program staff believes it is still too soon to assess outcome for many of the projects. 

ARC funding was a small part of the overall program, but enabled leveraging of other funds, 

making it difficult to separate out impacts. The project has continued beyond the ARC funding 

and expanded. The loan fund was re-capitalized with a $120,000 loan from a financial 

institution and USDA award.

 

∗

 Job creation numbers do not include jobs generated through self-employment. Self-employment is 

captured in the business creation data.
Evaluation of the ARC Entrepreneurship Initiative

55

Clients’ Reactions: Two owners used the program to upgrade their managerial and technical 

skills. One client, the owner of a fly fishing and guide services shop, was able to build up his 

inventory to nearly double the size of his business. His yearly sales have increased from 

$37,000 to $70,000.

The other client, who owns a stable and horse boarding service, relocated and renovated her 

business to improve its chances for success. As a result of the program, she made a $35,000 

investment in her business. Both clients were seeking a stable, lifestyle-supporting business 

rather than rapid growth, and both clients were satisfied (one very satisfied) with the services 

they received.
Evaluation of the ARC Entrepreneurship Initiative

56

MS-12908: Lee County Youth Entrepreneurial Program 

This project serves youth in Tupelo and Lee County Mississippi. The City of Tupelo was the 

grant recipient and formed a partnership with the Boys and Girls Club of Tupelo/Lee County, Big 

Brothers/Big Sisters of Lee County, and Junior Achievement to implement the project.

ARC $ Total $ Organizational 

Budget (s)

Project 

FTE Staff

Length in 

Months

Months since 

Completion

$70,577 $88,192 >$10 M 3 15 +13

Project Goals were to (1) provide entrepreneurial education and training to 100 at-risk youth 

aged 10 to 18 in Lee County; (2) assist youth entrepreneurs in starting businesses; (3) build 

local capacity for an entrepreneurial economy; and (4) reduce crime and drop-out rates and 

increase math and writing skills and build self-esteem among at-risk youth. The goals did not 

change during the project operation.

ARC Metrics Target Actual

(ARC Final Report)

Actual

(RTS Survey)

# of participants 100 70 120

# of new businesses created Not reported 16* 18 (by youth)

# of jobs created∗ 3 0 0

* Businesses reported to ARC were mostly temporary youth projects and not sustainable 

income-generating businesses, and thus they were not reported as jobs created. 

Measurable Achievements 

• 120 at-risk youth learned entrepreneurial skills; 48 completed a business plan

• Participants’ school grades and self-esteem improved; drop-out rates were lower than would 

have been expected

• 18 new single (and part-time) employee businesses were formed by youth participants 

(animal balloons, lawn services, baseball card selling, auto detailing, nail and beauty supply, 

concession sales)

Notes: An outcome evaluation conducted by Boys & Girls Club and Big Brother/Big Sisters 

found slightly more new businesses (21) than the survey form reported. It also noted that 

parents reported (a) that participating young people’s attitudes improved and (b) a significant 

difference in confidence levels and vocabulary. The project is continuing at an expanded level 

with a new partnership that includes the local SBDC but no longer includes the Big Brothers/Big 

Sisters program.

Client/Partner Reactions: Since all participants in this program were under eighteen years of 

age, no client surveys were conducted. Two partner organizations, however, were contacted for 

their perspectives on the project. One of the partner organizations was an educational 

institution and the other was a human services agency. Both expect to continue working with 

the program over the next year and indicated that their own organizations’ services had been 

improved by their involvement with the project.

 

∗

 Job creation numbers do not include jobs generated through self-employment. Self-employment is 

captured in the business creation data.
Evaluation of the ARC Entrepreneurship Initiative

57

MS 12909: MSU Extension Service Entrepreneurial Education Training Center

This project provided support to entrepreneurs and small businesses—including farmers and 

youth – in Calhoun, Chickasaw, Itawamba, Layayette, Lee, Monroe, Pontotoc, and Union 

counties. The state university delivers on-going entrepreneurial training that integrates 

computer application of technology-based business skills and tools. 

ARC $ Total $ Organizational 

Budget (s)

Project 

FTE Staff

Length in 

Months

Months since 

Completion

$100,000 $125,000 >$10 M 1.5 19 +11

Project Goals were to: (1) establish an Entrepreneurial Education Training Center for counties in 

the Three Rivers Planning and Development District; (2) develop educational materials and 

provide training; (3) develop and maintain and entrepreneurial service provider resource 

network; and (5) evaluate outcomes and possible expansion to other counties. Specific 

objectives for the Training Center were to train (1) 40 home-based/micro businesses in 

marketing and record keeping within the first year, (2) 40 youth in entrepreneurial start-up and 

job preparedness, and (3) 40 small farmers in farm business management.

ARC Metrics Target Actual

(ARC Final Report)

Actual

(RTS Survey)

# of participants 120 580 560

Measurable Achievements 

• Total of 580 people received services

• 41 people (11 of which were business owners) participated in the short course.

• 319 participated in an antiquing short course, 19 in Bed & Breakfast course, 53 in Clowning 

as a Business, 32 in Pottery as a Business, and 36 in Technology as a Tool

• 80 attended a financial investment fair

• New businesses were started by adult and youth participants but the number is not known.

• Businesses have developed new products and upgraded technologies, but number not 

known.

Notes: For achievements, this project survey reports activities but no quantified outcomes 

beyond one youth who got a job and one woman who bought and is using a computer in her 

business. Evaluations of responses to each event are included in the report to ARC, however, 

and indicate high satisfaction rates. The youth program appears to have been successful in 

giving students hands-on experience in business management. 

Clients’ Reactions: A teacher who owns a two-bedroom bed and breakfast participated in a 

videoconference for B&B owners about health, handicapped access, and other relevant issues. 

The most important outcome for her was networking—connections to other B&B owners and 

university resources—and an increase in customer referrals. 

Because the program’s other referred clients were under eighteen years old, only one client 

survey was conducted for this program.
Evaluation of the ARC Entrepreneurship Initiative

58

NY-12784: Ithaca Business Innovation Center

The project was initiated by Cornell University and the State and local government. It was 

designed to provide management and technical assistance to leverage Cornell’s research 

resources into job growth in high growth industries and firms. 

ARC $ Total $ Organizational 

Budget 

Project 

FTE Staff

Length in 

Months

Months since 

Completion

$43,000 $400,000 $250K - $1M 2 12 +25

Project Goal was to create new, quality jobs by delivering intensive business management 

assistance to a small number of businesses with significant growth potential. Specific objectives 

for the first three years were to (1) provide technical assistance to 45 firms, including intensive 

technical assistance to 15 (2) create 275 jobs, and (3) raise $350,000 for the Ithaca Business 

Innovation Center. 

ARC Metrics Target Actual

(ARC Final Report)

Actual

(RTS Survey)

# of jobs created∗ 286 Not reported 25

# of new businesses created - Not reported 9

# of jobs retained - Not reported 40

Measurable Achievements 

• 9 new businesses providing a total of 19 jobs

• Net gain of 40 jobs among clients - 10 created in new businesses, 40 created in existing 

businesses, and 10 lost at existing businesses 

• Five businesses have developed new products and five have updated technologies

• Businesses have increased out-of-state sales, but the amount is not known.

Notes: The project is an ongoing project that received ARC funding for year one. It has no 

formal internal evaluation. The 10 jobs created plus the 19 in new companies are well below 

the original goal of 286 jobs created; the 40 companies assisted is near the goal of 45, but the 8 

receiving intensive assistance is below the target of 15. While helping local firms, the grantee 

did not achieve the intended scale of activity or impact. Problems noted were in marketing 

(attracting entrepreneurs), attracting staff, and monitoring outcomes. The respondent implied 

that the original demand for services was overly optimistic by suggesting a better analysis of 

demand. The partner organization also proved difficult to work with. 

Clients’ Reactions: The two companies surveyed were both small high tech firms with rapid 

growth aspirations. The owner of a biotechnology company reported that he was satisfied with 

the financial management services he received, which helped him to develop a modified 

management system. In the past two years, his company has expanded from four to six 

employees. Ninety percent of the company’s sales are non-local.

The other client, who owns an information technology consulting firm, was very satisfied with the 

program. Because of the capital assistance he received, he was able to invest $500,000 in his 

firm, which has grown from 16 to 20 employees in 2 years. All of the firm’s sales are non-local. 

 

∗

 Job creation numbers do not include jobs generated through self-employment. Self-employment is 

captured in the business creation data.
Evaluation of the ARC Entrepreneurship Initiative

59

NY-12971: Worker Ownership Resource Center (WORC) Entrepreneurial Partnership

The project is a tri-county partnership between WORC, the Corning Community College SBDC, 

and the Hornell Business Assistance Center. WORC provided advanced “second stage” 

training to established small business owners in Chemung County, most of whom had 

previously accessed WORC’s entrepreneurial training.

ARC $ Total $ Organizational 

Budget 

Project 

FTE Staff

Length in 

Months

Months since 

Completion

$60,000 $81,898 $50K - $249K 2 22 9

Project Goal was to provide training and mentoring to assist entrepreneurs and microenterprises 

at all stages of growth, through the provision of training seminars, one-on-one technical 

assistance and a comprehensive package of services and supports.

ARC Metrics Target Actual

(ARC Final Report)

Actual

(RTS Survey)

# of new businesses created 40 15 15

# of jobs created∗ Not reported 52 52

# of jobs retained 11 11

Measurable Achievements 

• 15 new business start-ups

• 10 business expansions based on increased sales, new employees, and new product lines

• 52 new jobs created; 11 retained

• Provided assistance with securing over $500,000 in financing for both business start-up and 

expansion

Clients’ Reactions: The two companies surveyed were both in retail and primarily interested in 

operating a business for stability and lifestyle reasons. Each was assisted through more than 

20 interactions over a period of several weeks, and each was able, due to the program, to 

create a new business plan, make new investments, and enter new markets with new products 

and services. One client, a natural products retailer, received entrepreneurial training and 

management and technical assistance. She made a $90,000 investment to start her business 

due to the program, and has seen sales grow to $3,000 per month. Another client, a book store 

owner, received entrepreneurial training and has also increased investments in her business 

due to the program.

Both firms were very satisfied with the program and felt that the services exceeded their 

expectations. Both respondents expressed concern about the cancellation of the program in 

their particular area. Other locations in the state are too inconvenient for them to access 

services.

 

∗

 Job creation numbers do not include jobs generated through self-employment. Self-employment is 

captured in the business creation data.
Evaluation of the ARC Entrepreneurship Initiative

60

NY-12972: Ceramics Corridor Cluster Project

This project supports the continued development of a ceramics industry cluster organization in 

the southern tier counties of New York by creating an entity to function as the center of a 

Ceramics Corridor. The entity is charged with promoting closer relationships among firms and 

increased rate of growth for the industry. 

ARC $ Total $ Organizational 

Budget (s)

Project 

FTE Staff

Length in 

Months

Months since 

Completion

$120,000 $150,000 $250K - $1M 0.33 24 +1

Project Goal was to “develop and galvanize the Ceramics Corridor so that it has an atmosphere 

that nurtures entrepreneurship, innovation, and the attraction/retention/expansion of ceramics 

firms.” Specific objectives were to (1) survey interest and then develop a participatory industry 

trade association, (2) assist Alfred Technology Resource, Inc. in efforts to market its incubator, 

(3) start a venture development corporation, and (4) support the Alfred University internship 

program. During the project, goals were broadened to include prospective firms. 

ARC Metrics Target Actual

(ARC Final Report)

Actual

(RTS Survey)

# of participants 29 13 26

# of jobs created∗ 10 0 Not reported 

# of jobs retained 10 0 Not reported

Measurable Achievements 

• Documented business interest and established an organization

• Jobs created by participating firms but numbers not provided

• Five businesses have upgraded technologies

• Businesses have increased out-of-state sales but numbers were not provided

• Created web site for the organization (www.ceramicscorridor.org)

Note: This project is part of a very large-scale undertaking that claims responsibility for almost 

5,000 new jobs. It has other sizable sources of revenue and thus continues beyond the end of 

ARC funding.

Clients’ Reactions: The two clients are managers of companies that are part of the cluster and 

generally participate in cluster functions. Both appreciate the cluster and value its tangible and 

intangible services, including access to students and employees. Both have received 

management and technical assistance, and both appreciate the opportunity to be part of a 

network. One company, an electrical systems and design firm, attributes its recent success in 

obtaining new patents to its participation in the project. It is questionable, however, whether the 

participating firms understand what part of the regional effort is supported by ARC. 

 

∗

 Job creation numbers do not include jobs generated through self-employment. Self-employment is 

captured in the business creation data.
Evaluation of the ARC Entrepreneurship Initiative

61

NY-13215: Entrepreneurial Assistance Program (ACCORD)

This award went to the Allegheny County Community Opportunities and Rural Development 

Corporation (ACCORD) to shift its entrepreneurial emphasis to the second stage of business 

development. 

ARC $ Total $ Organizational 

Budget

FTE

Staff

Length in 

Months

Months since 

Completion

$39,424 $73,924 $1 - $10 M 1 24 +7

Project Goals are to increase entrepreneurship through (1) awareness campaigns, (2) 

developing relationships between businesses and counselors, and (3) “second stage” (small 

businesses ready for expansion) by training of adults. 

ARC Metrics Target Actual

(ARC Final Report)

Actual

(RTS Survey)

# of participants 20 8 178

# of new businesses created 3 1 12

# of jobs created∗ 18 14 14

# of jobs retained 9

Measurable Achievements

• Made 16 loans totaling $400,000

• Held 12 local meetings

• Started 99 businesses without employees and 18 with employees

• Created 11 jobs and saved 9 in existing firms

• Helped 2 firms develop new products and 12 upgrade technologies

• The project is continuing beyond the funding period

Notes: The outcomes appear high given the resources, a staff of one FTE, and the goals—

which do not, for example, include making loans. It is possible (probable) that some of the 

outcomes are organizational outcomes and indistinguishable from those funded by the grant. 

The outcomes, while impressive, appear slanted toward first stage, not second stage 

entrepreneurs. No major or minor obstacles were cited. 

Clients’ Reactions: Both clients surveyed received technical assistance and training through the 

program. One, the owner of a machining company with 8 employees whose goal is rapid 

growth, received technical assistance over three years. Due to this assistance, he has made a 

$70,000 investment in new equipment and has expanded into new markets. He has seen 100% 

sales growth over two years (from $50,000 to $100,000.

The other surveyed client is a self-employed hairdresser whose goal is a stable, lifestylesupporting business. She received technical assistance and training over four months, which 

helped her to develop a new business plan. Both clients were very satisfied with services 

received.

 

∗

 Job creation numbers do not include jobs generated through self-employment. Self-employment is 

captured in the business creation data.
Evaluation of the ARC Entrepreneurship Initiative

62

OH-13158: Byesville Area Entrepreneurial Development Partners

The grantee is a non-profit community development corporation that is targeting the town of 

Byesville and the county of Guernsey. 

ARC $ Total $ Organizational 

Budget

FTE

Staff

Length in 

Months

Months since 

Completion

$48,940 $61,175 > $50,000 2 12 +13

Project Goals were to (1) make micro-loans to twelve entrepreneurs, (2) assess the operations 

of forty enterprises, (3) award three scholarships to youth, (4) conduct twelve workshops, and 

(5) mail 200 newsletters per month. Goals also included the recruitment and training of a 

twelve-member entrepreneurship development council, and recruiting and assisting forty 

entrepreneurs.

ARC Metrics Target Actual

(ARC Final Report)

Actual

(RTS Survey)

# of participants 100 53 138

# of new businesses created 12 5 11

# of jobs created∗ 12 9 12

Measurable Achievements 

• 3 new micro-businesses started by youth

• 8 new micro-businesses started by adults

• 12 jobs created

• 6 scholarships awarded to area high school students, totaling $3,750

• Provides only wireless Internet service to Southeastern Ohio

Notes: The grantee noted a high failure rate that must be tolerated in high-risk situations. Key, 

according to grantee, are a committed entrepreneur and continual follow-up support. 

Clients Reactions: The owners of two “lifestyle” service micro-enterprises (each with two 

employees) were each very satisfied with the assistance they received. Both were helped with 

the paperwork needed to start or upgrade their businesses. The owner of an insulation services 

company received a loan to upgrade his business, to which he attributes his recent growth in 

sales to $200,000 per year. 

The other surveyed client was able to start a seafood retailing business as a result of the 

technical and capital assistance she received through the program, and generated $60,000 in 

sales in the past year. Each said they would have started the business without the help, but it 

would have taken longer and probably not been as successful. 

 

∗

 Job creation numbers do not include jobs generated through self-employment. Self-employment is 

captured in the business creation data.
Evaluation of the ARC Entrepreneurship Initiative

63

OH-13166: Jackson County Entrepreneurial Assistance

This project’s target population is the welfare recipients in a county with high poverty and 

unemployment rates. 

ARC $ Total $ Organizational 

Budget

FTE

Staff

Length in 

Months

Months since 

Completion

$38,850 $68,350 $50K - $250 K 1 12 +12

Project Goals were to provide technical assistance to 60 potential entrepreneurs and assist five 

companies in acquiring capital. 

ARC Metrics Target Actual

(ARC Final Report)

Actual

(RTS Survey)

# of participants 60 40 108

# of new businesses created 15 12 16

# of jobs created∗ 15 43 23

# of jobs retained 9

The program also provided technical assistance to 15 existing businesses and education for 6 

youth, not minor targets. 

Measurable Achievements 

• 1 new business started by student

• 7 new businesses with no employees started by adults

• 9 new businesses with employees started by adults (20 employees)

• 3 jobs created in and 9 jobs saved in existing businesses

• 2 upgraded technologies, 1 developed new products

Clients’ Reactions: Only one client contact was provided by the program. This client’s 

business, a building and construction company, was in decline when he bought it in January 

1997. Through the program, he was able to invest $150,000 in new facilities, product 

development and market expansion. As a result, the company’s sales (which are 98% nonlocal) have more than doubled – from $200,000 two years ago to $425,000 this year. Employee 

compensation has increased by $2.00/hour, and salaries will increase by another $1.50/hour as 

new equipment is added. The owner also attributes improvements in personnel management to 

his participation in this program. Of the several economic development programs he has 

worked with, he says that this is the best. 

 

∗

 Job creation numbers do not include jobs generated through self-employment. Self-employment is 

captured in the business creation data.
Evaluation of the ARC Entrepreneurship Initiative

64

OH-13401: ACENet “Building Entrepreneurial Networks”

This project in Southeast Ohio, an area that has had very high poverty rates and out-migration 

of youth, is managed by a well-established community based economic development agency 

that has fostered entrepreneurial activity and networking for 14 years. The grant is to expand 

the pilot Computer Opportunities Program, which prepares youth to start computer consulting 

businesses, to five high schools in the area. 

ARC $ Total $ Organizational 

Budget (s)

Project 

FTE Staff

Length in

Months

Months since 

Completion

$50,000 $62,500 $250K-$1M 1 14 +3

Project Goals were to (1) provide at least 45 high school students with entrepreneurial 

education and training that will lead them to start their own computer consulting business, get a 

job in an existing firm, or go on to higher education; (2) set up three new computer technology 

and entrepreneurship centers in high schools; and (3) involve at least 100 teachers, students 

board members, principals and 50 service providers in one of three workshops. A broader goal 

is to increase entrepreneurial community by the infusion of students. The project helped to 

expand the program from two to seven schools.

ARC Metrics Target Actual

(ARC Final Report)

Actual

(RTS Survey)

# of participants 45 30 32

# of new businesses created 3 0 0

# of jobs created∗ 14 0 0

Measurable Achievements 

• 29 school-based businesses were started by youth

• Documented curriculum modules with objectives, lesson plans, etc. 

• Leveraged more than $1 M in additional support

Notes: This grant supplements and expands a much larger program that began the previous 

year. Of 26 graduates in the pilot program at 2 schools, 2 students began new businesses, 8 

are employed in IT sector, and 14 went on to higher education. The prepared program 

materials are impressive and the program has had considerable media coverage. A large part 

of the curriculum is on line and therefore easily expanded and replicated in other places. 

Monitoring economic project outcomes has proven difficult although it has a detailed student 

evaluation instrument. The internal tracking and evaluation system is being revamped. 

Although the ARC funding has ended, the project is on-going with a $1 M federal line item 

appropriation. 

Client/Partner Reactions: Because this program’s participants were all under eighteen years 

old, no client surveys were conducted. However, two partner organizations were contacted for 

their perspectives on the project. One of the partner organizations was a farm and the other 

was a non-profit service provider. Both expect to continue working with the program over the 

next year and indicated positive impacts on their organization as a result of their involvement 

with the project.

 

∗

 Job creation numbers do not include jobs generated through self-employment. Self-employment is 

captured in the business creation data.
Evaluation of the ARC Entrepreneurship Initiative

65

PA-12786: Team Pennsylvania Entrepreneurial Education Initiative 

This grant was made to a state agency to implement a variety of existing approaches of 

targeting youth with entrepreneurial education in the elementary schools, middle schools, high 

schools and community colleges and assisting entrepreneurs in six rural counties. It utilizes 

established and proven models developed, implemented, and supported by REAL Enterprises. 

ARC $ Total $ Organizational 

Budget

FTE

Staff

Length in 

Months

Months since 

Completion

$333,368 $530,000 $250K - $1 M 2 36 +2

Project Goals were to develop entrepreneurial skills and foster new businesses by raising 

awareness, building infrastructure of school/business partnerships, and nurturing 

entrepreneurial environments in depressed or rural areas. This would be accomplished by 

implementing REAL in ten high schools and community colleges, and in ten primary schools in 

six Pennsylvania counties.

ARC Metrics Target Actual

(ARC Final Report)

Actual

(RTS Survey)

# of participants 184 151 240

# of students Not reported 6,489 1,620

# of new businesses created 10 3 4 (by youth)

Measurable Achievements 

• Few measurable outcomes, which is not surprising since the project primarily targeted youth 

and changing a culture. 

• The three success stories cited appear to describe potential growth businesses.

• The survey cited 4 new businesses started by youth and 60 businesses increased sales.

Note: It is difficult to find evidence of cultural shifts without intensive in-site interviews and 

project resources to track participants. Much of the success of this type of long term project 

must be taken on faith. Intermediate outcomes, however, suggest potential impacts. The web 

site is current and describes activities. Support from a national organization such as REAL is a 

factor in the success. The grantee is applying for funds from the state to continue the program. 

Clients’ Reactions: Two former adult students have started their own businesses. One owns a 

replacement glass installation service, and wants to maintain a stable, lifestyle-supporting 

business. He credits the program with his success in obtaining capital for a $25,000 start-up 

investment, and with giving him the technical assistance and management skills to run his own 

business. His sales have grown in two years from $75,000 to $129,000.

The other surveyed client is a former accountant who wants rapid growth for her business 

making viscose hair wraps. one installing replacement glass (his former job) as a lifestyle 

business and one, a former accountant, making viscose hair wraps (to reduce blow dry time) 

and wants to grow the business. Like the client described above, she credits the skills she 

learned in this program with her success in obtaining $2,300 in start-up funds. Both clients were 

satisfied with the experience; one ranked it excellent, one good. 
Evaluation of the ARC Entrepreneurship Initiative

66

PA 12904: Team PA Entrepreneurial Network Initiative (Phase I)

The purpose of this project is to provide managerial and technical assistance to entrepreneurs 

and support collaborative efforts to coordinate improved assistance for entrepreneurs. 

ARC $ Total $ Organizational 

Budget 

Project 

FTE Staff

Length in 

Months

Months since 

Completion

$550,600 $688,250 Over $10M 4 17

Project Goal was to develop entrepreneurial networks throughout the 52 ARC counties in 

Pennsylvania. Specific objectives in the first phase of this project were to: (1) implement 7 

regional plans to support entrepreneurship; (2) deliver 12 financial management seminars 

targeting 360 firms; (3) conduct 15 focus groups targeting potential and existing entrepreneurs; 

(4) conduct outreach to encourage cooperation among approximately 55-60 public/private 

service providers; and (5) increase awareness of entrepreneurial resources among the general 

public. 

ARC Metrics Target Actual

(ARC Final Report)

Actual

(RTS Survey)

# of participants 960 900 750

Measurable Achievements 

• Completed 7 regional plans to support entrepreneurship

• Conducted 15 financial management seminars involving 300+ people

• Conducted 20 workshops with over 600 private participants

Notes: Phase I of the project has been completed. Initial objectives were met or exceeded. 

Additional ARC funds were received for Phase II which is now underway. The first phase was 

essentially to build the organizational infrastructure for the initiative. Direct services to 

businesses will occur in the second phase. To date, the success of this program lies in the 

development of entrepreneurial networks that did not previously exist. Measurable business 

outcomes are expected to occur in year 3 of the project. 

Clients’ Reactions: Because this project was funded for its developmental phases, actual 

business clients have not yet been served. The participants indicated above were mostly 

service providers who participated in the “Train the Trainer” program. Reactions from firms 

served will be useful in evaluating Phase II of the project.
Evaluation of the ARC Entrepreneurship Initiative

67

CO-12988 (TN): Levi Strauss Employees Entrepreneurial Training

The purpose of the project was to help displaced Levi Strauss employees make a successful 

transition to new work opportunities following the closing of the Levi Strauss plant in Knoxville, 

Tennessee. 

ARC $ Total $ Organizational 

Budget 

Project 

FTE Staff

Length in 

Months

Months since 

Completion

$80,000 $170,000 $250-$999K 4 17 14

Project Goal was to provide start-up, small business management, and business development 

training, including courses on business plans, marketing, financial planning, taxes, licensing, 

and legal forms of organization. In addition, the project proposed to (1) establish an on-site 

resource center; (2) form community micro-loan peer lending groups; (3) conduct leadership 

development and financial management training; and (4) provide individual credit counseling.

ARC Metrics Target Actual

(ARC Final Report)

Actual

(RTS Survey)

# of participants 750 880 1,202

# of new businesses created 750 102 102

Measurable Achievements:

• Of the 880 former Levi Strauss workers who received training, 102 started their own full-time 

businesses.

• 778 trainees decided to seek full-time employment and use their training to start-up a parttime micro-business for supplemental income.

• 90% of the full-time business start-ups were by female heads of households. 

Clients’ Reactions: Both clients surveyed started their business for stability and lifestyle 

reasons. One, a restaurant owner, was assisted through 6-10 interactions over a period of 4 

weeks. Through the program, she developed a business plan and invested $15,000 to open her 

business; however, the restaurant is now closed due to problems with a building landlord. 

The other surveyed client had 3-5 interactions with program staff over a period of three weeks. 

Through the program, she created a business plan and invested $20,000 to open a child care 

center in her home. Over the past two years, her sales have grown 25%. Both clients were 

very satisfied with the services received and felt their expectations were met.
Evaluation of the ARC Entrepreneurship Initiative

68

VA-12923: Clinch Powell Value Added Agriculture Marketing Project1

The Clinch Powell Sustainable Development Initiative, in partnership with community-based 

organizations and Virginia Cooperative Extension, proposed to support efforts to develop valueadded agricultural businesses in Southwest Virginia and Northeast Tennessee. 

ARC $ Total $ Organizational 

Budget 

Project FTE 

Staff

Length in 

Months

Months since 

Completion

$36,000 $71,875 Unspecified Unspecified 18 12

Project Goal was to identify and develop markets for high value agricultural products produced 

in the region. Specific objectives were to: (1) gather additional market information about high 

value opportunities; (2) develop links to commercial and institutional buyers interested in 

sustainable raised products; (3) identify value-added processing needs and purchase needed 

equipment; and (4) educate and train entrepreneurs in equipment and regulatory issues related 

to value-added food production.

 

ARC Metrics Target Actual

(ARC Final Report)

Actual

(RTS Survey)

# of participants 30 108 No survey returned

Measurable Achievements 

• Identified 300 individuals with an expressed interest in producing value-added food products

• Identified a more focused group of 60 entrepreneurs with high potential for developing 

commercial food products

• A commercial kitchen was established in Sneedville, TN—the Jubilee Project Incubator

• A survey of 10,000 households in the region was completed, with results indicating 

significant interest in locally produced organic and sustainable products

• An Appalachian Harvest label and marketing program was initiated

• Locally produced products were distributed in 7 grocery stores

• A $25,000 contract for organic pepper products was negotiated with a major catalogue 

retailer

• 17 farmers obtained organic certification for their land

• 108 persons attended 4 workshops on sustainable production and marketing

 

1

 No survey was received for this project. All information collected from ARC project files. No 

clients/participants were contacted.
Evaluation of the ARC Entrepreneurship Initiative

69

VA 13302: Appalachian Partnership Project

This grant, which matches other foundation funding, was awarded to People, Incorporated, an 

existing community action agency. The purpose of the grant was to form a network of faithbased communities in Virginia and Tennessee, and of home-based businesses whose products 

are traditional Appalachian crafts. The network managing the operations is an existing 

cooperative called “Applamade.” 

ARC $ Total $ Organizational 

Budget (s)

Project 

FTE Staff

Length in 

Months

Months since 

Completion

$40,500 $121,034 $1M - $10 M 4 12 +10

Project Goals include reaching a broader market outside the Appalachian region, increasing 

income for producers (especially women), and providing an outlet for those people interested in 

assisting rural Appalachians. Specific program goals include (1) the creation of 40 designs for 

production and promotion, (2) training new craft producers, (3) submitting and marketing crafts 

for sale in a church-based project, and (4) assisting 40 low-income craft-skilled producers to 

expand their markets (triple aggregate sales) and increase their incomes by at least $1,000 

each, working with churches to market products. 

ARC Metrics Target Actual

(ARC Final Report)

Actual

(RTS Survey)

# of participants 50 60 37

# of new businesses created 20 30 10

# of jobs created∗ 50 30 0

Measurable Achievements 

• 10 new home-based businesses started

• 35 new products developed

• 45 businesses increased sales by total or $30,000 (target of $40,000)

Notes: Grantee noted difficulty in attracting customers to the services offered and getting 

church participation. Though it is not clear exactly what church participation means, it appears 

that the churches are the intermediaries for selling goods as part of fund raising events, keeping 

a percentage of the sale. Although total sales increased by $30,000, one person cited as 

success story apparently accounted for one third of the total. The grantee also noted that the 

training and marketing is very time consuming. 

Clients: This project is currently being evaluated by the Aspen Institute, which will be surveying 

and interviewing clients. So as not to interfere with this evaluation process, we did not survey 

clients from the program.

 

∗

 Job creation numbers do not include jobs generated through self-employment. Self-employment is 

captured in the business creation data.
Evaluation of the ARC Entrepreneurship Initiative

70

VA-13349: Wise Skills Center 

Wise is located in the western part of Virginia’s Appalachian region. The public school system 

is the grantee, and intends to build the capacity for the region (including parts of eastern 

Tennessee) to become part of the digital economy by creating a self-sustaining skill center that 

targets unemployed and underemployed residents. 

ARC $ Total $ Organizational 

Budget

FTE

Staff

Length in 

Months

Months since 

Completion

$50,000 $58,110 $250K – $1M 1 12 +11

Project Goals are (1) to build a self-sustaining local economy by training a work force to help the 

region become part of the digital economy, (2) to create enterprises to train computer 

technicians, and (3) to prevent out-migration of youth by providing better job opportunities. 

The project expected to give computer technician training to students selected from among the 

unemployed, underemployed, or laid-off workers, to establish a retail outlet for computer 

assembly and repair staffed by five students, place seven students in area businesses, and 

extend training of three students to further raise skill levels.

ARC Metrics Target Actual

(ARC Final Report)

Actual

(RTS Survey)

# of participants 15 21 24

# of new businesses created 1 1 1

# of jobs created∗ 6 7 8

Measurable Achievements: 

• One new business started (Lighthouse Computer Systems); 300 computers built or 

upgraded, resulting in a gross profit of $17,418

• Six jobs added to existing businesses

• 22 students enrolled in training program; 10 dropped out, 1 completed, 9 enrolled, 2 took 

jobs

Notes: Classes have continued past the expiration date, and students’ job search progress is 

supported and monitored for 13 weeks. 

Clients’ Reactions: One woman took the training because she wanted specific skills and the 

college required too many other courses. She remained in program for 18 months, assisted the 

instructor, and repaired about 100 computers. Through the program, she received technical 

assistance, networking opportunities, and developed a business plan. She was very 

enthusiastic about the program and is about to start her own business. 

The second client had an opposite reaction—he felt that he learned very little. What technical 

assistance he did get was from the book, not the class, for which he blames the instructor. The 

two very opposite reactions to the same intervention may reflect different aptitudes and 

interests.

 

∗

 Job creation numbers do not include jobs generated through self-employment. Self-employment is 

captured in the business creation data.
Evaluation of the ARC Entrepreneurship Initiative

71

WV-12917: West Virginia Micro Loan and Rural Outreach

This grant went to an existing small business development center to improve the quality of 

services offered to rural entrepreneurs and the quality of micro lending programs throughout the 

state. The funds were intended to support a micro loan program liaison to offer technical 

assistance to micro loan seed fund recipients. 

ARC $ Total $ Organizational 

Budget (s)

Project 

FTE Staff

Length in 

Months

Months since 

Completion

$68,680 $85,850 $1M - $10M 1.5 26 +9

Project Goals were to provide on-site technical assistance to micro lenders, train in all aspects 

of lending and entrepreneurial support, and to promote leveraging and networking among 

members, regional SBDCs, and community organizations. 

ARC Metrics Target Actual

(ARC Final Report)

Actual

(RTS Survey)

# of participants Not reported 24 14

# of jobs created∗ 1 1 Not reported

The project apparently did much of the assistance to and training of lenders that it proposed, 

although that was not reflected in the survey returned. 

Measurable Achievements (attributable to entire program, not solely ARC portion)

• 354 jobs created

• 204 jobs saved

• 85 firms without employees started, 148 firms with employees started

• 10 businesses developed new products

• 25 upgraded technologies

Note: Two interventions of successful micro loans to companies, one to a welfare client to start 

a business and one to expand a business by opening satellite offices, were cited to illustrate 

success of the program. There was no formal evaluation used. The “job” outcomes cited are 

clearly associated with the overall program, not the grant. Obstacles cited were low attendance 

at the workshops, too little experience managing a loan fund among lenders, and insufficient 

staff support. It is unlikely that this program will be sustainable.

Clients’ Reactions: Both clients surveyed seek rapid growth for their businesses, and both report 

that they are very satisfied with their participation in the program. The client who opened the 

day care center credits the program with giving her management assistance and the start-up 

funding she needed to open the business, which now generates $15,000 in sales per year.

The other client, who owns a vocational rehabilitation center, says that the program helped him 

to grow his business by providing the management assistance and expansion capital that he 

needed to open new locations. In two years, his yearly sales have grown from $460,000 to 

$700,000.

 

∗

 Job creation numbers do not include jobs generated through self-employment. Self-employment is 

captured in the business creation data.
Evaluation of the ARC Entrepreneurship Initiative

72

WV-12929: Small Business Innovation Research-Phase Zero

This project was designed to enhance the number of Small Business Innovation Research 

(SBIR) proposals being submitted from West Virginia firms. The SBIR Phase Zero program will 

enable eligible small businesses to develop proposals in response to actual SBIR Phase I 

solicitations from the various participating federal agencies. The grantee, the West Virginia 

High Technology Consortium Foundation, is a non-profit organization supporting research, 

development, and education.

ARC $ Total $ Organizational 

Budget (s)

Project 

FTE Staff

Length in 

Months

Months since 

Completion

$96,518 $120,648 $1M-$10M 3 28 +2

Project Goals were to (1) increase awareness of the SBIR program, (2) increase the number of 

West Virginia firms submitting SBIR phase I proposals to the federal government and improve 

the quality of the proposals by providing 20 firms with $3,000 bid and proposal money, and (3) 

foster a better working relationship between small businesses and the academic community. A 

non-operational goal was to create a model program that could be replicated elsewhere in 

Appalachia. During the project goals were modified by revising the target market. 

ARC Metrics Target Actual

(ARC Final Report)

Actual

(RTS Survey)

# of participants 20 17 25

Measurable Achievements 

• 23 firms received Phase Zero awards (out of 55 proposals received)

• Two West Virginia firms were awarded SBIR Phase I grants

• The number of West Virginia firms submitting SBIR proposals increased

• Businesses have developed new products and upgraded technologies, but the number of 

firms is not known.

Note: It does not appear that the program has continued beyond the ARC funding period.

Clients’ Reactions: Both clients surveyed are the owners of high-technology start-ups, for which 

they seek rapid growth. Both received grants through the program for start-up capital; while 

neither gave specific information about sales and payroll, both businesses seem to be 

performing well. One client, the owner of an information research firm that provides ecommerce and web-based services, said that the program helped him to develop new products. 

Another client, whose firm designs and produces optical devices, said that the grant helped the 

firm to solidify its technical approach and also facilitated certain aspects of commercialization.
Evaluation of the ARC Entrepreneurship Initiative

73

WV-13271: Sustainable Community Enterprise Project (Lightstone CDC)

The purpose of the project was to strengthen the entrepreneurial infrastructure by enhancing 

access to capital and technical assistance, and to provide low-income individuals with viable 

options as part of an effective welfare-to-work strategy.

ARC $ Total $ Organizational 

Budget 

Project 

FTE Staff

Length in 

Months

Months since 

Completion

$107,620 $134,620 $250-$999K 5 16 <1

Project Goal was to promote social, economic, and community capital development by: (1) 

building the capacity of two existing micro-loan funds; (2) providing loans and business start-up 

assistance to support self-employment for low-income individuals; (3) co-sponsoring regional 

workshops to create market opportunities for micro-enterprise; and (4) supporting welfare-towork through micro-enterprise.

ARC Metrics Target Actual

(ARC Final Report)

Actual

(RTS Survey)

# of participants 48 630 650

# of jobs created∗ 41 43 22

# of new businesses created 21

Measurable Achievements 

• Provided support to 159 individuals seeking micro-enterprise assistance

• Provided 111 referrals for business technical training

• Made 10 loans to individuals

• Made 11 loans and equity investments to support new and existing businesses

• 43 jobs created

• Generated total community wealth of $2,959,000

Clients’ Reactions: The three companies surveyed are very small (1-2 employees) and primarily 

in business for stability and lifestyle reasons, although one expressed a desire to grow rapidly. 

None of them have experienced any growth over the past two years. One client, whose firm is 

engaged in windshield and glass repair, reported receiving technical assistance and access to 

capital, enabling him to invest between $10,000 and $15,000 in the business. Another client 

surveyed installs carpet, vinyl, and tile, and also reported receiving technical assistance and 

access to capital. Though her business has not grown, this client credited the program with her 

success in leaving welfare and buying a home. 

These two business owners were very satisfied with the services received and felt that their 

expectations were exceeded. The third client surveyed, who makes and sells hand-painted 

ceramics, was dissatisfied with the services and felt that support was not provided as promised 

(she reported access to capital as the only service received.) That same business owner 

indicated that there were administrative problems and delays in processing loan, and blames 

the program for her worsened debt status.

 

∗

 Job creation numbers do not include jobs generated through self-employment. Self-employment is 

captured in the business creation data.
Evaluation of the ARC Entrepreneurship Initiative

74

Appendix B 

Summary of Results from Project Survey

What kind of organization is the host organization for this 

project?

Frequency Percent

Non-profit economic development organization 12 52.2

Other (explain): 4 17.4

Multi-county or regional planning agency 2 8.7

University 2 8.7

Local or county government agency 1 4.3

Community organization 1 4.3

Community college or 2-year institution 1 4.3

Total 23 100.0

N=23.

What is the approximate annual budget of the host organization? Frequency Percent

Under $50,000 1 4.3

$50,000 - $249,000 2 8.7

$250,000 - $999,000 10 43.5

$1 million - $10 million 6 26.1

Over $10 million 4 17.4

Total 23 100

N=23.

The project that ARC funded was Frequency Percent

A new program 14 60.9 

A program that built on earlier funded projects 9 39.1 

Total 19 100.0

N=23.

If the program built on earlier funded projects, how were those 

projects sponsored?

Frequency Percent 

of 

projects

Previous ARC funds 2 22.2

Other federal funds 2 22.2

State funds 2 22.2

Organization’s own resources 1 11.1

Foundation 2 22.2

N=9. Note: Multiple responses for a single project possible.

ARC project status at time of survey (Sep-Oct 2000) Frequency Percent

Closed 15 75.0

Open 5 25.0

Total 20 100.0

N=20.

Months ARC project in operation through to September 2000 Months

Mean 21.5

Median 18.0

Shortest 10.0

Longest 48.0
Evaluation of the ARC Entrepreneurship Initiative

75

N=21.

How were the goals established for the project? Frequency Percent 

of 

projects

Formal planning study or needs assessment 8 34.8

Discussion and experience within the host organization 15 65.2

Discussion involving other members of the local community 17 73.9

Other 2 8.7

N=23. Note: Multiple responses for a single project possible.

Did you change or add new goals during the period of the 

project?

Frequency Percent 

of 

projects

Yes 10 43.5

No 13 56.5

N=23.

Principal activities undertaken by the ARC project. Major 

emphasis

Percent 

of 

projects

One-on-one assistance for new start-up businesses 17 73.9

Education and training of adults 15 65.2

Business network groups for new start-up businesses 11 47.8

One-on-one assistance for existing businesses 11 47.8

Business opportunities research and marketing 10 43.5

Business seminars 10 43.5

Business network groups for existing businesses 9 39.1

Curriculum development 8 34.8

Loan or equity fund 8 34.8

Local and regional needs assessment 8 34.8

Training of staff and other trainers 8 34.8

Referral systems to other service providers 7 30.4

Community conferences, organizational meetings 6 26.1

Education and training of students (high school, vocational school) 5 21.7

New local economic development strategies 4 17.4

New physical facilities developed 4 17.4

Technology opportunities research 4 17.4

Distance learning 2 8.7

Government procurement assistance 2 8.7

Web site 2 8.7

Business internships 1 4.3

Custom software development 1 4.3

Other 1 4.3

N=23. Note: Multiple responses for a single project possible.
Evaluation of the ARC Entrepreneurship Initiative

76

Participants 

reported served

Participants served by activities during 

period of ARC funding

Projects 

reporting 

this 

activity

Percent 

of 

projects Total Median

Education and training of youth, high school 

students, or vocational students

8 34.8 1,509 53

Education and training of adults 19 82.6 3,050 68

Individual assistance to existing businesses 

which already have employees

12 52.2 132 10

Individual assistance to existing businesses 

which do not have employees (e.g. selfemployed or micro-business)

12 52.2 150 15

Individual assistance to adults seeking to start 

new businesses

16 69.6 854 42

Assistance to network groups

(*Businesses **Groups)

12 52.2 275*

102**

26*

5**

Direct financial assistance, loans, or loan 

guarantees (*Businesses **Thousands)

10 43.5 78*

$1,164.6**

8*

$179.8**

Referrals to other sources of assistance 19 82.6 800 29

Meetings with community leaders, business 

representatives, and other local organizations to 

discuss projects or plans (*Meetings; **People 

participating, without duplication)

22 95.7 313*

1,063**

12*

60**

N=23

Education and training of youth, high 

school students, or vocational 

students

Under 8 hours 8 to 30 hours More than 30

hours

Projects reporting 1 2 4

Number of participants by training 

contact hours category

1 38 770

Percent of participants by training 

contact hours category

0.1 4.7 95.2

N=6 projects reporting

Education and training of adults Under 8 hours 8 to 30 hours More than 30 

hours

Projects reporting 9 6 12

Number of participants by training 

contact hours category

1,432 884 504

Percent of participants by training 

contact hours category

50.8 31.3 17.9

N=17 projects reporting
Evaluation of the ARC Entrepreneurship Initiative

77

Individual assistance to existing

businesses which already have 

employees

Under 8 hours 8 to 30 hours More than 30 

hours

Projects reporting 6 5 2

Number of business assists by service 

time categories

51 35 46

Percent of business assists by service 

time categories

38.6 26.5 34.8

N=9 projects reporting

Individual assistance to existing

businesses which do not have 

employees (e.g. self-employed or 

micro-business)

Under 8 hours 8 to 30 hours More than 30 

hours

Projects reporting 2 6 4

Number of business assists by service 

time categories

12 79 56

Percent of business assists by service 

time categories

8.2 53.7 38.1

N=8 projects reporting

Individual assistance to adults 

seeking to start new businesses

Under 8 hours 8 to 30 hours More than 30 

hours

Projects reporting 4 4 8

Number of business assists by service 

time categories

104 127 107

Percent of business assists by service 

time categories

30.8 37.6 31.7

N=11 projects reporting

Direct financial assistance, loans, or 

loan guarantees

Under $5,000 $5,000-

$25,000

Over $25,000

Projects reporting 4 3 4

Number of business assists by service 

time categories

23 17 16

Percent of business assists by service 

time categories

41.1 30.4 28.6

N=7 projects reporting

Referrals to other sources of 

assistance

Financial 

assistance

Technological 

assistance

Other business 

assistance

Projects reporting 9 7 5

Number of business assists by service 

time categories

167 113 532

Percent of business assists by service 

time categories

20.6 13.9 65.5

N=10 projects reporting
Evaluation of the ARC Entrepreneurship Initiative

78

Able to report total 

participants served

Able to provide 

service details+

Activities during period of ARC 

funding

Projects 

reporting 

this 

activity No of 

projects

Percent of 

projects

No of 

projects

Percent of 

projects

Education and training of adults 19 18 94.7 17 89.5

Education and training of youth, 

high school students, or vocational 

students

8 7 87.5 6 75.0

Assistance to network groups 12 10 83.3 7 58.3

Direct financial assistance, loans, 

or loan guarantees 

10 8 80.0 7 70.0

Individual assistance to existing 

businesses which already have 

employees

12 9 75.0 9 75.0

Individual assistance to adults 

seeking to start new businesses

16 11 68.8 11 68.8

Individual assistance to existing 

businesses which do not have 

employees (e.g. self-employed or 

micro-business)

12 8 66.7 8 66.7

Referrals to other sources of 

assistance

19 10 52.6 10 52.6

Total - all reported activities 108 81 75.0 75 69.4

N=23. +Number of participants by different categories of service, e.g. by length of training, size of loans, 

or types of referral.

What was the primary geographical area served by the 

project?

Frequency Percent

Single county 7 30.4

Multi-county, within one state 13 56.5

State 2 8.7

Multi-county, two or more states 1 4.3

Total 23 100

N=23.

Budget for the total 

period of the project 

Projects 

reporting 

Percent of 

projects

Total

X 1,000

Share of 

program 

budget by 

source

Per 

project 

median

X 1,000

ARC 20 100% $2,605.5 52.0% $59.3

Other federal 4 20% $466.6 9.3% $108.3

State 9 45% $484.6 9.7% $43.0

County or local 8 40% $371.9 7.4% $35.0

Fee income 1 5% $5.0 0.1% $5.0

Other 11 55% $1,072.7 21.4% $60.0

N=20. Budget data as reported by respondents to the survey. Per project median is only for those 

projects receiving funds from line source.
Evaluation of the ARC Entrepreneurship Initiative

79

What was the staffing level 

for the project at peak 

staffing?

Projects 

reporting 

Percent of 

projects

Total Median

Full-time equivalent staff 23 100.0 62.5 2.0

- 2 or fewer staff 13 56.5 - -

- 3-4 staff 8 34.8 - -

- 5 or more staff 2 8.7 - -

N=23.

If you offered education and training courses as 

part of the ARC project, how did you track 

participant satisfaction and outcomes?

Projects 

reporting

Percent of 

projects

No formal evaluation used 3 15.0 

End-of-course student evaluations 13 65.0 

Time-elapsed students evaluations, at a time well 

after training has been completed

2 10.0 

Other 5 25.0 

N=20. Note: Multiple responses for a single project possible.

If you offered business assistance as part of the 

ARC project, how did you track participant 

satisfaction and outcomes?

Projects 

reporting

Percent of 

projects

No formal evaluation used 7 38.9 

End-of-course student evaluations 3 16.7 

Time-elapsed project evaluations, at a time well after 

projects are completed

3 16.7 

Other 4 22.2 

N=18. Note: Multiple responses for a single project possible. Two projects assisting businesses did not 

indicate the evaluation methods they used.

Summary of project reported business and employment impacts, 

as a direct result of project assistance over the total period of the 

ARC project

Projects 

reporting 

YES

Percent 

of 

projects

New businesses started by youth or students without employees 8 34.8

New businesses started by youth or students with employees 0 0.0

Number of new businesses started by adult participants w/o employees 12 52.2

Number of new businesses started by adult participants with employees 13 56.5

Jobs created in existing businesses 12 52.2

Jobs saved in existing businesses 9 39.1

Jobs lost in existing businesses 2 8.7

Businesses that have developed new products 18 78.3

Businesses that have upgraded technologies or management methods 13 56.5

Businesses that have increased sales outside of the state 12 52.2

Businesses that have increased export sales outside the United States 3 13.0

N=23.
Evaluation of the ARC Entrepreneurship Initiative

80

New businesses started by youth or students, as a 

direct result of ARC project assistance

Without 

employees

Projects reporting YES 8

Projects able to provide data on businesses started 5

Percent of projects able to provide data 63%

Reported number of youth businesses started 46

Median, by project 4

Minimum, by project 1

Maximum, by project 20

N=23. No project reported starting of youth businesses with employees.

New businesses started by adult participants, as a 

direct result of ARC project assistance

Without 

employees

With 

employees

Projects reporting YES 12 13

Projects able to provide data on businesses started 9 7

Percent of projects able to provide data 75% 54%

Reported number of adult businesses started 181 33

Median, by project 10 7

Minimum, by project 4 1

Maximum, by project 102 18

Projects able to provide data on employment in 

businesses started

- 4

Percent of projects reporting employment impacts - 31%

Reported number of employees 54

Median, by project - 15

Minimum, by project - 2

Maximum, by project - 20

Employment impacts on existing businesses as a 

direct result of ARC project assistance

Jobs created Jobs saved Jobs lost

Projects reporting YES 12 9 2

Projects able to provide data on employment impacts 7 4 1

Percent of projects able to provide data 58% 44% 50%

Reported number of jobs 121 85 10

Median, by project 6 10

Minimum, by project 3 9 10

Maximum, by project 12 40 10

Business impacts, as a direct result of ARC 

project assistance

Businesses that 

have developed 

new products

Businesses that 

have upgraded 

technologies or 

management 

methods

Projects reporting YES 18 13

Projects able to provide data on impacts 12 8

Percent of projects able to provide data 67% 62%

Reported number of businesses with impact 83 64

Median, by project 2.5 5.5

Minimum, by project 1 1

Maximum, by project 35 25
Evaluation of the ARC Entrepreneurship Initiative

81

Business sales impacts, as a direct result of 

ARC project assistance

Businesses that 

have increased 

sales outside of 

the state

Businesses that 

have increased 

export sales 

outside the United 

States

Projects reporting YES 12 3

Projects able to provide data on impacts 5 2

Percent of projects able to provide data 42% 67%

Reported number of businesses with impact 63 2

Median, by project 4 1

Minimum, by project 1 1

Maximum, by project 45 1

To what extent did the project result in Percent of projects

new partnerships with other public and 

private entities

Increased 

consultation 

or 

information 

exchange

Increased 

co-operation 

in tangible 

service 

delivery 

activities

No change in 

relationship

Colleges or universities 39.1 34.8 21.7

Non-profit economic dev. organizations 52.2 30.4 17.4

High schools or vocational schools 30.4 30.4 39.1

Community organizations 47.8 26.1 21.7

Small business development centers 26.1 26.1 47.8

Industrial extension or technology centers 21.7 26.1 26.1

Chambers /other local business groups 43.5 21.7 34.8

Private companies 39.1 21.7 30.4

Local or county government agencies 39.1 21.7 39.1

State agencies 56.5 17.4 26.1

Econ. dev, planning, policy consultants 39.1 17.4 39.1

Banks or other private lending institutions 39.1 17.4 43.5

Business consultants 30.4 13.0 47.8

Federal agencies 34.8 4.3 52.2

Multi-county or regional planning agencies 52.2 - 47.8

N=23. 
Evaluation of the ARC Entrepreneurship Initiative

82

Percent of projects 

reporting

For the following factors, please compare the position before the 

start of the ARC project with the position after the end of the 

project. Significant 

increase

No 

change

Host organization’s willingness to seek outside state and federal 

funding for entrepreneurial assistance projects after the completion of 

the ARC project

56.5 4.3

Entrepreneurship education and training opportunities available in the 

community after the completion of the ARC project 

52.2 8.7

Business assistance services available in the community after the 

completion of the ARC project 

47.8 21.7

Host organization’s willingness to invest its own resources in 

entrepreneurial assistance projects after the completion of the ARC 

project

47.8 8.7

Host organization’s willingness to seek local community funding for 

entrepreneurial assistance projects after the completion of the ARC 

project

43.5 8.7

Your knowledge of the capabilities of other organizations within the 

community who provide entrepreneurial training or business 

assistance

39.1 8.7

Your knowledge of the capabilities of other organizations outside your 

local community who provide entrepreneurial training or business 

assistance

34.8 91.3

Amount of funding available for new business start-ups after the 

completion of the ARC project

17.4 43.5

Number of business network groups in operation in the community 

after the completion of the ARC project

8.7 47.8

N=23.

To what extent did you face any of the following problems in 

implementing the ARC project

Number of projects 

reporting a 

problem

MAJOR MINOR

Attracting existing businesses to use your services 1 9

Monitoring client and project outcomes 5 5

Attracting new start-up entrepreneurs to use your services 1 7

Attracting competent staff to implement the program 2 3

Developing your own technical materials 1 4

Obtaining or using technical materials from other organizations 0 5

Obtaining support from other organizations in the community 0 5

Recruiting target youth or student population to participate in education 

and training programs

1 3

Securing exposure in local media to provide information about your 

program to others

0 4

Recruiting target adult population to participate in education and training 

programs

1 2

Obtaining support from your host organization 1 1

N=23. Projects reporting problem either as major or minor.
Evaluation of the ARC Entrepreneurship Initiative

83

How would you characterize the financial support provided by ARC Frequency Percent 

of 

projects

Provided a match to other funds that were available 7 30%

Led to the leveraging of other funds that would not have been available 16 70%

Other 1 4%

N=23. Multiple responses received in one case.

What would have happened if you had not received ARC funding? Frequency Percent 

of 

projects

Would have secured other funding so as not to delay the project 2 9%

Would have delayed the project to revise project plan 9 39%

Would not have continued with the project 11 48%

Other 1 4%

N=23.

How would you characterize the technical support provided by 

ARC

Frequency Percent 

of 

projects

ARC’s technical support was very useful 11 48%

ARC’s technical support was moderately useful 4 17%

ARC’s technical support was not useful 1 4%

ARC did not provide any technical support 7 30%

N=23.

Did the ARC provide opportunities for you to learn about the 

experience of similar projects elsewhere in the ARC region

Frequency Percent 

of 

projects

Yes 20 87%

No 3 13%

N=23.

Did the project have follow-on goals, which you aimed 

to undertake after ARC funding was completed?

Frequency Percent of 

projects

No 4 17%

Yes 18 78%

If yes, are you pursuing these goals? Percent of 

projects with 

follow-on goals

Yes, without change 7 39%

Yes, with some change 10 56%

No, not pursuing 1 6%

N=23.
Evaluation of the ARC Entrepreneurship Initiative

84

What are the sources of funding for follow-on project(s) Frequency Percent of 

projects

State funds 13 72%

Federal funds 8 44%

Fee income from clients 7 39%

Municipal or county government 7 39%

Host organization from its own resources 5 28%

Foundations 3 17%

Local business organizations 2 11%

Other 1 6%

N=18.
Evaluation of the ARC Entrepreneurship Initiative

85

Appendix C

Survey Instruments

ARC PROJECT SURVEY

Project name: Project reference number:

Your name: Telephone number:

A. PROGRAM DEVELOPMENT

1. What best describes your organization? o Local or county government agency

o Multi-county or regional planning 

agency

o Non-profit economic development 

organization

o Community organization

o High school or vocational school

o Community college or 2-year institution

o University

o Consultancy group

o Other (explain):

2. What is the approximate annual budget 

of your organization?

o Under $50,000

o $50,000 - $249,000

o $250,000 - $999,000

o $1 million - $10 million

o Over $10 million

o Don’t know

3. The project that ARC funded was: o A new program

o A program that built on earlier funded 

projects

o Other (explain):

4. If the program built on earlier funded 

projects, how were those projects 

sponsored?

o Previous ARC funds

o Other federal funds

o State funds

o Local or county gov’t funds

o Organization’s own resources

o Foundation

o Other (explain):

5. When did the ARC funded project 

begin?

Month: Year:

6. When did the ARC funded project end?

(If ongoing, indicate when the ARC 

project will end)

Month: Year:
Evaluation of the ARC Entrepreneurship Initiative

86

7. What were the major goals of this ARC project?

Goal Describe goal

#1

#2

#3

#4

#5

8. How were the goals established for the 

project derived?

o Formal planning study or needs 

assessment

o Discussion and experience within your 

organization

o Discussion involving other members of 

the local community

o Other

9. Did you change or add new goals during 

the period of the project? If yes, please 

describe

o Yes

o No
Evaluation of the ARC Entrepreneurship Initiative

87

B. PROGRAM ACTIVITIES

10. We would like to understand the 

principal activities undertaken by the 

ARC project. For each possible activity, 

please indicate whether it was a major or 

minor activity emphasis, or not an 

emphasis:

Major 

activity 

emphasis

Minor 

activity 

emphasis

Not an 

emphasis

Curriculum development o o o

Education and training of students (high 

school, vocational school) o o o

Education and training of adults o o o

Training of staff and other trainers o o o

Local and regional needs assessment o o o

Business opportunities research and 

marketing o o o

Technology opportunities research o o o

One-on-one assistance for existing

businesses o o o

One-on-one assistance for new start-up 

businesses o o o

Business network groups for existing

businesses o o o

Business network groups for new start-up

businesses o o o

Business seminars o o o

Business internships o o o

Referral systems to other service providers o o o

Government procurement assistance o o o

Loan or equity fund o o o

New physical facilities developed o o o

Distance learning o o o

Web site o o o

Custom software development o o o

Community conferences, organizational 

meetings o o o

New local economic development 

strategies o o o

Other (describe) o o o
Evaluation of the ARC Entrepreneurship Initiative

88

11. During the period of 

ARC project funding, 

how many participants 

did you serve by these 

activities?

Activity 

undertaken 

as part of 

the project?

If YES, how many participants by these 

categories:

Education and training of 

youth, high school 

students, or vocational 

students

o Yes

o No

_____ TOTAL NUMBER

Number of participants by the following 

categories:

_____ # under 8 hours 

_____ # 8 to 30 hours

_____ # more than 30 hours

o Unable to say

Education and training of 

adults

o Yes

o No

_____ TOTAL NUMBER

Number of participants by the following 

categories:

_____ # under 8 hours 

_____ # 8 to 30 hours

_____ # more than 30 hours

o Unable to say

Individual assistance to 

existing businesses that 

already have employees

o Yes

o No

_____ TOTAL NUMBER

Number of business assists by service time 

categories:

_____ # under 8 hours 

_____ # 8 to 30 hours

_____ # more than 30 hours

o Unable to say

Individual assistance to 

existing businesses that 

do not have employees 

(e.g. self-employed or 

micro-business)

o Yes

o No

_____ TOTAL NUMBER

Number of business assists by service time 

categories:

_____ # under 8 hours 

_____ # 8 to 30 hours

_____ # more than 30 hours

o Unable to say

Individual assistance to 

adults seeking to start 

new businesses

o Yes

o No

_____ TOTAL NUMBER

Number of business assists by service time 

categories:

_____ # under 8 hours 

_____ # 8 to 30 hours

_____ # more than 30 hours

o Unable to say

Assistance to network 

groups

o Yes

o No

_____ TOTAL NUMBER OF GROUPS

_____ TOTAL NUMBER OF BUSINESSES

Direct financial assistance, 

loans, or loan guarantees

o Yes

o No

_____ TOTAL NUMBER OF DIRECT LOANS 

OR OTHER FINANCIAL PACKAGES

Number of loans or financial packages:

_____ under $5,0000

_____ $5,000 - $25,000

_____ over $25,000

$_________Total value of all loans/packages

o Unable to say
Evaluation of the ARC Entrepreneurship Initiative

89

Referrals to other sources 

of assistance

o Yes

o No

_____ TOTAL NUMBER OF REFERRALS

Number by categories:

_____ Financial assistance

_____ Technological assistance

_____ Other business assistance

o Unable to say

Meetings with community 

leaders, business 

representatives, and 

other local organizations 

to discuss projects or 

plans 

o Yes

o No

_____ TOTAL NUMBER OF MEETINGS

_____ TOTAL NUMBER OF PEOPLE 

PARTICIPATING

(without duplication)

12. What was the primary geographical 

area served by the project?

o Single county

o Multi-county, within one state

o Entire state

o Multi-county, two or more states

C. PROJECT INPUTS

Total budget $

ARC $

Other federal 

sources

$

State gov’t $

County or 

local gov’t $

Fee income $

13. Please confirm the total budget for the 

project. Include direct cash receipts only.

Other $

14. What was the staffing level for the 

project at peak staffing? __________ Full-time equivalent staff

D. TRACKING AND ASSESSMENT

15. If you offered education and training courses 

as part of the ARC project, how did you track 

participant satisfaction and outcomes?

o Not applicable – skip to next question

o No formal evaluation used

o End-of-course student evaluations

o Time-elapsed student evaluations, at a 

time well after training had been completed

o Other (describe):

16. If you offered business assistance as 

part of the ARC project, how did you 

track participant satisfaction and 

outcomes?

o Not applicable – skip to next question

o No formal evaluation used

o End-of-project evaluations

o Time-elapsed project evaluations, at a 

time well after the project was completed

o Other (describe):
Evaluation of the ARC Entrepreneurship Initiative

90

E. PROJECT OUTCOMES

17. As a direct result of project assistance 

over the total period of the ARC project, 

have there been:

Number of 

businesses

Number of 

employees

• New businesses started by youth or 

students without employees

o Yes

o No

_____ #

o Unable to 

say

• New businesses started by youth or 

students with employees

o Yes

o No

_____ #

o Unable to 

say

_____ #

o Unable to 

say

• Number of new businesses started by 

adult participants without employees

o Yes

o No

_____ #

o Unable to 

say

• Number of new businesses started by 

adult participants with employees

o Yes

o No

_____ #

o Unable to 

say

_____ #

o Unable to 

say

18. As a direct result of project assistance 

through the ARC project, have there 

been:

• Jobs created in existing businesses o Yes

o No

Jobs created

_____ #

o Unable to 

say

• Jobs saved in existing businesses o Yes

o No

Jobs saved

_____ #

o Unable to 

say

• Jobs lost in existing businesses o Yes

o No

Jobs lost

_____ #

o Unable to 

say
Evaluation of the ARC Entrepreneurship Initiative

91

19. As a direct result of project assistance 

through the ARC project, have there 

been:

Number of 

businesses

• Businesses that have developed new 

products

o Yes

o No

• Businesses that have upgraded 

technologies or management methods

o Yes

o No

20. As a direct result of project assistance 

over the total period of the ARC project, 

have there been:

Number of 

businesses

Total added 

annual sales

• Businesses that have increased sales 

outside of the state

o Yes

o No

_____ #

o Unable to 

say

$_____

o Unable to 

say

• Businesses that have increased export 

sales outside the United States

o Yes

o No

_____ #

o Unable to 

say

$_____ 

o Unable to 

say

21. Has the project resulted in any 

remarkable “success stories” where 

participants achieved unexpectedly good 

results as a result of project assistance?

o Yes

o No

If yes, please briefly describe up to three of your success stories, including project’s 

intervention:

#1

#2

#3
Evaluation of the ARC Entrepreneurship Initiative

92

22. To what extent did the project result in 

new partnerships with other public and 

private entities?

Increased 

consultation 

or 

information 

exchange 

Increased 

cooperation 

in tangible 

service 

delivery 

activities

No change 

in 

relationship

Local or county government agencies o o o

Multi-county or regional planning agencies o o o

State agencies o o o

Federal agencies o o o

High schools or vocational schools o o o

Colleges or universities o o o

Non-profit economic development 

organizations o o o

Community organizations o o o

Chambers of commerce / other local 

business groups o o o

Small business development centers o o o

Banks or other private lending institutions o o o

Business consultants o o o

Economic development, planning, policy 

consultants o o o

Industrial extension or technology centers o o o

Private companies o o o

Other:______________________________ o o o
Evaluation of the ARC Entrepreneurship Initiative

93

23. For the following factors, please 

compare the position before the start of 

the ARC project with the position after

the end of the project.

Comparing before with after, to what 

extent has the project produced 

sustained changes in:

Significant 

increase

Moderate

increase

No 

change

Moderate

decrease

Significant 

decrease

Your organization’s willingness to invest 

its own resources in entrepreneurial 

assistance projects after the completion 

of the ARC project

o o o o o

Your organization’s willingness to seek 

local community funding for 

entrepreneurial assistance projects 

after the completion of the ARC project

o o o o o

Your organization’s willingness to seek 

outside state and federal funding for 

entrepreneurial assistance projects 

after the completion of the ARC project

o o o o o

Entrepreneurship education and training 

opportunities available in the 

community after the completion of the 

ARC project 

o o o o o

Business assistance services available in 

the community after the completion of 

the ARC project 

o o o o o

Number of business network groups 

operating in the community after the 

completion of the ARC project

o o o o o

Amount of funding available for new 

business start-ups after the completion 

of the ARC project

o o o o o

Your knowledge of the capabilities of 

other organizations within the 

community that provide entrepreneurial 

training or business assistance

o o o o o

Your knowledge of the capabilities of 

other organizations outside your local 

community that provide entrepreneurial 

training or business assistance

o o o o o
Evaluation of the ARC Entrepreneurship Initiative

94

F. PROBLEMS

24. To what extent did you face any of the 

following problems in implementing the 

ARC project?

Major

problem

Minor 

problem

Not a 

problem

Not 

applicable

Obtaining support from your parent 

organization o o o o

Obtaining support from other organizations 

in the community o o o o

Securing exposure in local media to 

provide information about your program 

to others

o o o o

Developing your own technical materials o o o o

Obtaining or using technical materials from 

other organizations o o o o

Recruiting target youth or student 

population to participate in education and 

training programs

o o o o

Recruiting target adult population to 

participate in education and training 

programs

o o o o

Attracting existing businesses to use your 

services o o o o

Attracting new entrepreneurs to use your 

services o o o o

Monitoring client and project outcomes o o o o

Attracting competent staff to implement the 

program

o o o o

Other problems (describe): o o o o

G. ARC ROLE AND FOLLOW-ON ACTIVITIES

25. How would you characterize the 

financial support provided by ARC?

o Provided a match to other funds that 

were available

o Led to the leveraging of other funds that 

would not otherwise have been 

available

o Other

26. What would have happened if you had 

not received ARC funding?

o Would have secured other funding so as 

not to delay the project

o Would have delayed the project to 

revise project plan

o Would not have continued with the 

project

o Other
Evaluation of the ARC Entrepreneurship Initiative

95

27. How would you characterize the 

technical support provided by ARC?

o ARC’s technical support was very useful

o ARC’s technical support was 

moderately useful

o ARC’s technical support was not useful

o Not Applicable – ARC did not provide 

technical support

28. Did the ARC provide opportunities for 

you to learn about the experiences of 

similar projects elsewhere in the ARC’s 

service region?

o Yes

o No

29. Did the project have follow-on goals, 

which you aimed to undertake after ARC 

funding was completed? If yes, please 

describe:

o Yes

o No

30. Are you now pursuing these follow-on 

goals?

o Yes, without change

o Yes, with some changes 

o No

31. What are the sources of funding for 

this follow-on project

o Federal funds

o State funds

o Local or county government

o Foundations

o Local business organizations

o Fee income from clients

o Your organization from its own 

resources

o Other:___________________________
Evaluation of the ARC Entrepreneurship Initiative

96

32. What have you learned from this project that will influence how you plan future 

entrepreneurship activities in your community?

33. Any additional comments?

34. If there are additional reports about the project, including outside assessments, or news 

media articles, we would like to see them. If possible, send such reports to: 

Regional Technology Strategies, Inc.

P.O. Box 9005

Chapel Hill, NC 27515

35. If your organization has an Internet address, what is the URL link? HTTP://

36. Finally, could you please provide contact information for the three success stories you 

mentioned earlier? We would like to follow-up.

#1 Name/organization Tel:

#2 Name/organization Tel:

#3 Name/organization Tel:

THANK YOU FOR YOUR PARTICIPATION! Should you have any questions, please give 

us a call at Regional Technology Strategies, at (919) 933-6699.
Evaluation of the ARC Entrepreneurship Initiative

97

ARC CLIENT SURVEY

ARC PROGRAM:

Name of person interviewed:

Telephone number:

Name of company (if applicable):

1. Status of interviewee ® Owner (of company)

® Employee

® Manager (of company owned by another)

® Not currently employed

® Other:

® Are you considering starting a new business?

2. Could you briefly describe your prior 

work experience or occupation?

3. How many employees work in this 

establishment?

4. What products or services does your 

company produce?

5. What percentage of your sales go to 

customers:

Local (under 50 

mi.)

Non-Local

To foreign 

countries

____%

____%

____%

6. Which best describes your main 

business objective?

® To operate a business that will grow 

rapidly in sales over the next three years

® To operate a business that is stable and 

supports my lifestyle and/or current 

employees

® To reverse the decline of my current 

business

® Other:

SERVICES PROVIDED

7. Over what period of time were you 

involved in this program?

8. What services did you receive from the 

program?

(Have client describe, and later fill in boxes)

® Management / technical assistance

® Access to capital

® Technology transfer

® Network creation / enhancement

® Entrepreneurial training / education
Evaluation of the ARC Entrepreneurship Initiative

98

® Other:

9. How did you first learn of the services 

offered by the program?

(Have client describe, and later fill in 

boxes)

® Contacted by program staff

® Referral from another program

® Flyer or advertisement

® Word of mouth

® E-mail or web site

® Other:

10. Why did you decide to use the services 

of this program rather than use another 

public or private provider?

(Have client describe, and later fill in 

boxes)

® Program offered higher quality services

® Program offered lower cost services

® Program offered services at the time I 

needed them

® No other local provider offers these 

services

® Other:

11. Can you give a rough estimate of how 

many interactions you had with the 

program’s staff during this period?

(An interaction includes face-to-face 

contact, telephone contact, or an 

exchange by e-mail)

® 1-2 interactions

® 3-5 interactions

® 6-10 interactions

® 11-20 interactions

® more than 20 interactions

12. About how much time did you and/or 

your staff spend interacting with the 

program staff?

________ hrs./month

13. How satisfied were you with the 

services provided by the program?

® Very satisfied

® Satisfied

® Neutral

® Dissatisfied

® Very dissatisfied

14. Did you receive the services you 

expected to receive? If no, please explain:

® Yes – exceeded my expectation

® Yes – met my expectation

® No

15. Did you pay for any of the services you 

received from the program?

® Yes Ë If Yes, how much: $ _________

® No

16. Do you expect to obtain additional 

services from the program over the 

next year?

® Yes

® No

® Maybe
Evaluation of the ARC Entrepreneurship Initiative

99

IMPACTS OF SERVICES

17. As a result of the services provided by 

the program, did you receive any of the 

following?

® Improved personal or business skills

® Specific recommendations for business 

improvement

® Assistance with new or improved 

technology

® Improved contacts with other 

businesses

® Better knowledge of local business 

resources and services

® A grant or loan from the program

® Referral to other source of financial aid

® Referral to other source of management 

assistance

® Other services received:

18. As a result of this program, have your 

attitudes and perspectives changed in 

any of these areas:

® Improved knowledge of opportunities for 

developing or expanding business

® Greater willingness to take risk in 

developing or expanding my business

® Improved confidence in local sources of 

business assistance

19. As a result of this program, what 

changes have occurred with you or 

your business?

(Have client describe, and later fill in 

boxes)

® New business plan

® Started a new business

® Invested in facilities, equipment, major 

software

® New production or management 

methods

® Developed or improved products

® Developed new services

® Entered new markets

® Other:

20. If you made investments as a result of 

the services provided by the program, 

please estimate the total amount. $ ____________________

21. If you have employees, did you make 

any of the following changes:

® Don’t have employees

® Improved human resource management

® New training programs established

® Improved quality control systems 
Evaluation of the ARC Entrepreneurship Initiative

100

PERSONAL INFORMATION

22. Please provide information about past and current sales and employment. In the 

last column, please estimate what the value would be now if you had not received 

services from the program. 

Estimate for current period if you 

had not received services. Current 12-

month period

Two years 

ago

If higher or lower, 

please estimate 

value

Total employees ® Higher

® Same

® Lower

® Can’t say

#

Total sales ® Higher

® Same

® Lower

® Can’t say

$

Total payroll ® Higher

® Same

® Lower

® Can’t say

$

Total cost of 

materials you 

bought to run your 

business

® Higher

® Same

® Lower

® Can’t say

$

23. Is there anything else you would like 

add?

CONTACT INFORMATION

24. Address

INTERVIEW INFORMATION

Date of interview

Interviewer
Evaluation of the ARC Entrepreneurship Initiative

101

Appendix D

Detailed Literature Review and Bibliography

1. Access to Capital

Programs that provide entrepreneurs with access to capital fall into two very different 

categories: seed/venture capital programs and microfinance programs. Seed and 

venture capital programs generally target businesses in fast-growing sectors with high 

capital needs. Seed or venture capital investments can be in the hundreds of 

thousands or millions of dollars, giving investors a share in the profits or ownership of 

the start-up firm. In contrast, microfinance programs focus on very small businesses or 

on self-employed individuals, and make much smaller investments—sometimes as 

small as hundreds of dollars. Microfinance can take the form of credit, loans, or grants, 

but rarely gives the lender an equity share in the new business. These two types of 

programs often have different missions; seed and venture capital is generally an 

economic development measure, whereas microfinance programs often serve social 

development as well as economic development goals.

In evaluating access to capital programs, some of the same issues arise for both types 

of program. For instance, both are subject to the debate over the use of 

accomplishment versus activity performance measures. However, the differences 

between the programs’ goals mean that each type of program requires its own 

evaluation approach and its own performance metrics.

a. Evaluating Seed and Venture Capital Programs

The metrics used to evaluate seed and venture capital programs are a mixture of 

activity and accomplishment measures. Internal activity measures include deal flow, 

debt/equity ratio, number of deals exited, leveraging of private funds, and internal rate of 

return. Meanwhile, accomplishment measures include tax revenue generated, success 

of funded businesses, and whether the target population or area was served. With the 

exception of tax revenue, the metrics mostly focus on micro-level program impacts 

rather than macroeconomic changes, highlighting the difficulty of detecting large-scale 

impacts even of high-investment programs.

The more significant characteristic of these metrics, however, is that (again with the 

exception of tax revenue) measurements used to evaluate publicly funded capital 

programs, which exist to promote industrial and economic growth, are the same 

performance measures that would be used to evaluate private venture capital 

companies that operate with the goals of private growth and profit. This choice of 

metrics is related to the most common finding in evaluations of public venture capital 

programs: that they perform best when they behave, not like public programs, but like 

private venture capital companies.

Different evaluations state this finding with varying degrees of vehemence. A USAID 

(Agency for International Development) evaluation of venture capital in developing 

countries found that focusing on public goals causes venture financing funds to fail, and 

that venture capital should be left to private companies. The National Governors’ 
Evaluation of the ARC Entrepreneurship Initiative

102

Association (NGA) stated that economic development and job creation are best served 

by a healthy business climate, not by programs that seek to benefit a targeted 

population or geographic area. While public leaders (and public funding) can launch 

venture financing programs, the NGA says fund management should be left to venture 

capital professionals who are “not afraid to make money.” The Rural Policy Research 

Institute supports this stance, noting that too little attention to internal rate of return and 

too much public involvement can hinder a capital fund’s ability to serve economic 

development goals. RUPRI’s analysis also states, however, that a public venture 

capital fund’s performance should be assessed according to its particular public 

mission, because a fund that focuses on economic development or job creation cannot 

be expected to maximize profit. The convergence of these findings suggests that the 

public mission orientation of some publicly funded venture capital programs may pose 

conflicts with a program’s need to compete for deals and profits.

b. Evaluating Microfinance Programs

As one would expect, the procedure for evaluating microfinance programs differs 

substantially from the above. The most common metrics used to evaluate such 

programs include participant income, rates of entry into self-employment, length of selfemployment spells, length of unemployment spells, the degree to which the program 

reaches its target population, and a variety of qualitative variables that fall under the 

category of “empowerment.” Like those used to assess the performance of venture 

capital programs, these metrics are largely micro-level rather than macro-level. As the 

name indicates, the relatively small scale of microfinance programs means that it is 

difficult to connect them with macroeconomic change.

The evaluation procedures and methodologies of microfinance programs have received 

a great deal of scholarly attention over the past few years, the most notable example 

being the Aspen Institute’s Performance Counts initiative. This project is working to 

establish a set of rigorously tested and applied output and outcome performance 

measures for microfinance programs. Its recommendations to date include the outcome 

measures listed above; the recommended output metrics include measures of program 

services performance, institutional capacity, efficiency, and sustainability. Other 

scholars have focused on the conflicts many programs encounter between economic 

and social goals. Lisa Servon of the University of Texas has conducted a number of 

studies showing that microfinance programs do not serve the neediest populations; the 

difficulty of self-employment, she argues, limits the economic development impact of 

microenterprise17. Mark Schreiner of Washington University’s Center for Social 

Development proposes that difficulties in reconciling social with economic goals be 

addressed by conducting cost-effectiveness evaluations, rather than simply 

performance evaluations. In this way, he argues, we can assess whether the costs 

associated with microenterprise programs’ different goals meet our sense of the goals’ 

importance18.

 

17 Servon, Lisa J. “Microenterprise Programs in U.S. Inner Cities: Economic Development or 

Social Welfare?” Economic Development Quarterly, May 1997 v11 n2 p166(15).

18 Schreiner, Mark. “A Review of Evaluations of Microenterprise Programs in the United States.” 

http://gwbweb.wustl.edu/Users/csd/workingpapers. March 1999.
Evaluation of the ARC Entrepreneurship Initiative

103

2. Networks

The field of network evaluation is still relatively young. Evaluators continue to debate 

appropriate methodologies, metrics, and performance indicators for networks. Two 

primary issues emerge from recent network evaluations. First, there is no uniform 

definition of a business network. This makes it difficult to establish evaluation 

measures, because different conceptions of the network point to different measures of 

success. Second—and perhaps more important—evaluators, network managers, and 

firms all want hard data relating to networks’ usefulness and success. Many of the most 

significant characteristics of networks, however, are qualitative (i.e., level of trust among 

firms, degree of knowledge growth), and it is difficult to examine them using quantitative 

methods. Some evaluators have gathered qualitative data and analyzed them using 

quantitative methods, but the data themselves are still qualitative.

Many evaluators have sought to construct a typology of networks prior to evaluating 

them, to understand the significant characteristics of networks and the firms within 

them, and to answer questions such as what motivates firms to join and stay within 

networks. Evaluation can then grow out of these typologies as evaluators determine 

whether network membership has fulfilled firms’ needs and expectations.

a. Evaluating Networks

As noted above, most evaluations of networks include qualitative characteristics. 

Among the most often used metrics are firms’ perceptions of the usefulness and impact 

of network membership, changing attitudes about inter-firm collaboration, firms’ level of 

trust and cooperation with other firms, level of commitment to the network, and the 

competitive environment of member firms. Evaluations focus on these characteristics 

because they are some of the most significant for determining a company’s willingness 

to participate in and commit to a network. Other qualitative characteristics include 

impact on supply chain relationships, firms’ assessment of services offered by the 

network sponsor, and firms’ opinions of the hindrance and success factors of business 

networks.

Evaluations also include quantitative characteristics, especially those evaluations that 

seek to construct a typology. Quantitative variables most commonly measured include 

differences in employment levels, revenues, and profitability; number of shared activities 

undertaken by member firms; and characteristics of members firms such as size, sales, 

and markets. There does not, however, appear to be a set of benchmarks determining 

a standard for the level of quantitative change that would mark success or failure of a 

network. Most evaluators, therefore, include qualitative as well as quantitative 

measures.

b. Commonly Used Methodologies

Because many of the metrics examined by evaluators are qualitative, many of the 

methods employed are qualitative as well. Most evaluations of networks gather 

information through questionnaires, key informant interviews, case studies, and review 

of program documents. Some of the evaluations combined qualitative and quantitative 

methods; one, for example, used program documents and interviews to gather 
Evaluation of the ARC Entrepreneurship Initiative

104

quantitative data as well as qualitative. Another asked executives of network member 

firms to estimate the dollar value of the benefits and costs of participating in the 

network; the resulting variable thus had both qualitative characteristics, in that it 

reflected individuals’ opinion, and quantitative characteristics, in that it could be 

analyzed numerically. A study undertaken by USNet and RTS examined several 

network case studies, codified the case study variables, and used qualitative statistical 

analysis to glean overall results from the individual case studies. 

c. Common Findings

One of the most common findings of network evaluations was that success depended 

upon the level of commitment from the individual firms. Outside sponsorship, such as 

funding for training or for the initial establishment of the network, was often helpful in 

getting networks off the ground, but public agencies cannot be the sole initiator, nor can 

they be the driving force behind the long-term functioning of the network. Most firms, it 

was found, had a positive assessment of the benefits of network membership, though 

many still had concerns about other firms' commitment and the risks posed by 

cooperating with potential competitors. Studies also found that where significant 

quantitative firm-level impacts were found, such as increases in revenues, they were 

likely to be concentrated among a few members firms rather than spread among all the 

members. More commonly found were qualitative firm-level impacts such as attitudes 

toward collaboration and policy awareness of networks. Macro-level economic impacts 

were difficult to detect.

Evaluators continue to develop new ways of studying networks, both to understand their 

functioning and to determine predictors of success. Particularly, they seek to reconcile 

the need for hard data with the significance of “soft” variables in network formation and 

growth.

3. Entrepreneurial Education

Entrepreneurial education provides skills that are not ordinarily taught in general 

curricula, while also giving students an opportunity to experience the practical effects of 

the skills being acquired. While there is a long tradition of business concepts being 

taught in higher education, entrepreneurial programs are now expanding into primary 

and secondary school as well. Studies of these programs demonstrate that 

entrepreneurship is a topic of interest by students of all ages.

Many studies of entrepreneurial education have highlighted specific components of 

successful programs, and to what extent such components can be replicated in various 

academic environments. Factors that are both successful and reproducible are of the 

greatest interest to an analysis such as this, and will receive especial attention in this 

section.

Entrepreneurial programs can be divided into two types: Primary/Secondary and Postsecondary. Each targets a different type of student, having different goals and 

capabilities. The different programs also exhibit different resources and commitments 

to the community that must be recognized.
Evaluation of the ARC Entrepreneurship Initiative

105

a. Primary/Secondary Programs

Many primary/secondary programs specifically target disadvantaged or “at risk” 

students, as an attempt to keep these youths in school and invested in education. 

Other entrepreneurial education courses are directed at “interested” students, or as a 

student activity, in order to expand students’ skills and post-school options.

Secondary programs rely upon the support of the host school system, and of outside 

partnerships, to ensure resources. Examples of resources utilized by secondary 

programs are:

• Outside businesses serving as mentors and as sources of apprenticeships or 

business experiences for students.

• Outside organizations and community groups becoming involved in a partnership 

with the schools.

ð Outside interests donating computers and technical resources. While perhaps not 

essential, these resources are important for successful training, and often cannot be 

provided solely by the school system.

b. Post-secondary Programs 

In contrast to the youth-oriented practices described above, post-secondary 

entrepreneurial programs target those involved in higher or continuing education. 

Often, these students have innovative ideas but require assistance in putting these 

ideas into practice. Studies reveal that entrepreneurial students, while requiring many 

of the same skills as traditional MBA business students, exhibit different behaviors and 

attitudes. Successful programs are those that cater to these behaviors and attitudes 

and can be distinguished from traditional business courses in the following ways:

• Courses contain ambiguous circumstances and risk, as with real-life 

entrepreneurial ventures. Stress and frustration are expected to develop during 

the course.

• Simulated ventures that can give students “real-life” references for later 

entrepreneurial experiences.

• Connections to successful alumni are important in giving programs prestige and 

credibility.

• Innovation and community involvement.

Two-year colleges have traditionally not provided entrepreneurial programs. However, 

this is an area receiving increasing interest, as many community and technical colleges 

have the capacity to deliver entrepreneurial programs as part of a continuing education 

program. The National Center for Research in Vocational Education conducted a 1996 

study entitled Fostering Entrepreneurship through Business Incubation; a handbook for 

two-year colleges was released as part of this study and contains information on how 

two-year colleges can become more fully invested in entrepreneurial programs. Among 

its recommendations are to:

1. Establish a business incubator
Evaluation of the ARC Entrepreneurship Initiative

106

2. Provide education and training opportunities

3. Support the local business community, and

4. Facilitate school-to-work-to-business ownership transition

Among four-year colleges, those that consistently rank highest in polls of business 

schools all offer at least three entrepreneurial courses as a requirement for the MBA. 

What stands out is that to be a top business school, there must be at least an 

acknowledgement of entrepreneurship, as different from traditional business classes. In 

addition, the faculty, curriculum and resources are large influences on how successful 

the program will be.

Entrepreneurial education can be imbedded in an educational program as a core 

curricular component. Doing so emphasizes the importance of entrepreneurial 

principles and ensures that students receive business training as a normal part of the 

educational process. This in turn could have a marked change on how students and 

adults view the risks and rewards of entrepreneurship.

Descriptions of entrepreneurship and the characteristics that contribute to 

entrepreneurial success vary. Jeffrey Timmons of Babson University has developed a 

useful definition that emphasizes the special set of skills needed by entrepreneurs:

Entrepreneurship is the ability to create and build something from practically 

nothing. It is initiating, doing, achieving and building an enterprise or 

organization, rather than just watching, analyzing, or describing one. It is the 

knack for sensing an opportunity where others see chaos, contradiction and 

confusion. It is the ability to build a “founding team” to complement your own 

skills and talents. It is the know-how to find, marshal and control resources and 

to make sure you don’t run out of money when you need it most. Finally, it is the 

willingness to take calculated risks, both personal and financial, and then do 

evertything possible to get the odds in your favor.

Entrepreneurial programs have been very successful in fostering the transition from 

idea to product or service. The continued diversification in program nature and student 

body will not only contribute to skills development but also to increased investment in 

fundamental local economies.

4. Assessing Technical and Managerial Assistance

The methodology for assessing performance of programs that provide managerial and 

technical assistance to entrepreneurs is a work in progress, but there is an emerging 

consensus on which indicators best measure achievement. Performance measurement 

starts with the program goals that define the desired outcomes. Entrepreneurial 

programs typically have multiple goals. For example:

• to help economically disadvantaged individuals start a business;

• to increase the number of jobs in micro-enterprises—especially well-paying jobs;

• to create/build wealth in the community;
Evaluation of the ARC Entrepreneurship Initiative

107

• to enhance the success of existing firms

A potential for conflict among these goals contributes to the difficulty in assessing 

program performance. It is important to recognize that in the short term, creating new 

businesses is not synonymous with creating jobs and wealth. Few new businesses –

even those that ultimately succeed – provide many jobs or generate large profits in their 

early years. Moreover, the different goals suggest different target populations. Are 

targeted clients potential entrepreneurs? Are there existing small businesses that could 

be helped to move to the next level? Is the program trying to serve both – perhaps with 

different service delivery mechanisms? 

The review of program benchmarks examined assessments of diverse programs 

providing management and technical assistance to entrepreneurs. These programs 

range from micro-enterprise programs such as those participating in the Aspen Institute 

Self-Employment Learning Project to the national network of Small Business 

Development Centers, a partnership between the Small Business Administration and 

local institutions of higher education.

Micro-enterprise programs target potential or existing entrepreneurs and focus 

performance measurement on indicators such as the number of new business start-ups, 

survival rates, and expansions. Small business assistance programs measure their 

success in the number of new jobs created and the wages paid new workers at assisted 

firms. The examples of performance indicators are not mutually exclusive; rather the 

difference is a matter of emphasis.

Both micro-enterprise and small business assistance programs consider customer 

satisfaction, which can be measured with a customer survey and/or indicated by a 

willingness to pay for services. All programs face resource constraints, and so 

performance indicators include efficiency measures. Funders and program 

administrators track not just the number of jobs created but also the cost per job 

created. 

Business incubators are often the vehicles through which assistance is delivered to 

start-up companies. An incubator can take the form of a multi-tenant building that 

provides affordable office space for new firms. Incubator firms typically share critical 

office services and equipment in order to reduce overhead costs. In addition to office 

services, incubator programs may provide affordable rent, managerial and technical 

assistance, financial assistance, and increased opportunities for interaction with firms 

both inside the incubator and within the larger local economy. 

The benefits resulting from management and technical assistance to small businesses 

are defined at three levels – the individual assisted, the firm (if there is one) assisted, 

and the community. While the program goal is to help the individual start and grow a 

business, that is not the only potential positive outcome. There is a benefit from 

someone learning that they lack an entrepreneurial aptitude, personality, or skills and/or 

have a weak business idea and thus avoid a debacle. For some, entrepreneurial 

training may convey entrepreneurial skills that help them obtain better jobs or advance 

in their current jobs. A comprehensive performance measurement system goes beyond 
Evaluation of the ARC Entrepreneurship Initiative

108

counting business start-ups. It tracks the economic situation of program clients (or a 

sample of clients) for a period after they have received services.

Desired outcomes at the firm level include increased profitability and growth in size as 

indicated by sales and employment. Desired outcomes at the community level also are 

measured in job growth. Additional community-level benefits might be a more diverse 

economic base, greater wealth, and more examples of good corporate citizenship. The 

North Central Regional Center for Rural Development used the time and money that 

assisted firms donated to non-profit activities in the community as a performance 

indicator in its assessment of community benefits from business incubators.

The key for any effort is that the selected performance indicators (1) reflect program 

goals and (2) measure changes in the status of clients. For example, the Office of 

Minority Enterprise is charged with helping minority firms grow and one strategy is 

accessing government contracts. Among performance indicators for that program are 

firm revenues and the amount of federal procurement dollars awarded to client minorityowned firms.

Another characteristic of good programs is performance indicators that are credible. 

This means that they use data that are readily available, easily understood, and can be 

verified. The minority business program requires clients to submit their year-end 

financial statements, which provide the data needed for its performance indicators. This 

illustrates an important precept. Programs should build in procedures to collect data for 

performance measurement from the beginning.

It is not always easy to link program activities with the desired outcomes described in 

goal statements. A tiered performance measurement system helps make the 

connection. Typically, the lowest level of performance indicator counts program 

activities. The second level consists of intermediate outcomes, changes in behavior or 

status that lead to the third level, the desired outcomes. For programs providing 

management and technical assistance to entrepreneurs, the number of clients assisted 

is an activity count; the number of assisted clients who complete a plausible business 

plan and/or access capital is an intermediate outcome. Final outcomes include 

measures such as the number and percent of assisted clients who improve their 

economic status and documented increases in employment and profitability at assisted 

firms.

Program managers are usually most comfortable measuring performance with activity 

counts, which are the indicators most susceptible to the managers’ control. However, 

activity counts are not sufficient to measure program performance, and should be used 

in concert with the higher level indicators.

5. Technology Transfer

Technology transfer is in practice a fairly broad concept that covers a wide variety of 

efforts designed to aid the innovation process in firms. Since most basic research is 

conducted in university settings, many technology transfer programs are intended to 

improve the linkages between universities and industry. Such initiatives may consist of 
Evaluation of the ARC Entrepreneurship Initiative

109

formal, long-term collaborations between universities and firms like research consortia 

or joint research ventures. Or they might take the form of research centers or industrial 

extension/modernization programs where technical assistance or short-term research is 

provided to firms on an as needed basis. Technology transfer also includes efforts to 

speed up the patenting and commercialization of both university and industrial research.

For the purpose of evaluating the ARC Entrepreneurship Initiative, the research team 

primarily concentrated on previous assessments of industrial modernization and 

extension initiatives. The major national program in this area is the Manufacturing 

Extension Partnership (MEP), managed by the National Institute of Standards and 

Technology (NIST) in collaboration with the states. The MEP provides technology and 

business assistance services, targeted towards small and medium-sized manufacturers. 

National surveys sponsored by NIST emphasize a series of quantitative measures of 

program performance, including sales, cost savings, jobs, capital investment, and 

productivity improvements. Since many firms have difficulty in placing an absolute 

dollar value on program impacts, one recent change has been to ask assisted firms 

whether particular impacts are present or not. If yes, the customer is then asked to 

estimate impacts. In one local survey, a series of impact ranges is offered, although the 

national survey continues to probe for a single dollar figure.

In addition to the post-project surveys, the MEP and local programs have supported a 

series of other evaluation studies. Logic-based case studies have been conducted to 

better understand high impact projects and the transformation of firms through MEP 

intervention. Controlled studies have been undertaken which compare similar assisted 

and non-assisted firms. NIST has also supported several state-of-the art workshops on 

the evaluation of industrial modernization and organizes an evaluation working-group to 

bring together those working on evaluation. Finally, there are regular external reviews 

of program performance by outside reviewers. NIST has established a performance 

review system based on the Baldridge quality criteria. Centers are reviewed under 

seven groups of criteria: 1) leadership, 2) planning, 3) customer knowledge and 

relationships, 4) performance information and analysis, 5) internal workforce practices 

and workforce environment, 6) process management, and 7) performance results.

Individual centers have established their own procedures to evaluate customer 

satisfaction and program impact. The methods used vary greatly in terms of 

sophistication, metrics, and robustness. Some centers do very little, leaning instead on 

NIST’s national evaluations. A few centers sponsor extensive efforts. For example, the 

Georgia MEP has a distinct evaluation element that has conducted a series of 

evaluative studies including controlled surveys, cost-benefit analyses, and other special 

studies. The Michigan MEP has used a comprehensive benchmarking protocol, known 

as the Performance Benchmarking Service, to undertake controlled studies of program 

impact.
Evaluation of the ARC Entrepreneurship Initiative

110

Access to Capital

C-1 U.S. General Accounting Office. CDFI Fund Can Improve Its Systems to Measure, 

Monitor, and Evaluate Awardees’ Performance. Washington, D.C.: General 

Accounting Office, GAO/RCED-98-225, July 1998.

C-2 Barkley, David, Deborah Markley, and Julia Sass Rubin. Public Involvement in 

Venture Capital Funds: Lessons from Three Program Alternatives. Columbia, Mo.: 

Rural Policy Research Institute, November, 1999. 

<www.rupri.org.pubs/archive/reports/p99-9/index.html>.

C-3 Heard, Robert and John Sibert. Growing New Business with Seed and Venture 

Capital: State Experiences and Options. Washington, D.C.: National Governors’ 

Association Center for Best Practices, 2000.

C-4 Fox, James W. The Venture Capital Mirage: Assessing USAID Experience with 

Equity Investment. USAID Program and Operations Assessment Report No. 17. 

Washington, D.C.: U.S. Agency for International Development Center for 

Development Information and Evaluation. 1996.

C-5 Schreiner, Mark. Lessons for Microenterprise Programs from the Unemployment 

Insurance Self-Employment Demonstration. St. Louis, Mo.: Washington University 

in St. Louis Center for Social Development, 1999. 

<http://gwbweb.wustl.edu/Users/csd/workingpapers/workingpapers.html>.

C-6 Barnes, Carolyn, Gayle Morris and Gary Gaile. An Assessment of the Impact of 

Microfinance Services in Uganda: Baseline Findings. Washington, D.C.: 

Management Systems International, 1998. 

<www.mip.org/componen/AIMS/Pubs/English/Impact1.htm>.

C-7 Postlethwaite & Netterville. CAPCO Study. Baton Rouge, La.: Louisiana 

Department of Economic Development, 1999.

C-8 Doyle, Karen. Performance Measures for Microenterprise in the U.S. Draft paper 

presented at the International Labour Office program “Enterprise Creation by the 

Unemployed: The Role of Microfinance in Industrialized Countries,” November, 

1999.

C-9 Microcredit in Transitional Economies. Paris: Organization for Economic Cooperation and Development, 1996.

Entrepreneurial Education

E-1 Callan, K. and M. Warshaw. “Choosing the Best,” in Success Sept. 1996, pg. 40.

E-2 Vesper, K. and W.B. Gartner. “Measuring Progress in Entrepreneurship 

Education,” in Journal of Business Venturing 12: 403-421, 1997.
Evaluation of the ARC Entrepreneurship Initiative

111

E-3 Sexton, D.L. and N.B. Upton. “Evaluation of an Innovative Approach to Teaching 

Entrepreneurship,” in Journal of Small Business Management, Jan. 1987: 35-43.

E-4 Clouse, V.G.H. “A Controlled Experiment Relating Entrepreneurial Education to 

Students’ Start-up Decisions,” in Journal of Small Business Management, April 

1990; 45-53.

E-5-7 Hernandez-Gantes, V.M. Fostering Entrepreneurship through Business 

Incubation: A 3-part Series. Berkeley, Calif.: National Center for Research in 

Vocational Education (NCRVE), 1996.

Other References:

Benson, G.L. “Teaching Entrepreneurship through the Classics,” in Journal of 

Applied Business Research 8 (4): 135-144, 1992.

Emshwiller, J.R. “Students Take Entrepreneurship Out of the Classroom: Many 

Small Business Schools Add Programs Offering Hands-on Experience.” The Wall 

Street Journal 11/1/90, B2.

Fleming, P. “Entrepreneurship education in Ireland: A longitudinal study,” in The 

Academy of Entrepreneurship Journal, 1996.

Gorman, G., D. Hanlon, and W. King. “Some Research Perspectives on 

Entrepreneurship Education, Enterprise Education and Education for Small 

Business Management: A Ten-year Literature Review. International Small 

Business Journal 15 (3): 56-77, 1997.

Management and Technical Assistance

M-1 U.S. Small Business Administration. A Report to the U.S. Congress on Minority 

Small Business and Capital Ownership Development for Fiscal Year 1996. 

Washington, D.C.: U.S. Small Business Administration Office of Minority Enterprise 

Development, 1997.

M-2 U.S. Small Business Administration. Opportunities for Success, Small Business 

Administration FY99 Performance Report. Washington, D.C.: U.S. Small Business 

Administration, 2000.

M-3 Johnson, S.L. and Holly M. Mudd. Performance Measurement in the SBDC 

Program, Inspection Report No. 98-09-01. Washington, D.C.: U.S. Small Business 

Administration Office of Inspector General, 1998.

M-4 Field Forum, Issue 1. The Aspen Institute, Washington, D.C. October, 1999. 

NOTE: Additional information from telephone interview with Elaine Edgecomb, 

Aspen Institute consultant.

M-5 “Funding for the Future: The Young Entrepreneur Program.” Association for 

Enterprise Opportunity Newsletter, September/October 1999. Arlington, Va.: 

Association for Enterprise Opportunity.
Evaluation of the ARC Entrepreneurship Initiative

112

NOTE: PYEP Evaluation Outcomes outline, QED Group, and other information 

provided by Jim Weglarz, AEO.

M-6 “Measuring the Success of Economic Development Tools.” Rural Development 

News, Fall 1998. Ames, Iowa: North Central Regional Center for Rural 

Development.

Networks

N-1 Shapira, Philip, Mary Ann Alabanza Akers, and Eric Oldsman. Collaborative 

Networking Projects Evaluation Study: An Evaluation of Flexible Manufacturing 

Networking Projects. Final Report sponsored by the Appalachian Center for 

Economic Networks and the University of Toledo Urban Affairs Center. Chicago: 

Joyce Foundation, 1996.

N-2 Kingsley, Gordon. Establishing the Building Blocks: A Follow-up Evaluation of 

USNet’s Foundation Forum on Inter-Firm Collaboration. USNet Evaluation Working 

Paper No. 9504. Chapel Hill, N.C.: Regional Technology Strategies, Inc., 1995.

N-3 Kingsley, Gordon and Hans Klein. Lessons Learned from Inter-Firm 

Collaborations: A Survey of Case Studies. USNet Evaluation Working Paper No. 

9703. Gordon Kingsley and Hans Klein. Chapel Hill, N.C.: Regional Technology 

Strategies, Inc., 1997.

N-4 Welch, Doug, Eric Oldsman, Philip Shapira, Jan Youtie, Julie Lee. Net Benefits: 

An Assessment of a Set of Manufacturing Business Networks and Their Impacts 

on Member Companies. Chapel Hill, N.C.: Regional Technology Strategies, Inc., 

1997.

N-5 Fulop, Liz, and Jo Kelly. Survey of Industry Network Initiatives in New South 

Wales. Final Report prepared for the Strengthening Local Economic Capacity

Project, an initiative of the Commonwealth Department of Housing and 

Development. Kingswood, N.S.W., Australia: University of Western Sydney 

Nepean, 1995.

Technology Transfer

T-1 Ellis, S. “Evaluation of Massachusetts Manufacturing Partnership: Selected 

Findings,” in Manufacturing Modernization: Implications of Evaluation Results for 

Program Improvement and Policy Development: Proceedings of Fourth Workshop 

on the Evaluation of Industrial Modernization Programs (P. Shapira and J. Youtie, 

eds.). Atlanta: Georgia Institute of Technology, 1998.

T-2 Glasmeier, A., K. Fuellhart, I. Feller, and M. Mark. “The Relevance of FirmLearning Theories to the Design and Evaluation of Manufacturing Modernization 

Programs,” Economic Development Quarterly 12 (2): 107-124, 1998.
Evaluation of the ARC Entrepreneurship Initiative

113

T-3 Jarmin, R. “Evaluating the Impact of Manufacturing Extension on Productivity 

Growth,” Journal of Policy Analysis and Management 18 (1): 99-119, 1999.

T-4 Luria, D. “Toward Lean or Rich? What Performance Benchmarking Tells Us About 

SME Performance, and Some Implications for Extension Center Services and 

Mission,” Journal of Technology Transfer 23 (1): Spring 1997.

T-5 Mt. Auburn Associates. Evaluation of New York State’s Technical Assistance and 

Financing Programs. Somerville, Mass.: Mt. Auburn Associates, 1997.

T-6 Shapira, P. and J. Youtie. “Evaluation Industrial Modernization: Methods, Results, 

and Insights from the Georgia Manufacturing Extension Alliance," in The Journal of 

Technology Transfer 23 (1): 17-28, Spring 1998.