Innovative Metrics for Economic Development 1

March 1, 2016

November 17, 2017 

This report was developed through a cooperative agreement 

between EDA and SRI International

Innovative Metrics for Economic 

Development: Final Report
Innovative Metrics for Economic Development 2

Acknowledgements

This project was funded by a grant from the U.S. Department of Commerce, Economic Development Administration 

(EDA). Special thanks to EDA headquarters staff that contributed to the project:

• Ksenia Shadrina

• Bryan Borlik

• Ryan Smith

• Eric Smith 

The Center for Innovation Strategy and Policy at SRI International conducted this research. Report authors and key 

contributors include Roland Stephen, Laura Ross, and Nikhil Kalathil. 

The findings and observations contained in this report are those of the authors and do not necessarily reflect the 

views of any particular interviewee or of the Economic Development Administration in general.
Innovative Metrics for Economic Development 3

Table of Contents

Table of Tables.....................................................................................................................................................................4

Table of Figures...................................................................................................................................................................5

Project Highlights ................................................................................................................................................. 6

Executive Summary.............................................................................................................................................. 7

Project Overview................................................................................................................................................... 8

Logic Model ...........................................................................................................................................................10

Establishing the Logic Model for Evaluation of Non-Infrastructure Economic Development Programs.. 10

Using an Econometric Model in Conjunction with the Logic Model.......................................................................... 12

Econometric Model.............................................................................................................................................14

Methodological Approach.............................................................................................................................................14

Explaining Long-Term Economic Performance................................................................................................................. 14

Research on Long-Term Economic Performance ............................................................................................................. 15

Model Design.................................................................................................................................................................................... 16

Variables ............................................................................................................................................................................. 17

Control Variables............................................................................................................................................................................ 17

Capacity Variables.......................................................................................................................................................................... 18

Data Limitations.............................................................................................................................................................................. 22

Capacity Variable Analysis and Results...................................................................................................................23

Detailed Discussion of Results .................................................................................................................................................. 24

Summary of Results from Capacity Analysis..........................................................................................................25

Differentiating Capacity Variables: Economic Drivers and Networking Capacity .............................................. 29

Summary of Networking Capacity Analysis ...........................................................................................................31

Program Level Impacts: Collecting and Assessing Data........................................................................34

Pilot Survey and Results................................................................................................................................................36

Conclusions ...........................................................................................................................................................38

Appendix A: Summary Statistics....................................................................................................................40

Appendix B: Robustness Checks ....................................................................................................................43

Appendix C: Selected Literature ....................................................................................................................50

Literature for Capacity Outcomes..............................................................................................................................50

Literature for Realized Outcomes..............................................................................................................................51

Appendix D: Survey Results ............................................................................................................................52

Appendix E: Cited References.........................................................................................................................59

Appendix F: Survey Challenges and Lessons Learned for EDA...........................................................61

Survey Fatigue ..................................................................................................................................................................61

Difficulty Reaching Clients ...........................................................................................................................................61
Innovative Metrics for Economic Development 4

Table of Tables

Background...........................................................................................................................................................10

Research on Capacity Outcomes....................................................................................................................11

Table 1: Normalized variable abbreviations and descriptions ..........................................................23

Table 2: Identified relationships, combined metropolitan and micropolitan regions ..............26

Table 3: Identified relationships, metropolitan regions only.............................................................26

Table 4: Identified relationships, micropolitan regions only .............................................................27

Table 5: Capacity indicators on outcome variables................................................................................28

Table 6: Capacity indicators on outcome variables with networking capacity analysis...........33

Table 7: Summary of survey recipients and respondents....................................................................36

Table 8: Selected summary statistics from grantee program outputs survey ..............................37

Table 9: Average client rating of improvement due to grantee support and services...............37

Table 10: Selected summary statistics of client capacity outcomes .................................................37

Table A- 1: All regions .......................................................................................................................................40

Table A- 2: Metropolitan regions...................................................................................................................41

Table A- 3: Micropolitan regions ...................................................................................................................42

Table B- 1: Correlation matrix of normalized capacity variables......................................................43

Table B- 2: Correlation matrix of non-normalized capacity variables.............................................44

Table B- 3: Non-normalized capacity indicators on outcome variables..........................................45

Table B- 4: Networking capacity and capacity variable interaction on long-term earnings 

growth tests ..........................................................................................................................................................47

Table B- 5: Capacity variables on earnings growth with different definitions of the 

networking variable ..........................................................................................................................................49

Table D- 1: Summarized results from the i6 grantee program outputs survey............................52

Table D- 2: Summarized results from the i6 client capacity outcomes survey.............................54

Table D- 3: Average amount of funding obtained by i6 clients, by source......................................55

Table D- 4: Summarized results from the SFS grantee program outputs survey.........................56

Table D- 5: Average amount of funding obtained by SFS grantee clients, by source ..................57

Table D- 6: Summary of combined SFS and i6 program output survey responses......................58
Innovative Metrics for Economic Development 5

Table of Figures

Figure 1: Logic model for non-infrastructure economic development programs .......................12

Figure 2: Econometric study for non-infrastructure programs .........................................................13

Figure 3: Capacity variables ............................................................................................................................19

Figure 4: Project components.........................................................................................................................34

Figure 5: Economic development program logic model .......................................................................35

Figure 6: Overview of survey flow.................................................................................................................36

Figure 7: Dual-level approach to assessing impacts...............................................................................38
Innovative Metrics for Economic Development 6

Grantee and Client Survey

SRI piloted the survey instrument to collect data from EDA’s Regional 

Innovation Strategies (RIS) grantees and their clients from 2014 and 2015: 

• Respondent grantees in the RIS program served over 3,600 clients 

through their EDA grants.

• 56 respondent clients reported increasing the TRL of a product or 

products through services supported by EDA grants, with an 

average TRL increase of 3.39.

• 88 respondent clients accessed venture capital, seed, and/or 

angel funding through the services supported by the EDA grant.

• The results from the pilot, albeit limited in sample size, indicate a 

link between EDA non-infrastructure programs and development 

of capacities integral to long term economic outcomes. 

.hmok

Study Overview

SRI International’s Center for Innovation 

Strategy and Policy conducted this study in 

coordination with EDA. The project consists of:

• The development of metrics to assess 

impact of economic development 

programs on capacity building and long 

term economic performance as part of a 

new program evaluation system. A 

corresponding toolkit for implementation 

of the system, including a survey 

instrument, has also been developed.

• An econometric model to demonstrate the 

impact of non-infrastructure grant 

programs on regional economic capacities, 

and the relationship between those 

capacities and long-term realized outcomes 

such as job creation and per capita or 

earnings growth. 

• A pilot of the survey instrument to directly 

collect data from a subset of EDA’s noninfrastructure grantees.

While the project is modeled around EDA 

programs and uses EDA-specific data in its 

analyses, the work is intended to inform a 

broader body of work on program evaluation in 

economic development. 

As a whole, the project suggests that noninfrastructure grant programs, such as those 

administered by EDA, build economic 

development capacities and estimates the 

degree to which these capacities have an 

impact on long-term economic performance 

such as employment and income. The study 

also reveals differences in impact of

economic development interventions 

depending on whether they occur in 

metropolitan or micropolitan areas in the 

United States. 

Capacity Variables and Economic Performance

Through regression analysis, SRI found the following relationships:

• Increases in financing events are correlated with higher rates of 

both earnings and employment growth.

• Larger numbers of membership associations and organizations 

and a higher creative class share of the workforce are correlated 

with higher rates of earnings growth.

• Higher rates of establishment churn are associated with higher 

employment growth, but reduced levels of earnings growth.

• Tech intensive non-employer firms (or small enterprises) tend to 

contribute to employment growth, as well as per capita income 

growth.

EDA Grants as a Facilitator of Networking Capacity

After creating a combined variable that captures the interaction of regional 

organizations and EDA activity, SRI found the following through additional 

regression analysis at a regional level:

• EDA activity and organizational density impact economic 

performance for micropolitan regions.

• Networking capacity is important in micropolitan regions, and 

relies heavily on the presence of EDA grants. This supports the 

hypothesis that institutional capacity and networks in 

micropolitan regions is an important indicator of future economic 

performance.

Project Highlights
Innovative Metrics for Economic Development 7

Executive Summary

This project represents a major step forward in the assessment of non-infrastructure economic development 

initiatives towards a broad, generalizable approach to data collection and analysis. The assessment is grounded in 

a comprehensive logic model that captures the progression of a grant program from inputs, to activities, to both 

short-term and long-term outcomes. As a definitional matter, in this report and analysis, when we refer to 

performance measurement or program evaluation, we refer to monitoring the performance of or evaluation of 

economic development programs, respectively.

To monitor short-term impacts at the program level, SRI developed flexible survey instruments for data collection 

from both, grantee organizations and their clients, and piloted the instruments using a set of EDA grantees from 

2014-2015.1 The results of this pilot were used to both assess the impact of EDA programs on grantee capacities, 

as well as to refine and improve the survey instruments themselves. Ultimately, these instruments aim to help 

economic development practitioners to monitor the activities performed by grantees as well as to evaluate the 

impact of these activities on the building or improving of capacities that businesses and communities need to 

succeed. 

To evaluate long-term impacts of economic development programs at the regional level, SRI developed an 

econometric model based on measures of economic development capacity. SRI identified a set of measures at the 

level of metropolitan or micropolitan geographic regions that serve as reliable proxies for the aggregate capacities 

developed through economic development programs. SRI’s regression analysis based on those measures captures 

the relationships between economic development grant activity and long-term economic growth and development. 

SRI conducted quantitative analysis to ascertain the validity of the proposed metrics and the logic model through 

two econometric methodologies. The first was a regression of capacity measures on long-term economic 

performance with appropriate controls. In the second model, SRI separated out three variables that represent a 

region’s networking capacity to support economic development. In this study, networking capacity is calculated by 

interaction of the number of economic development organizations, the number of EDA non-infrastructure grants 

received, and the number of membership associations and organizations in a region. 

The results from SRI’s regression analysis elucidate the pathways through which non-infrastructure economic 

development programs indirectly influence long-term economic performance. One intriguing result is an observed 

difference between metropolitan and micropolitan regions when it comes to the importance of networking 

capacity. Stronger networking capacity appears to have a greater impact on smaller regions with limited 

institutional capacity. An understanding of these differences can be used to guide economic development 

investment and policy decisions to ensure a strategic place-based implementation conducive to a more inclusive 

economic development and growth process. 

 

1 Specifically, SRI piloted the survey instruments among the 2014 and 2015 recipients of the i6 Challenge program and the Seed Fund 

Support program. 
Innovative Metrics for Economic Development 8

Project Overview

The U.S. Department of Commerce’s Economic Development Administration (EDA) is the only federal government 

agency focused exclusively on economic development. Economic development:

“Creates the conditions for economic growth and improved quality of life by expanding the capacity of

individuals, firms, and communities to maximize the use of their talents and skills to support innovation, 

lower transaction costs, and responsibly produce and trade valuable goods and services."

A key component of this economic development capacity is effective, collaborative institutions focused on

advancing mutual gain for the public and private sectors.

EDA has a broad portfolio of non-infrastructure grant programs that support start-ups, small firm development, 

technology development, regional innovation ecosystems, workforce development, planning activities, and 

research. These economic development and innovation-focused programs yield improved capabilities for 

participants, which include entrepreneurs, small businesses, and regional organizations. These programs also have 

long-term, broader economic benefits, such as increased earnings and job growth. These benefits can be difficult 

to capture in the short term or as part of a single assessment. Yet, in difficult economic times and in an environment 

of increasing support for evidence-based policymaking, it is important to develop tools that allow policymakers to 

forecast the future economic benefits of programs and policies.

Economic development programs are difficult to evaluate due to the temporal and analytical distance between 

focused program activities and desired long-term economic performance. An additional challenge is the 

identification of valid indicators for complex programs that fuel economic development capacity-building. While 

standard approaches to program evaluation focus on long-term economic indicators such as company or job 

creation, this project demonstrates connections between program activities and shorter-term or capacity outcomes 

which, as supported by a broad body of research as well as this study, are integral to longer-term desirable economic 

outcomes.

SRI International was awarded a grant by EDA to formulate an approach that would capture the impact of noninfrastructure economic development investments on long-term realized outcomes. The approach adopted 

elaborates a carefully refined logic model, suitable for general application to any similar program. The model is 

grounded in the deep experience of the project team, in the relevant literature, and in the counsel and guidance of 

EDA staff. The model defines categories of economic development capacity targeted by such programs. These 

categories map to capacities and are best characterized as short-term effects or outcomes. While, as part of this 

project, SRI proposes that capacity outcomes can and should be measured at the program level through direct 

collection of data, these outcomes should also be ascertained in terms of a regional impact where the program is 

administered. Therefore, in order to estimate the long-term impact of these capacities, the project team established 

a set of indicators as proxies for the categories of capacities. These indicators, although refined throughout the 

analysis, originate in the earlier research conducted by SRI in collaboration with other distinguished institutions 

(UNC Chapel Hill, GWU, etc).2 The team compiled a dataset containing these indicators for all metropolitan and 

micropolitan regions of the United States. In addition to the capacity measures, the dataset includes outcome 

 2 The reports detailing this work can be found on: https://www.eda.gov/tools/research-reports/.
Innovative Metrics for Economic Development 9

indicators (earnings, employment, and personal income) and standard social and economic control variables. The 

team then employed a variety of econometric analyses designed to sift the relationships between capacities and 

long-term economic performance. The findings are reported below.

In summary, this project developed a logic model for capturing the impact of economic development programs on 

economic development capacity, and then tested aggregate measures of those capacities against long-term 

economic performance. This project also developed a comprehensive toolkit for the direct collection of program 

data. The operational toolkit comprises a survey instrument and a protocol for the use of economic development 

practitioners. The toolkit and all accompanying instruments have been carefully designed to meet the needs of any 

non-infrastructure economic development program. The instrument was piloted among recipients of grants from 

the 2014 and 2015 Regional Innovation Strategies competitions. The results reported in Section III of this report

indicate in a straightforward but important way that the program is generating the outputs expected of it. 
Innovative Metrics for Economic Development 10

Logic Model

Establishing the Logic Model for Evaluation of Non-Infrastructure Economic Development Programs

For most economic development programs, it is challenging to establish direct, causal relationships between 

program activities and long-term outcomes. Logic models help to address this challenge by establishing theoretical 

“if-then” relationships across a program’s activities, outputs, and immediate outcomes. As such, they are useful 

tools for structuring program evaluations, helping account for immediate, measurable impacts that stem from a 

program’s activities, while logically relating these outputs to longer-term quantitative outcomes.

To review, the logic model captures the logical progression of a grant program, from inputs to activities to 

immediate outcomes. Within each component of the logic model, different categories of inputs, outputs, and 

outcomes might occur depending on the program’s focus. For example, a program’s “outputs” or activities might 

focus on events, networking, and referrals; or some other area(s) as illustrated in the logic model presented in 

Figure 1 below.

The specific components of the logic model are defined as follows:

• Initial Conditions & Capacities: The “baseline” or underlying factors and externalities that affect the program 

and its participants, and that may shape the types and magnitude of outcomes achieved.

• Inputs: The investments in the program, which may include financial resources (e.g., grants), as well as 

intangible resources such as staff expertise, space, or other factors. 

• Outputs (Program Activities): A measurement of the quantity and quality of the activities performed by the 

program grantees, partners, and clients or participants. 

• Capacity Outcomes: The direct, shorter-term outcomes that occur as a result of the program that enhance the 

capabilities of the program’s clients or participants.

• Realized Outcomes: The broader long-term impacts of a program, including at a regional level, which may 

include measurable growth in terms of jobs or other quantifiable economic indicators. 

The logic model as illustrated below grounded in several years of cooperative research by SRI, EDA, and other 

institutions. This work both, furthered SRI’s understanding of EDA’s programs and helped define a general 

framework for economic development initiatives, more broadly. The components of the logic model, as delineated 

in this research, draw from a vast body of work on economic development theory and praxis. The literature on the 

inputs, activities, and outputs of programs that build capacity, and on the impact of those capacities on long-term 

outcomes such as earnings and employment is quite expansive.

Background

There is an expansive body of work on program evaluation that focuses on program inputs, outputs, and near-term 

capacity outcomes. This literature tends to be focused on individual and program level activities, as exemplified by 

assessments of incubator and accelerators as well as broader programs targeting innovation through investment in 

institutions and networks. While this literature often seeks to identify longer-term impacts, these prove hard to 

estimate due to limitations of the data and underlying attribution problems. The data in these studies are almost 

always directly collected or self-reported and rely heavily on survey results.
Innovative Metrics for Economic Development 11

There is a separate (if logically connected) literature on economic development that focuses on the relationships 

between near-term capacity outcomes and long-term realized outcomes. This literature is focused on the sector or 

regional level, and is exemplified by econometric studies that suggest the importance of key cluster or regional 

capacities—such as a skilled workforce or risk capital—for long-term outcomes such as employment, income, and 

investment. In this literature, the details of policies that shape workforce or finance are not usually addressed, but 

their relative importance for outcomes are estimated, based on third party indicators.

The methodological design of this project is grounded in both bodies of work. The first grouping of literature offers 

analyses of impact, performance, and best practices for specific programs. The second presents evidence for the 

impact of specific capacities on long-term economic development outcomes. As discussed below, this literature has 

convincingly shown and empirically demonstrated the importance of capacities—for example, human capital—in 

determining long-term outcomes such as income and employment. However, as also discussed below, there are 

gaps in this literature, and the purpose of the econometric work that lies at the heart of this study was to fill a select 

number of these gaps.

The findings from literature were an important input guiding SRI’s selection of data collection requirements at the 

program level as well as of the kinds of third party indicators at the regional and aggregate levels. Taken together, 

the components and variables developed, based in part on established findings, were tested to validate a carefully 

selected set of program metrics and demonstrate their importance for the long-term outcomes (a selected 

bibliography for these literatures is provided in Appendix C).

Research on Capacity Outcomes

One of the primary goals of this project was the development of general purpose metrics for non-infrastructure 

programs, insofar as it is practicable. However, from the beginning it has been recognized that some programs are 

home to highly specialized activities, in particular programs that support a similar population of grantees, such as 

incubators and accelerators. In these cases, the importance of specific institutional variables can be disentangled in 

ways that are not possible for programs with more heterogeneous populations of grantees. For example, 

evaluations of incubators, which have been extensively investigated, have often discovered that key institutional 

attributes of an incubator, such as for-profit versus non-profit status, governance, partnerships, funding mix, size, 

etc., are predictors of success.3 Thus, an analytical framework that captures these attributes is very important. The 

goal in this project, however, was to build a framework that can encompass a broader range of programs. As such, 

the framework described below seeks to capture the specific institutional variables that have been previously 

identified as significant, while simultaneously accounting for generic variables applicable to a heterogeneous 

population of grantees. 

Program evaluations tend to investigate the institutional attributes that result in desirable outputs and outcomes 

to inform future programs. However, there is wide variation in institutional characteristics across grantee 

organizations. For example, the i6 program run by EDA has supported grantees that have included a single, freestanding non-profit, a program administered by a partnership between two regional research institutions, and a 

network of institutions addressing clean tech needs of a multi-state region. This heterogeneity reflects the 

importance of the “bottom-up” identification of economic opportunity, and emphasizes the fact that economic 

development organizations tend to invest in communities, in regional development, and in potential innovation, 

 

3 Lewis, David A., Elsie Harper-Anderson, and Lawrence A. Molnar. 2011. “Incubating Success: Incubation Best Practices That Lead to 

Successful New Ventures” Institute for Research on Labor, Employment, and the Economy, Ann Arbor, Michigan.
Innovative Metrics for Economic Development 12

not particular types of institutions. As such, any general-purpose framework must account for this very diverse 

population of programs and participants; however, developing this framework requires evaluators to confront the 

challenge of overwhelming institutional variation among grantees, rendering institutional attributes – a common 

basis for evaluation – impractical. 

To overcome this challenge, SRI sought to identify established process measures that could be included in a logic 

model and be generally applicable to a broad population of grantees, in order to avoid relying on institutional 

attributes. For example, the research on incubators (referenced in footnote 2 above) shows that time spent by staff 

on building client business skills, and time spent tracking incubator activities are indicators of good performance. 

This type of activity can be used as a process measure, and is an example of the type of measures that the SRI team 

has folded into the toolkit for directly collecting program data. 

Figure 1: Logic model for non-infrastructure economic development programs

Using an Econometric Model in Conjunction with the Logic Model

The principal goal of the cooperative agreement between SRI and EDA was to develop and operationalize a 

complete set of metrics for tracking the activities and assessing the impact of non-infrastructure economic 

development programs, both those implemented by EDA as well as those implemented by other economic 

development organizations. While the project relied on EDA programs as a model, the work was conducted with a 

broad frame of reference and is intended to be generally applicable across the economic development field. 
Innovative Metrics for Economic Development 13

The project has settled on a set of program metrics for inputs, outputs, and new capacities. These measures have 

been embodied in a reporting mechanism for grantees and their beneficiaries and clients which will yield directly 

collected data on the development of economic development capacity at the program level.

4 However, as noted 

above, the long-term impact of programs cannot be easily estimated at the program level, because of scale, and 

because of the difficulty of collecting data months and years after programs come to an end.

The other piece of this project was designed to directly address the problem of assessing long-term impact. This 

piece is comprised of an econometric model based on aggregate measures of economic development capacity. Such 

a model can exploit independently collected aggregate data that reflects both economic development capacity and 

long-term realized outcomes. Accordingly, this project identified a set of measures for economic development 

capacity, at the regional metropolitan or micropolitan level, that help assess the regional economic impacts of 

development programs and policies. These indicators were used to estimate the connection, if any, between 

economic development capacity and long-term realized outcome. Figure 2 illustrates the connections between 

program-level activities, short-term capacity outcomes, and aggregate level capacities and long-term outcomes.

Figure 2: Econometric study for non-infrastructure programs

The econometric model described below estimated (where possible) connections between economic development 

capacity, as fostered by non-infrastructure programs, and long-term realized outcomes. Specifically, realized 

outcomes were represented by measures of long-term economic performance in the form of regional growth in 

average annual earnings, employment, and income. The way that program level measures of capacity map onto 

aggregate measures of capacity, (as illustrated above,) is the hinge upon which this project turns. The project shows 

that such programs build economic development capacities, and the degree to which these capacities, in the 

aggregate, have an impact on long-term economic performance. 

 4 As described later in this report, SRI piloted this reporting mechanism with a group of recipients of EDA grants in the spring of 2017. 
Innovative Metrics for Economic Development 14

Econometric Model

Methodological Approach

The regression analysis is a key element in understanding the chain of relationships between economic 

development grant activity and long-term economic growth and development. The analysis relied on regressions of 

capacity measures against long-term economic performance, with appropriate controls. In performing this analysis, 

SRI used data on EDA non-infrastructure grants, a dataset which has previously not been used in this type of work. 

However, although the model was built using EDA-specific data, the intent of the project was to shed light on a 

broad set of relationships between regional capacities and long-term economic performance and to generate 

insights that are generally applicable across the economic development field.

Explaining Long-Term Economic Performance

Policymakers, program staff, grantees, and other practitioners are interested in programs that have a positive, longterm impact on jobs, earnings and overall economic growth. In order to detect that impact, SRI developed a crosssectional regression analysis using aggregate data for all metropolitan and micropolitan regions in the United 

States.5 The unit of analysis for this model was the Core Based Statistical Area (CBSA), as defined by Census to refer 

to both Metropolitan Statistical Areas and Micropolitan Statistical Areas. Per Census, Metropolitan Statistical Areas 

are CBSAs associated with at least one urbanized area that has a population of at least 50,000. CBSAs include the 

central county or counties containing the core, as well as adjacent outlying counties that have a high degree of 

social and economic integration with the core, as measured through commuting. Micropolitan Statistical Areas are 

CBSAs associated with at least one urban cluster that has a population of at least 10,000, but less than 50,000. As 

with Metropolitan Statistical Areas, Micropolitan Statistical Areas are comprised of the county or counties 

containing the core area, as well as adjacent outlying counties with a high degree of social and economic integration 

as established through commuting.6

SRI performed each analysis three times: once using the combined set of metropolitan and micropolitan regions, 

once using just metropolitan regions, and once using just micropolitan regions. For discussion purposes, the terms 

“regions” is sometimes used to refer to the CBSAs; this term is not referring to some other geographic designation. 

When discussing results, if a particular finding only exists at the metropolitan or at the micropolitan level, the team 

will specify which level the finding applies to (i.e. “At the micropolitan level, we found that...”). If neither the 

metropolitan nor the micropolitan level is specified in the reporting, then the result pertains to the combined set 

of regions. 

The independent variables initially consisted of two buckets. First, control variables were selected based on existing 

literature and designed to account for well-established correlations between underlying characteristics of a region 

and its future economic performance. Next, capacity outcome variables were selected that were also, in some cases, 

supported by existing literature but also informed by the logic model. 

 

5 Due to the fact that metropolitan and micropolitan statistical area definitions are not stable, but change over time as area populations 

change, we had to account for variations in composition over the timespan of our models. To do so, we created a crosswalk between the 

2013 definitions and the counties that comprise them, and used this crosswalk to aggregate historical county-level (or sometimes zip-code 

level data, when necessary) data to align with the 2013 definitions, thereby consistently reflecting areas across time. 

6 https://www.census.gov/geo/reference/gtc/gtc_cbsa.html.
Innovative Metrics for Economic Development 15

Research on Long-Term Economic Performance

The key claim is that the inputs and activities build capacity outcomes in a straightforward way, and that these 

capacities have long-term impacts on economic performance (growth in earnings, employment, and income). This 

long-term impact is supported by a significant body of literature. For example, one study finds that existing smallbusiness networks account (in part) for variation in entrepreneurship across the United States; anotherstudy shows 

that college attainment determines (in part) state income growth; and a third one suggests that serial entrepreneurs 

fuel innovation ecosystems.7

The overall logic of marrying direct collection of program data to aggregate, regional-level indicators of long-term 

impact runs as follows: based on the first study above, we may suppose that if the inputs, activities, and outputs of 

a particular program report data that show a measured impact on small business networks, then a longer-term 

impact on entrepreneurship will follow. As noted, actually tracking the impact of outputs and new capacity over 

many years is often impractical. In addition, investments are often relatively small. We can detect their impact at 

the program level, but their impact on aggregate outcomes cannot be easily discerned. 

The literature reviewed by the SRI team addressed, with varying degrees of success, the relationship between

capacity outcomes of the logic model at the aggregate level and long-term economic performance, such as growth 

in earnings and employment. This literature was then used to inform the selection of variables to be used as proxies 

for capacities in the econometric model.

In some literature, the impact of building certain kinds of capacity is well established. Regions with higher skilled 

workforces were found to have higher levels of population growth,

8 higher wages,9 or per capita income,10 and 

higher changes in gross state product over time.

11

In other literature, there is also a fair amount of agreement on impacts. For example, there have been a variety of 

analyses of the impact of high levels of R&D intensity at the regional and national level. R&D intensity, often 

described as R&D expenditures (public and/or private), or patents registered, has been found to correlate to higher 

GPD per capita12 and, in a wide variety of studies, strongly related to “opportunity” entrepreneurship and higher 

regional growth.

13

 

7 See Glaeser, E.L. and Kerr, W. 2009. “Local Industrial Conditions and Entrepreneurship: How Much of the Spatial Distribution Can We 

Explain?” Journal of Economics and Management Strategy, Vol. 18, No. 3: 623-663; Bauer, P.W., Schweitzer, M.E., and Shane, S.A. 2012, 

“Knowledge Matters: The Long Run determinants of State Income Growth.” Journal of Regional Science, 52: 240-255; and Feldman, M.P. 

and Zoller, T. 2012. “Dealmakers in Place: Social Capital Connections in Regional Entrepreneurial Economies.” Regional Studies, Vol. 46, No. 

1: 23-37. 8 Glaeser, E.L. and Saiz, A. 2003. “The Rise of the Skilled City.” Federal Reserve Bank of Philadelphia, Working Paper 04-2. 9 Florida, R, Mellander, C., Stolarik, and A Ross. 2010. “Cities, Skills and Wages.” Institute for Competitiveness & Prosperity, Working Paper. 10 Weissbourd, R & C. Berry. 2004. The Changing Dynamics of Urban America, R.W. Ventures. 11 Evans, P., and Karras, G. 1994. “Are Government Activities Productive? Evidence from a Panel of U.S. States.” The Review of Economics 

and Statistics, Vol. 76, No. 1: 1-11. 12 Sterlacchini, A. 2008. “R&D, Higher education and regional growth: Uneven linkages among European regions.” Research Policy Vol. 37, 

No. 6-7: 1096–1107; and Crescenzi, R. 2005. “Innovation and Regional Growth in the Enlarged Europe: The Role of Local Innovative 

Capabilities, Peripherality, and Education.” Growth and Change, Vol. 36 No. 4: 471–507. 13 Mueller, Pamela, 2007, "Exploiting entrepreneurial opportunities: The impact of entrepreneurship on growth." Small Business Economics, 

Vol. 28, No. 4: 355-362.
Innovative Metrics for Economic Development 16

On the other hand, less understood are the effects of formal and informal networks linking start-ups and small 

businesses to markets. Social capital (associated with high levels of trust) embodied in networks is believed by many 

to be critical for long-term growth.14 Many programs, including EDA programs, support the development of such 

networks through dedicated program activities. However, while their importance is very often asserted, there is 

little in the way of formal measurement or analysis of the long-term impact of this capacity.

The independent variables employed as proxy measures of capacity in the econometric model were selected based 

on the literature reviewed and on the logic model. Taken together they are intended to represent the waterfront 

of capacities that are at the heart of this project. Sifting their relationship to long-run economic performance is the 

goal of the econometric analysis.

Model Design

Ultimately, SRI identified that capacity variables belonged to two separate groups: variables that acted as the 

primary drivers of economic development and growth, and variables that acted as proxies for the extent of a 

region’s networking capacity and the strength of its institutional support structures. Accordingly, there are two sets 

of results presented here. The first set of results are from an analysis that regressed each of the capacity variables 

against long-term economic performance. The second set of results investigated the impact of these networking 

capacity and institutional support structure variables. For both sets of results, the capacity variables were regressed 

against the percent change from 2010 to 2015 for three different dependent economic performance variables: 

earnings, employment, and per capita income. 

In order to avoid the distorting effects of the business cycle, in particular the powerful national impact of the Great 

Recession, the time frame chosen for initial applications of the model was the period from trough to peak prior to 

the last recession. The National Bureau of Economic Research (NBER) defines recessions as a “significant decline in 

economic activity spread across the economy, lasting more than a few months, normally visible in real GDP, real 

income, employment, industrial production, and wholesale-retail sales.” 15 Recessions start at the peak of a business 

cycle, before any prolonged decline begins, and end at the trough, prior to a prolonged recovery. The first full year 

following the dot com bust was 2002, and the last full year prior to the peak immediately preceding the great 

recession was 2007.16 However, the team judged that this approach relied on too narrow a range of years that could 

not capture long-term impacts. In order to use a longer time period, the team used initial levels in 2000 for control 

variables (relying on Census data), capacity levels from between 2005 and 2007 (2007 being the last complete year 

before the peak of the economic cycle), and outcome variables from between 2010 and 2015 (2010 being the first 

full year of the recovery from the great recession). Due to the lags in the economic impact of capacity variables, SRI 

chose a time period that was sufficient to capture impact, but not too long to distort results. Capacity levels prior 

to the Great Recession reasonably account for the economic development conditions that could influence future 

outcomes after a complete return to growth following the Great Recession. Unfortunately, SRI was not able to 

account for any changes to the capacity variables that may have resulted from the response to the Great Recession 

(such as the stimulus bill or other programs). 

 

14 For a review of the first wave of social capital literature see Sobel, Joel, 2002, “Can We Trust Social Capital?” Journal of Economic 

Literature Vol. 40: 139–154. 15 The key problem of the impact of the great recession is that it varied dramatically across regions in ways unrelated to the components 

being estimated in this analysis.

16 See: http://www.nber.org/cycles.html.
Innovative Metrics for Economic Development 17

The control variables applied in the two models reported below are from 2000, being a Census year, and 

representing characteristics not likely to change as a result of a recession. The capacity variables, (the main object 

of the analysis,) were selected for the year 2006, ensuring that their relationship with the dependent variable is 

framed by one business cycle. For patents, Small Business Innovation Research (SBIR) and Small Business 

Technology Transfer (STTR) grants, and new product introductions, SRI aggregated the variables over the years 

2005, 2006, and 2007 because data for these variables are irregular. Additionally, when they are reported, these 

events are somewhat arbitrarily assigned to a particular year, and do not necessarily align with such a discrete time 

period. By aggregating the three years of data, SRI sought to ensure that data for these variables would be 

comprehensive.

For control variables, the population level, unemployment level, and educational attainment level are 

straightforward predictors of future regional economic performance. In addition, SRI wanted to investigate the 

importance of the structure of a regional economy (as measured by the share of manufacturing jobs in the 

economy) as a determinant of outcomes. The independent or capacity variables map to, and represent measures 

of, the capacities developed by economic development programs, as described in greater detail below. 

Variables

Control Variables

Given the vast differences between baseline conditions across the metropolitan and micropolitan regions included 

in the analysis, it was critical that the team identified an appropriate set of controls that could account for existing 

regional differences, without swamping model results. Ultimately, the variables and data sources contained in the 

following table were identified as an appropriate set of controls for the analysis. 

Controls

Variable 

Abbreviation Long Name Description Source

Employment 

2000 Employment, 2000 Employment levels of the MSA in 2000 BLS LAUS

Earnings, 2000 Earnings, 2000 Average annual pay of wage and salary workers, 

in 2000 BLS QCEW

Log Pop, 2000 Log Population, 

2000 Log Population count of the MSA Census

Mfg. Share, 

2000

Manufacturing as a 

Share of the 

Economy, 2000

Manufacturing jobs as a percent of total jobs in 

the metropolitan/micropolitan economy BLS QCEW

% Bachelors 

Plus, 2000 

Educational 

Attainment, 2000

Portion of the population 25+ that has a 

bachelor’s degree

Census

Per Capita Inc, 

2000

Per Capita Income, 

2000 Personal income per capita, as defined by BEA

BEA Local 

Area 

Personal 

Income

Unemp Rate, 

2000

Unemployment 

Rate, 2000

Annual average unemployment rate, not 

seasonally adjusted

BLS LAUS
Innovative Metrics for Economic Development 18

While employment, earnings, and per capita income were all included as dependent variables in the model, SRI

opted to include their baseline levels in the model as controls. Regions with higher starting levels of employment 

and earnings are likely to see stronger growth rates in those variables in later years, regardless of improvements in 

capacities. For several of the capacity variables, there is likely to be a strong correlation between the included metric 

and the size of the region. For instance, larger regions are likely to have more financing deals than smaller regions. 

As such, and to account for any influence due to population, SRI included population as a control variable, and took 

the log of the value for each region to moderate the significant range in values across regions.

17

To include a control for the structure of the economy within each region, SRI used manufacturing as a share of the 

economy, calculated as the percent of jobs within a region that belong to the manufacturing sector. Finally, 

educational attainment has long been shown to foster economic growth. To avoid erroneously attributing any 

growth in long-term performance to capacity outcomes, when in fact the growth is due to the established 

relationships between education and earnings and employment, SRI included educational attainment as a control, 

calculated as the portion of the population that is 25 and older that holds a bachelor’s degree. 

Capacity Variables

SRI identified variables and data sources to act as proxies for the capacity outcomes that are central to the 

econometric model. The SRI team continued earlier work on identifying proxy metrics18 and conducted extensive 

new research to find data to leverage in the model. While the team identified and reviewed a substantial number 

of datasets for use in the model, there were two key criteria that dictated whether a dataset could be used or not:

• Free and public access: To the fullest extent possible, the SRI team tried to only use free and publicly 

accessible datasets for the model. This was to serve two purposes: first, the use of free and public data was 

intended to support replicability of SRI’s work in the future, both to validate and expand upon the model 

and findings. Second, SRI wanted to make the model sustainable for EDA and other practitionersto continue 

to leverage in future years; relying on expensive, private data sources would likely hinder the ability to 

continue to run analyses using the model. Ultimately, free and publicly accessible data were not found for 

three capacity variables, and SRI ended up purchasing three private datasets to address those variables. 

• Available at the county-level across a time: SRI required historical, time-series data for the econometric 

model, in order to assess the impact of economic development grant activity across lagged time periods. 

While the model analysis is at the Core Based Statistical Area-level, SRI required county-level data for all 

variables in the model, in order to account for changes in CBSA-definitions over time.19 County-lines do not 

change year-over-year, while several changes in CBSA-definitions (and in fact, identification of new CBSAs) 

occurred over the time period of SRI’s model. As such, SRI collected data at the county-level, and rolled it 

up to align with current CBSA-definitions, to reflect consistently-defined CBSAs across the years of analysis. 

However, this ultimately limited the ability of SRI to use certain sources of data, which were available at the 

CBSA- or state-level, but not at the county-level. Additionally, there were some potential datasets 

considered by the team that ultimately did not cover the full time-period of the model, and therefore were 

not valid options for the model. 

 17 Wooldrige, Jeffery. 2006. Introductory Econometrics: A Modern Approach. 3rd Edition, Thomson South-Western. 18 SRI International, Building & Using a new EDA Evaluation System. 2014. 19 https://www.census.gov/programs-surveys/metro-micro/about.html.
Innovative Metrics for Economic Development 19

Ultimately, these two criteria led to the identification of the following data sources for the capacity outcome 

variables as shown in Figure 3 below. While these data sources accurately approximate the capacities captured in 

the logic model, it is important to consider that they are largely proxy measures, and do not necessarily fully or 

exclusively represent the capacities. Alternative proxy measures for these capacities may generate additional 

findings. SRI encourages further exploration of new and innovative data sources to capture different 

representations of the capacity outcome variables in future analyses. 

Figure 3: Capacity variables

To assess product, production processes, and business capacities, SRI looked for a data source that could reflect the 

production capabilities of firms and act as a proxy for product innovation. Following the earlier work done by SRI 

and University of North Carolina, Chapel Hill to identify metrics, the team worked with ThomasNet to acquire a 

database of new product introductions that had been published on their site. ThomasNet is an online website that 

allows companies to submit announcements of new products, largely industrial business-to-business (B2B)

products, which it publishes and distributes to its audience of manufacturers. Improving processes and creating 

new products is a key goal of programs such as the i6 Challenge, and regions that generate increasing numbers of 

new products, particularly on the industrial B2B side, represent strong production processes and business 

capacities. As such, we expect the number of new product introductions in a region, employed as a proxy in this 

analysis, will be positively associated with desirable long-term economic performance. 

Markets and business networks are important capacities that demonstrate strong social capital within regions, and 

building that social capital is a goal of economic development programs. To incorporate this capacity in the model, 

we used the number of membership associations and organizations, as measured by establishments reporting 

under NAICS code 813 to the Bureau of Labor Statistics. NAICS code 813 refers to “religious, grant-making, civic, 

professional, and similar organizations” and provides a proxy measure for the number of groups that create 

networks and facilitate social capital in a region. The number of such organizations reported through BLS 

(normalized by population) should be positively associated with desirable outcomes. 
Innovative Metrics for Economic Development 20

To assess the innovation, technology transfer, and commercialization capacity of regions, SRI decided to use patent 

data from the US Patent and Trademark Office. Patents are a measure that is inherently positively associated with 

innovation and technology development, and may serve as an indicator of how much innovation is occurring within 

a region. However, SRI acknowledges that this measure suffers from some well-known deficiencies. Primarily, 

patents are a lagging indicator and represent potential rather than actual innovation. Acknowledging this caveat, 

however, patent data is still a widely-used, valid proxy measure for innovation, technology transfer, and 

commercialization for this model. 

Finance is a key ingredient in innovation ecosystems, and access to a wide range of financing sources can be crucial 

to business development and growth. Economic development programs provide important support to organizations 

looking to find new sources of financing, from support on grant applications to guidance on developing business 

plans and pitches. To analyze regions’ capacities for financing and investment, SRI used two data sources. The first 

was the number of federal SBIR and STTR grants awarded in a region, accessed through the Small Business 

Administration’s award database. These grants are awarded to small businesses to pursue research and 

development that could potentially lead to technology commercialization, and provide federal funding to innovative 

small businesses to support their development capabilities. The second source of financing data was the number of 

private financing events – angel, seed, and venture capital deals – per region. These data were purchased from

Pitchbook, and represent the accessibility of private financing across regions. Both federal and private financial 

access are key capacities for regions, and we expect a positive association between these metrics and desirable 

long-term economic performance.

In assessing human capital and workforce capacities, SRI looked for data to represent the development of a 

particular set of skills associated with starting and growing businesses – skills that economic development programs 

support. We identified three data sources that could act as proxies for the development of this capacity. The first 

was the rate of establishment churn in a region.20 While the Commerce Department is not a workforce agency, its 

non-infrastructure programs are designed to build the skills needed by entrepreneurs and small business leaders. 

Establishment churn signifies the dynamism of small enterprise and, by extension, the presence of those skills, 

filtered by NAICS code to represent tech-intensive firms. 21

The second was the number of non-employer firms in a region, also filtered to represent tech-intensive firms. Nonemployer firm data is provided by Census and represents the number of businesses that have no paid employees 

and are subject to federal income tax. These types of businesses embody entrepreneurial endeavors by citizens, 

and can be used as a proxy measure for the entrepreneurial talents that exist among the workforce. SRI filtered the 

data by NAICS code to only include tech-intensive firms (as measured by share of STEM-intensive occupations), as 

these firms better illustrate the type of innovation that drives regional economic growth.22

 

20 Establishment churn is calculated as the number of business establishment startups and business failures as a share of total 

establishments. To calculate this variable, the team summed the number of establishment births and deaths in a CBSA for year t and year t1, and divided each by the total number of establishments for the respective year. These two values were then averaged to calculate the 

final variable. 

21 Per the National Science Foundation, “An industry is considered a high SET employment industry if employment in technologyoriented occupations accounts for a proportion of that industry’s total employment that is at least twice the average for all

industries”. See https://www.nsf.gov/statistics/2016/nsb20161/#/stateind, technical note #5. SRI used BLS data to determine 

such industries for the purpose of this model. 

22 Hecker, Daniel E., 2005, “High-technology employment: a NAICS-based update” Monthly Labor Review, 57-72.
Innovative Metrics for Economic Development 21

Finally, SRI included data from USDA on the “creative class” share of the workforce, occupations that require 

“thinking creatively” as a core job function. This dataset was derived from Richard Florida’s thesis in his book Rise 

of the Creative Class, which asserts that certain occupations specialize in the task of combining knowledge and ideas 

in novel ways. His argument continues that urban development hinges on these novel combinations, and individuals 

in these “creative” occupations are drawn to regions providing a high quality of life. As such, we expect to see an 

association between regions with higher creative class shares and improved long-term economic performance,such 

as increased employment and earnings. 

Finally, the organizational capacity of a region is inevitably difficult to capture through data. However, capacity is 

built by organizations, and as such, the number of organizations dedicated to economic and community 

development can be used as a proxy for the level of organizational capacity within a region. SRI purchased data 

from Guidestar on the number of economic development organizations (EDOs), by region, to assess the 

organizational capacity in the model.23

A description of these variables, their nomenclature in the model, and their underlying source data is provided in 

the table on the following page.

 23 Guidestar provides data on non-profit organizations based on their Form 990-filings with IRS. Non-profits are categorized by the National 

Taxonomy of Exempt Entities (NTEE), and SRI used data on the organizations categorized as code S, “Community Improvement, Capacity 

Building”. For more, see: https://learn.guidestar.org/help/ntee-codes.
Innovative Metrics for Economic Development 22

Variable 

Abbreviation Long Name Description Source

Capacity

Patents, 05-07

Patents from 2005 –

2007

Number of patents granted in a region between 2005 

and 2007

US PTO

Establishment 

Churn

Establishment Birth 

and Death Rates, 2006

The annual number of establishment births + deaths in 

a year, as a share of all establishments at the beginning 

of the year24

Census SUSB

Non-Employer 

Firms

Nonemployer Firms, 

Tech Intensive, 2006

As defined by Census, number of firms that have no 

paid employees and are subject to federal income tax, 

in tech-intensive industries

Census 

Nonemployer 

Statistics

Financing 

Events

Private Financing 

Events, 2006

Number of private financing events, such as angel, 

seed and/or venture capital funding deals

Pitchbook25

EDO Economic Development 

Organizations, 2006

Number of organizations in a region that are classified 

by the National Taxonomy for Exempt Entities under 

the S Community Improvement, Capacity Building code

Guidestar26

Assoc./Orgs 

(NAICS 813)

Membership 

Associations and 

Organizations (NAICS 

813), 2006

Number of firms categorized under NAICS 813: 

Membership associations and organizations BLS QCEW

SBIR + STTR, 

05-07

SBIR and STTR Awards 

between 2005 and 

2007

Number of SBIR and STTR awards granted by SBA

SBA SBIR 

Database

New Products, 

Sum 05-07

Sum of new product 

announcements, 2005-

2007

New products announced through ThomasNet ThomasNet27

Creative Class 

Share

Creative Class share of 

the workforce, 2007-

2011 (pooled)

Share of the workforce employed in occupations that 

involve a high level of thinking creatively, i.e. 

developing, designing or creating new applications, 

ideas, systems or products28

USDA 

Economic 

Research 

Service

Data Limitations

For some of the capacity variables identified, the source data were very irregular and sporadic, and presented 

quality issues when used on a year-by-year basis. In particular, patents, SBIR/STTR grants, and new product 

introductions only had a handful of counts for any region in any given year, making it hard to select a single year’s 

worth of data to work with. To address this issue as stated above, SRI aggregated three years of data to smooth out 

year variances and create a more robust variable. Therefore, while the model only used data from the year 2006 

 

24 Exact methodology is taken from the Information Technology and Innovation Foundation’s 2014 State New Economy. See 

http://www2.itif.org/2014-state-new-economy-index.pdf?_ga=1.6859442.1645463451.1492437691, p 60 for details. 25 SRI worked with University of Wisconsin Extension Division for Business and Entrepreneurship to purchase Pitchbook data. 

26 SRI purchased a custom dataset from Guidestar to account for the number of EDO’s by county, by year.

27 ThomasNet does not make this data publicly available, however SRI worked with ThomasNet staff to purchase a historical dataset of new 

product announcements.

28 This share is derived from Richard Florida’s thesis in Rise of the Creative Class, which argues that creative occupations are critical for rural 

and urban development. See the USDA ERS page for information on this data: https://www.ers.usda.gov/data-products/creative-classcounty-codes/documentation/.
Innovative Metrics for Economic Development 23

for the other capacity variables, for patents, SBIR/STTR grants, and new product introductions, the variables used 

in the model were the sum of each capacities total from 2005 to 2007. 

Capacity Variable Analysis and Results

The research team estimated the following model for each of the three outcome variables (earnings, employment, 

and per-capita income): 

OOOOOOOOOOOOOO = BB0 + BBii( cc cc ccccii) + BBjj� cccccc cc ccjj� +

Where controlsi is the set of 7 control variables, capacityj is the set of 9 capacity variables, and e is the error term. 

The three dependent variables chosen are for the change in levels for years 2010 to 2015. The independent capacity 

variables were for the period at or around 2006. Controls, as noted, are selected for the year 2000. The initial results 

from this model are displayed in Appendix B.

However, many of the variables were highly correlated (see tables in Appendix B, Table B-1) with one another, 

indicating that this analysis may suffer from multi-collinearity. For example, new product introductions, nonemployer firms, and economic development organizations are all highly correlated with four other variables. 

In assessing this multicollinearity, SRI acknowledged that generally, regions with larger populations will have higher 

levels of patents, new product introductions, non-employer firms, and economic development organizations. As 

such, we believed that while we are controlling for population with the log population variable, these other capacity 

variables were still acting as a proxy for region size. To account for this, the team normalized the variables (except 

for variables already presented as a rate or share) according to population levels in 2006. Table 1, below, describes 

the normalization procedure used for each variable.

Table 1: Normalized variable abbreviations and descriptions

Variable Abbreviation Description

2000 Earnings per Capita Earnings in 2000 divided by MSA population in 2006

Employed Share of the Population Employment levels in 2000, divided by population in 

2006 

Patents, 05-07, Normalized Aggregate patents from 2005 – 2007 per thousand 

people in 2006

New Products, 05-07, Normalized New product introductions from 2005-2007 per 

thousand people in 2006

Normalized Non-Emp firms, Tech Intensive, 2006 Number of high-tech non-employer firms per thousand 

people in 2006 

SBIR and STTR Grants, Sum 05-07, Normalized SBIR and STTR grants between 2005 and 2007 per 

thousand people in 2006

Normalized Number of Financing Events, 2006 Financing events per thousand people in 2006

Normalized Assoc & Orgs (NAICS 813) Membership association and organizations per 

thousand people in 2006 

Normalized Economic Development Orgs, 2006 Economic Development Organizations per thousand 

people in 2006
Innovative Metrics for Economic Development 24

The correlation coefficients for these normalized variables were all substantially lower (see Appendix B, Table B-2), 

indicating that SRI’s hypothesis about the source of the collinearity in the initial results was accurate. The results of 

this regression are shown in Table 5 on the following page. Compared to the initial, non-normalized results in 

Appendix B: Table B-3, this set of results represents a more accurate account of relationships between capacity 

variables and long-term economic performance, due to the additional modifications to address population size.

Detailed Discussion of Results

After accounting for population size, the coefficient of earnings per capita in 2000 on future earnings and income 

growth was positive and significant in metropolitan regions. Micropolitan regions with greater levels of initial 

earnings were likely better off, and thus better able to support future economic growth and development. The 

significant positive coefficient on patents dropped away except on earnings growth at the micropolitan level. This 

result indicated that in earlier regressions, the patents variable was acting as a proxy for region size. Controlling for 

population, the positive coefficient of patents on earnings growth in micropolitan regions suggests future earnings 

growth was positively associated with the inventiveness of a micropolitan region.

Similarly, larger numbers of membership associations and organizations were correlated with increased earnings 

growth across all regions and for micropolitan regions particularly. These results suggest that membership 

associations and organizations represent institutions that can support other types of economic activity, and may 

have contributed indirectly to long-term earnings growth. That the coefficient for membership organizations was

significant for micropolitan regions but not metropolitan regions suggests that a larger density of membership 

organizations may have been necessary to augment economic activity in larger regions compared to smaller ones. 

Furthermore, the positive coefficient of the creative class share of the workforce on employment growth across all 

regions and specifically in micropolitan regions is consistent with Richard Florida’s Rise of the Creative Class 

hypothesis. Florida postulates that urban development depends on novel combinations of knowledge and ideas.29

The higher the share of such specialized occupations, the more likely it is that new knowledge and ideas will be 

generated, driving urban development in the form of increased employment growth. 

EDA programs such as the i6 Challenge program and Seed Fund Support program help foster development of the 

complex, cognitive skills and talents that are reflected in the creative class share. These results on the creative class 

share were in line with our findings about membership organizations and associations. In both cases we found

evidence that micropolitan regions benefit from institutional support provided from membership organizations and 

associations, and from higher shares of the creative class. We did not see similar evidence in metropolitan regions. 

Larger regions are arguably more complex, socially and economically, and house more developed institutions. As 

such, these regions experience diminishing marginal returns from additional institutional activity or higher shares 

of the creative class. Conversely, small regions that are not already home to robust institutional networks will be 

more sensitive to the addition of institutional support, as captured by these two variables. However, this distinction 

has not been well-studied and could represent an area for additional future research.

Higher rates of establishment churn were negatively correlated with future earnings growth, but positively 

correlated with future employment growth. These results were logical. Increased rates of firm births and deaths 

will drive down the average age of firms in a region. Establishment churn will create new job opportunities, as 

 29 Florida, R. 2002. The Rise of the Creative Class: And How It’s Transforming Work, Leisure, Community and Everyday Life. 
Innovative Metrics for Economic Development 25

explained by a study conducted by the Kauffman Foundation.30 Young firms hire for newly created jobs at a much 

higher rate than older firms. These newly created jobs are relatively immature, and naturally, will pay less. 

New product introductions were negatively and significantly correlated with employment growth in metropolitan 

regions. This result could possibly be due to the fact that productivity and process improvements may make some 

workers redundant. The relationship between new product introductions, earnings growth, and income growth is 

worth further exploration. Controlling for population, high-tech non-employer firms were positively correlated with 

income growth in both metropolitan and micropolitan regions as well as employment growth in micropolitan 

regions, while having no correlation with earnings growth. 

We found significant negative coefficients for SBIR and STTR grants on income growth in metro regions and on 

employment growth in both metro and micro regions. We also found significant negative coefficients of EDOs on 

employment growth in micro regions and income growth over all regions. These negative coefficients were present 

in both the normalized and non-normalized results. 

The strong positive significant coefficient on financing events and the negative significant coefficient on SBIR/STTR 

grants and long-term growth in employment was surprising. Even when dropping the financing variable, the effect 

of SBIR/STTR grants remained consistent. SBIR/STTR grants are a highly specialized subset of financing events that 

are only awarded to firms performing very specific types of R&D. However, not every firm that receives an SBIR or 

STTR grant succeeds. In fact, a recent paper found that nascent firms that receive a phase II R&D award from SBIR 

are more likely to fail in their SBIR-supported R&D endeavors than are established firms.31 The research team 

considered several possible explanations for this counter-intuitive finding, the most likely being that such grants are 

more likely to go to applicants from distressed regions, and so are less likely to perform well over time (a form of 

negative selection bias). 

Summary of Results from Capacity Analysis

SRI found that financing events, membership associations and organizations, and a higher creative class share of 

the workforce were all correlated with improved long-term economic performance in the form of higher rates of 

earnings and employment growth. Higher rates of establishment churn were correlated with higher levels of 

employment growth, but reduced levels of earnings growth. Employment and per-capita income growth rates were

also positively correlated with the number of tech intensive non-employer firms in a region.

The tables below present a summary of these relationships. Strong relationships are those at the .01 level, while 

medium relationships are at the .05 level; the color of the cell indicates the direction of the relationship, with green 

cells indicating positive relationships and red cells indicating negative relationships. Table 2 is the results from the 

combined set of metropolitan and micropolitan regions, Table 3 is from just the metropolitan regions, and Table 4

is from just the micropolitan regions. 

 

30 http://www.kauffman.org/what-we-do/research/business-dynamics-statistics/business-dynamics-statistics-briefing-job-creation-workerchurning-and-wages-at-young-businesses. Haltiwanger, J., Hyatt, H., McEntarfer, E., Sousa, L. 2012. “Job Creation, Working Churning, and 

Wages at Young Businesses.” Business Dynamics Statistics Briefing, Kauffman Foundation.

31 Gicheva, Link. 2016. “On the Economic Performance of Nascent Entrepreneurs.” European Economic Review 86: 109-117.
Innovative Metrics for Economic Development 26

Table 2: Identified relationships, combined metropolitan and micropolitan regions

Earnings Employment Per Capita Income

Patents

New Product Introductions

Establishment Churn Medium Strong

Non-Employer Firms Strong Strong

SBIR/STTR Strong Medium

Financing Events Strong Strong

Associations/Organizations Strong

EDOs Strong Medium

Creative Class Share Strong Medium

Table 3: Identified relationships, metropolitan regions only

Earnings Employment Per Capita Income

Patents

New Product Introductions Medium

Establishment Churn Strong Strong

Non-Employer Firms Strong

SBIR/STTR Medium Medium

Financing Events Strong Strong Strong

Associations/Organizations

EDOs

Creative Class Share Medium
Innovative Metrics for Economic Development 27

Table 4: Identified relationships, micropolitan regions only

Earnings Employment Per Capita Income

Patents Medium

New Product Introductions

Establishment Churn Medium

Non-Employer Firms Strong Strong

SBIR/STTR Medium

Financing Events

Associations/Organizations Strong

EDOs Strong

Creative Class Share Strong

Overall, the results of this regression analysis were encouraging. The different capacity measures that economic

programs, such as EDA’s non-infrastructure grants, are designed to support were all associated with various 

measures of improved long-term economic performance. With a better understanding of how these capacity 

variables influence long-term economic performance, practitioners can build on current best practices, as well as

better tailor their programs depending on the type of activity they seek to promote. Furthermore, differences in 

the behavior of the capacity variables between metropolitan and micropolitan regions provide a guideline for how 

economic development strategies should vary based on the characteristics of the region they are deployed in. 

At the metropolitan level, establishment churn was positively correlated with long-term employment growth, while 

the presence of non-employer firms was correlated with long-term income growth. Financing events at the 

metropolitan level were positively correlated with all three measures of improved long-term economic 

performance. These results suggest that for metropolitan regions, the level of activity was an especially important 

indicator of future economic performance, namely growth, and that these regions could support higher rates of 

creative destruction and benefit from freely available capital. 

At the micropolitan level, patents and membership associations and organizations were positively correlated with 

long-term earnings growth. The number of non-employer firms were correlated with long-term employment and 

income growth, and the creative class share of the workforce was also correlated with long-term employment 

growth. These results indicate that inventiveness and creativity of micropolitan regions, as indicated by the level of 

patent activity, the presence of a creative class, and high-tech non-employer firms, were important indicators for 

future economic growth. Our results suggest that for micropolitan regions, the composition and nature of the region 

was a more relevant indicator of future economic growth than the level and quantity of new business activity. These 

results suggest that regions with strong network capacity effectively allocate the resources from investment and 

interventions. This therefore suggests that investments that foster improved regional characteristics and capacities, 

such as EDA’s non-infrastructure grants, are particularly useful in micropolitan regions. 
Innovative Metrics for Economic Development 28

Table 5: Capacity indicators on outcome variables Standard errors clustered at the CBSA level. * p < 0.10, ** p < 0.05, *** p < 0.01

(1) (2) (3) (4) (5) (6) (7) (8) (9)

All, 

Earnings

All, 

Employment

All, Income Metro, 

Earnings

Metro, 

Employment

Metro, 

Income

Micro, 

Earnings

Micro, 

Employment

Micro, 

Income

2000 Earnings per Capita 0.0158*** -0.00169 0.0420*** -0.00302 -0.0291 -0.148 0.0249*** 0.0177 0.0727**

(0.00506) (0.00999) (0.0143) (0.0177) (0.0368) (0.140) (0.00695) (0.0132) (0.0336)

Employed Share of the Population 0.0213 -0.0469 -0.0739 0.0244 -0.0835 0.0382 -0.00772 -0.0598 -0.235

(0.0219) (0.0324) (0.0970) (0.0267) (0.0534) (0.115) (0.0319) (0.0408) (0.196)

Unemployment Rate, 2000 -0.00232*** -0.00385** 0.00878 -0.00195* 0.000519 0.0235* -0.00252* -0.00660** -0.00322

(0.000872) (0.00181) (0.00709) (0.00103) (0.00177) (0.0135) (0.00136) (0.00275) (0.00660)

Mfg Share, 2000 -0.0167 0.0987*** -0.00363 -0.0300 0.179*** -0.0715 -0.00436 0.0911** 0.0582

(0.0191) (0.0314) (0.106) (0.0299) (0.0548) (0.125) (0.0237) (0.0368) (0.128)

% Bachelors Plus, 2000 0.000320 -0.00124* 0.00255 0.000399 0.00136 0.00868* 0.000621 -0.00228** 0.00197

(0.000427) (0.000682) (0.00202) (0.000500) (0.000897) (0.00492) (0.000615) (0.000962) (0.00249)

Patents, 05-07 Normalized 0.00260 -0.00197 0.00992* 0.000498 -0.000936 0.00791 0.00891** -0.00746 0.0166*

(0.00210) (0.00155) (0.00540) (0.00230) (0.00167) (0.00535) (0.00356) (0.00491) (0.00950)

New Products, 05-07, Normalized 0.0193 -0.0864 0.248 -0.0837 -0.317** 0.869 0.0418 0.00170 -0.0427

(0.0594) (0.0944) (0.270) (0.0841) (0.138) (0.908) (0.0737) (0.117) (0.141)

Establishment Churn -0.116** 0.400*** -0.289 -0.217*** 0.738*** 0.0499 -0.113* 0.162 -0.450

(0.0494) (0.0841) (0.202) (0.0803) (0.143) (0.470) (0.0637) (0.0994) (0.275)

Normalized Non-Emp Firms, Tech Intensive, 2006 -0.000301 0.00253*** 0.00957*** 0.000476 0.00228* 0.00735*** -0.000581 0.00311*** 0.00954***

(0.000513) (0.000877) (0.00168) (0.000659) (0.00123) (0.00233) (0.000696) (0.00115) (0.00249)

SBIR and STTR Grants, Sum 05-07, Normalized 0.00621 -0.0412*** -0.129** -0.00596 -0.0387** -0.243** 0.0132 -0.0448** -0.0336

(0.00699) (0.0117) (0.0580) (0.0108) (0.0188) (0.110) (0.00970) (0.0174) (0.0354)

Normalized Number of Financing Events, 2006 0.166*** 0.292*** 0.122 0.276*** 0.449*** 0.905*** 0.107* 0.239* -0.161

(0.0477) (0.105) (0.158) (0.0689) (0.117) (0.334) (0.0584) (0.131) (0.197)

Normalized Assoc. & Orgs (NAICS 813), 2006 0.0153*** 0.00365 0.0253* 0.0114 0.00954 0.0573* 0.0214*** 0.00517 0.0310*

(0.00480) (0.00792) (0.0139) (0.00705) (0.0112) (0.0322) (0.00583) (0.00965) (0.0176)

Normalized Economic Development Orgs, 2006 -0.0733 -0.264*** -0.268** -0.163* -0.170 -0.745 -0.0632 -0.268*** -0.151

(0.0467) (0.0813) (0.128) (0.0853) (0.131) (0.602) (0.0531) (0.0905) (0.0953)

Creative Class Share of Workforce -0.0641 0.325*** -0.629** -0.0802 -0.139 -1.818** -0.141 0.444*** -0.374*

(0.0647) (0.106) (0.280) (0.0975) (0.179) (0.915) (0.0873) (0.141) (0.217)

Constant 0.121*** -0.0704** 0.150 0.142*** -0.103** 0.137 0.131*** -0.0322 0.225*

(0.0173) (0.0295) (0.101) (0.0248) (0.0517) (0.166) (0.0222) (0.0362) (0.132)

Observations 903 903 900 380 380 378 523 523 522

Adjusted R2 0.113 0.225 0.026 0.185 0.377 0.048 0.103 0.155 0.024
Innovative Metrics for Economic Development 29

Differentiating Capacity Variables: Economic Drivers and Networking Capacity

As mentioned earlier, in our second analysis, SRI separated capacity variables into two categories: 

variables that acted as economic drivers and variables that represent networking capacity and support 

structures. Economic drivers consisted of patents, new product introductions, establishment churn, nonemployer firms, financing events, and SBIR/STTR grants. These variables all related directly to production 

of goods and services, increased employment opportunities, and direct injection of capital into businesses 

and regions. 

On the other hand, variables such as economic development organizational investments by various actors 

(federal, state, and local government, non-profit organizations, and private entities), economic 

development organizations, and membership associations and organizations acted as collaboration 

systems and support structures for other types of economic activity. 32 EDOs and membership 

organizations exist to establish, accelerate, and augment work done by regional actors. Similarly, 

economic development grants are generally for organizations to build networks, institutions, and 

technical capabilities, or to build projects (in the case of infrastructure funding), but do not often directly 

inject capital into a regional economy. 33 The research team hypothesized that these types of grants 

interacted with the presence of economic development organizations and membership organizations to 

influence long-term economic performance. When taken together, they served as a proxy for the 

presence of networking capacity systems within a region.

While our hypothesis could apply to any type of economic development grant, we conducted this analysis 

using data specifically on the impact of EDA non-infrastructure grants. These grants are funded through a 

variety of different programs and support a broad range of activities and outputs, from planning activities 

to business training and support. For instance, the Partnership Planning Grants helps regional 

organizations develop documents such as Community Economic Development Strategies (CEDS), while 

the Regional Innovation Strategies (RIS) program supports innovation and entrepreneurship activities. 

Our access to these data through our collaboration with EDA, as well as our belief in the importance and 

impact of these programs, were the motivation behind using this dataset for this analysis. However, while 

this EDA-specific data formed the basis of this work, we acknowledge that there are many other 

organizations – state, federal, and private/non-profit – that fund similar programs, and as such the 

analysis conducted here is applicable to a broader group of stakeholders than just EDA grantees. 

SRI sought to analyze how the presence of these networking capacity variables contributed to long-term 

economic performance. Each of the three independent variables captured different components of a 

region’s networking capacity. Through the first analysis, presented above, SRI was unable to identify the 

specific nature and impact of each individual variable. However, to test how the three terms worked 

 

32 As mentioned earlier, SRI used data on EDA grants to build and run this model; however, this variable is intended to be a 

proxy for economic development grant activity more broadly. 

33 There are some cases of economic development grants providing direct capital; for instance, the Rotating Loan Fund 

supported by EDA. 
Innovative Metrics for Economic Development 30

together to contribute to long-term economic performance, we dropped each individual variable from 

our analysis and focused instead on examining the combined impact of all three variables. 

The scale of each of these variables differed substantially. To weight each type of variable equally, for 

each of the three networking capacity component variables, we created three separate bins indicating a 

low concentration, a medium concentration, and a high concentration based on their distributions. For 

example, for membership organizations, the low bucket ranged from 0 to .005 membership organizations 

per thousand people, the medium bucket ranged from .005 to .01 membership organizations per 

thousand people, and the high bucket covered all instances with more than .01 membership organizations 

per thousand people. The research team then coded these binned variables from 1 to 3. The three coded

indicators were then multiplied together to construct the combined network variable, which ranged from 

0 to 27. This procedure effectively creates an interaction term between EDA grants, membership 

organizations, and EDOs. The interaction term captures how all three variables must be present at a high 

level to contribute to economic growth.

We first tested the following model: 

OOOOOOOOOOOOOO = BB0 + BBii( cc cc ccccii) + BBjj�ww wwh cccccc cc ccjj� + BBkk( nnkk) +

This model tested how the networking capacity variables worked together to influence long-term 

economic performance. Then, for each of the six capacity variables with direct capacity effects (patents, 

new products, churn, non-employer firms, SBIR/STTR grants, and financing) we tested the following 

model.

OOOOOOOOOOOOOO = BB0 + BBii( cc cc ccccii) + BBjj�ww wwh cccccc cc ccjj� + BBkk( nnkk)

+ BBkk� nnkk ∗ dd eeee cccccc cc ccjj� +

This model tested the interaction between the combined networking capacity variable and each direct 

effect capacity variable to explore whether or not the presence of networking capacity specifically 

augmented the impact of the direct effect capacity variables. 

In our results, presented in Table 6 below, we found that the presence of networking capacity in a region, 

as indicated by the concentration of EDOs, EDA grants, and membership organizations, was positively 

correlated with long-term earnings growth in all regions and at the micropolitan level. However, none of 

the interaction terms between networking capacity effects and any of the direct effect capacity variables 

had significant correlation. The signs and significance for the coefficients of all other variables remained 

the same. Thus, we did not find any evidence that the presence of a networking capacity influences the 

impact of the direct effect capacity variables. However, these results did suggest that the networking 

capacity variables (EDOs, EDA grants, and membership organizations) rely on each other to influence longterm economic performance in the form of earnings growth. 
Innovative Metrics for Economic Development 31

Recall that in the earlier results presented in Table 5, of these three variables, only membership 

organizations had a significant and positive correlation with long-term earnings growth in all regions and 

metropolitan regions. We had some concern that membership organizations were dominating the impact 

of the other networking capacity variables. We tested four versions of this model, each with a different 

definition of the networking indicator variable: 

1. networking = edo_bin*orgs_bin*eda_bin 

2. networking = edo_bin*orgs_bin

3. networking = orgs_bin*eda_bin

4. networking = edo_bin*eda_bin 

In these tests, we found that the networking capacity variable had a significant positive correlation with

long-term earnings growth in the first and third models, which confirmed that membership organizations 

act as a strong driver of the correlation between networking capacity and earnings growth. However, the 

lack of a positive result from the second model indicated that membership organizations are not solely 

responsible for this correlation. The lack of any significant coefficient on the networking capacity variable 

in the fourth model likewise indicated that EDOs and EDA grants without the presence of membership 

organizations are insufficient to influence long-term economic performance. 

As discussed earlier, membership organizations are a proxy of the degree of institutional support for 

economic activities present in a region. High levels of institutional support are clearly necessary for 

networking capacity to influence long-term economic performance. Similarly, EDOs can be thought of as 

a different type of institutional support structure that act in a separate and unique fashion. The results 

from this analysis suggest that for both membership organizations and EDOs to positively impact longterm earnings growth, economic development grants (in this case, from EDA) were necessary. 

These results supported our hypothesis that economic development grants, EDOs, and membership 

organizations each act to support networking capacity in a unique and complementary manner. However, 

we were unable to make any conclusions about the exact nature of the differences between the three 

variables. 

Summary of Networking Capacity Analysis

Networking capacity was positively correlated with long-term earnings growth at the micropolitan level 

and across all types of regions. This result was in-line with our hypothesis that networking capacity was

especially important in smaller, micropolitan regions. Particularly relevant for this study was the observed 

importance of EDA grants, which were used as a proxy for institutional investment. The presence of these 

grants in micropolitan regions were crucial for the significance of the relationships between the 

networking capacity variables and improved long-term economic performance. This suggests that EDA 

grants play a vital role in supporting and enhancing the networking capacity of regions, which is in turn 

associated with improved long-term economic performance.
Innovative Metrics for Economic Development 32

In addition, in our first model, we found economic development organizations to have a significant 

negative correlation with long-term employment and per capita income growth. These negative 

correlations disappeared when accounting for the presence of a networking capacity variable, suggesting 

that economic development organizations alone were insufficient at fostering ecosystems conducive to 

long-term economic performance, but can work in conjunction with other types of networking capacity. 
Innovative Metrics for Economic Development 33

Table 6: Capacity indicators on outcome variables with networking capacity analysis

(1) (2) (3) (4) (5) (6) (7) (8) (9)

All Earnings All Employment All Income Metro Earnings Metro 

Employment

Metro Income Micro Earnings Micro 

Employment

Micro Income

2000 Earnings per Capita 0.0190*** -0.00109 0.0457*** 0.00470 -0.0253 -0.123 0.0279*** 0.0186 0.0742**

(0.00497) (0.00984) (0.0140) (0.0185) (0.0369) (0.133) (0.00685) (0.0131) (0.0336)

Employed Share of Pop 0.0234 -0.0611** -0.0710 0.0135 -0.0913* 0.0119 0.00527 -0.0698* -0.206

(0.0212) (0.0309) (0.0955) (0.0259) (0.0508) (0.140) (0.0309) (0.0393) (0.185)

Unemployment Rate, 2000 -0.00219** -0.00382** 0.00923 -0.00192* 0.000642 0.0241* -0.00211 -0.00665** -0.00227

(0.000873) (0.00181) (0.00717) (0.00104) (0.00176) (0.0136) (0.00135) (0.00277) (0.00653)

Mfg Share of Emp, 2000 -0.0200 0.107*** -0.0114 -0.0255 0.179*** -0.0840 -0.0114 0.0973*** 0.0366

(0.0182) (0.0309) (0.103) (0.0276) (0.0513) (0.133) (0.0231) (0.0363) (0.127)

% Bachelors or >, 2000 0.000187 -0.00132** 0.00231 0.000296 0.00126 0.00817* 0.000398 -0.00236** 0.00167

(0.000425) (0.000672) (0.00196) (0.000513) (0.000896) (0.00474) (0.000616) (0.000952) (0.00237)

Patents, 05-07, Norm 0.00287 -0.00108 0.0107** 0.00110 -0.000341 0.0108* 0.00938** -0.00590 0.0173*

(0.00193) (0.00149) (0.00529) (0.00225) (0.00161) (0.00616) (0.00377) (0.00475) (0.00971)

New Prod, 05-07, Norm 0.00519 -0.130 0.208 -0.0784 -0.318** 0.878 0.0252 -0.0559 -0.0509

(0.0584) (0.0890) (0.275) (0.0838) (0.137) (0.920) (0.0738) (0.113) (0.136)

Establishment Churn -0.138*** 0.429*** -0.330* -0.206*** 0.737*** 0.00379 -0.150** 0.186* -0.539*

(0.0465) (0.0810) (0.194) (0.0735) (0.132) (0.355) (0.0619) (0.0962) (0.299)

Non-Emp Firms, HT, 2006, Norm 0.000113 0.00241*** 0.0103*** 0.000620 0.00249** 0.00857*** 0.0000698 0.00294*** 0.0107***

(0.000486) (0.000805) (0.00174) (0.000649) (0.00121) (0.00235) (0.000677) (0.00107) (0.00280)

SBIR/STTR Grants, 05-07. Norm 0.00448 -0.0402*** -0.131** -0.00564 -0.0378** -0.240** 0.0117 -0.0440*** -0.0357

(0.00706) (0.0113) (0.0580) (0.0106) (0.0188) (0.110) (0.00990) (0.0166) (0.0349)

Normalized Financing Events, 2006 0.160*** 0.303*** 0.110 0.271*** 0.443*** 0.860*** 0.107* 0.250* -0.179

(0.0477) (0.105) (0.162) (0.0699) (0.118) (0.322) (0.0603) (0.135) (0.208)

Creative Class Share of Workforce -0.0649 0.315*** -0.618** -0.0995 -0.156 -1.876** -0.116 0.449*** -0.307

(0.0645) (0.106) (0.282) (0.0974) (0.178) (0.906) (0.0883) (0.139) (0.204)

Networking Capacity 0.00185*** -0.000101 0.000178 0.00125 0.000112 0.00106 0.00230** -0.000113 -0.00230

(0.000679) (0.000983) (0.00226) (0.000879) (0.00118) (0.00339) (0.00110) (0.00195) (0.00328)

Constant 0.125*** -0.0756** 0.153 0.145*** -0.0990* 0.161 0.130*** -0.0392 0.225*

(0.0172) (0.0300) (0.0980) (0.0248) (0.0509) (0.154) (0.0225) (0.0363) (0.133)

Observations 903 903 900 380 380 378 523 523 522

Adjusted R2 0.109 0.213 0.025 0.181 0.375 0.047 0.090 0.139 0.024

Standard errors clustered at the CBSA level. * p < 0.10, ** p < 0.05, *** p < 0.01
Innovative Metrics for Economic Development 34

Program Level Impacts: Collecting and 

Assessing Data

Returning to the visual introduced earlier in the report and replicated in Figure 4 below, while the econometric 

model has advanced our understanding of the aggregate, regional level relationships between capacities and 

realized long-term outcomes, the second component to this project was understanding the relationship between 

non-infrastructure economic development grants and the identified capacity outcomes. 

Figure 4: Project components

To address this, SRI developed a data collection methodology and survey instruments to enable EDA and other 

funders of economic development initiatives to assess the impact of their programs on specific regional capacities.

The approach to data collection is purposefully broad and flexible, intended to accommodate a broad population 

of grantee organizations. The approach consists of two survey instruments: one intended to collect information 

from funded organizations on the exact outputs (activities, services, etc.) that were supported through the funding 

received (referred to as the “program outputs survey”); and a second intended to collect information from the end 

beneficiary of the activities or services on the impact of those services on the range of capacity outcomes (the 

“capacity outcomes survey”). More likely than not, the second survey will typically be distributed to a set of clients 

served by the funded organizations, however, it is possible that in some situations, the funded organizations 

themselves are the ultimate beneficiaries who have experienced improvements in capacity. 
Innovative Metrics for Economic Development 35

The survey instruments were built according to the logic model presented earlier in the report, and shown again 

here. 

Figure 5: Economic development program logic model

The program outputs survey collects data that align with the six categories in the middle column, asking grantees 

about the activities, services, equipment, training, and other support they provided due to the support of the grant 

or funds. The capacity outcomes survey asks detailed questions that align with the six capacity outcomes identified 

in the model, and probes respondents to assess their improvement in each capacity. It is not expected that any 

funded organization will have generated program outputs in each of the six categories, or that any beneficiary will 

have improved in all six capacities. The survey is structured such that respondents are only required to respond to 

questions that are applicable to their specific goals and approach. However, by including all six categories of 

activities and capacities in a single survey, program staff can more easily collect and aggregate meaningful data 

from a broad range of organizations. 

Data collection of this sort is not an insignificant endeavor and requires commitment to implementing survey 

procedures and collecting quality data. To support this endeavor, SRI developed a toolkit to accompany the survey 

instruments that can be used by grant-making organizations and economic development practitioners to 

successfully implement this approach. The toolkit, which can be found in its entirety in a separate report, provides 

detailed information on the survey instruments, support and justification for the metrics being collected, 

explanations of how to implement survey procedures, and best practices for achieving survey success. 
Innovative Metrics for Economic Development 36

Pilot Survey and Results

SRI piloted the survey instrument that was developed by distributing it to grantees of both the i6 Challenge program 

and the Seed Fund Support (SFS) Program in 2014 and 2015. SRI tested the fielding of the survey to grantee clients 

by having i6 grantees forward the survey on to anyone they had served through their i6 grant in the past year, as 

shown in Figure 6 below. 

Figure 6: Overview of survey flow

While fielding two surveysto both grantees and clients encountered some logistical difficulties (as discussed below), 

it nevertheless generated important insights into the full scope of the impact of EDA non-infrastructure grants. 

Table 7 below shows the total number of survey recipients, program grantees, and respondents, program clients or 

beneficiaries.

Table 7: Summary of survey recipients and respondents

Recipients Respondents

i6 Grantees 34 26

i6 Clients unknown34 129

SFS Grantees 17 13

Total 51+ 168

While the data instruments developed for this project were intended to be used across all non-infrastructure 

programs, and aggregated to generate a holistic view of the outputs of non-infrastructure grants portfolios, 

throughout this project SRI staff modified the data collection instrument following feedback received from EDA 

staff. As such, the survey that was fielded to the i6 grantees – who received the survey in February 2017 – was 

slightly different than the one that was fielded to SFS grantees in April 2017. Therefore, the two sets of survey 

responses cannot be perfectly aggregated. However, there was substantial overlap between the two surveys, and 

where possible, SRI aggregated the responses from grantees from both programs. Selected summary statistics from 

the grantee surveys are presented in Table 8 below, and the full table of aggregated responses is contained in 

Appendix D: Survey Results.

 

34 SRI does not know how many total clients received the survey, as SRI asked the i6 grantees to distribute the survey directly. There was no 

reporting or tracking mechanism to determine how many clients received the survey. 
Innovative Metrics for Economic Development 37

These findings establish that the protocol and instrument designed by SRI successfully captured program activities 

and measurable improvements to economic development capacity. This proof of principle provides a model for the 

survey of complex programs and confirms that useful data can be collected in this way. The summary statistics 

reported below give examples of the kind of data that can be collected through a well-designed survey instrument. 

Such data show program success in building capacity.

Table 8: Selected summary statistics from grantee program outputs survey

Program Output Total

Total number of clients or beneficiaries served 3,619

Number of clients assisted with technology commercialization, licensing, patenting, or 

other regulatory or government approvals

208

Number of clients assisted in obtaining angel, seed, or venture capital funding 168

Number of participants who attended training and skill assistance sessions 5,159

Number of conferences, showcases, and/or exhibitions and networking events held 881

The client-level survey asked the clients and beneficiaries about the impacts and improvements they saw as a result 

of services received from the grantees. The impacts were grouped into five categories, and clients were asked to 

rate the overall level of improvement they saw in each category, due to support and services from the grantees. 

Table 9 below shows the average client rating of improvement within each category, on a scale from 1 to 5 (where 

5 is substantial improvement). 

Table 9: Average client rating of improvement due to grantee support and services

Innovation, tech 

transfer, or 

commercialization 

capacities

Product and 

production 

processes 

and business 

capacities

Human 

capital and 

workforce 

capacities

Access to 

markets 

and 

business 

networks

Access to 

financing 

and 

investment

Average Client 

Rating of 

Improvement

3.99 4.02 3.52 3.77 3.53

Within each category of impact, the survey asked for additional details and specificity about the improvements seen 

from the support and services received. Table 10 below presents some of these summary impacts, and the full table 

of summary results can be found in Appendix D: Survey Results. 

Table 10: Selected summary statistics of client capacity outcomes

Client Capacity Outcome Total

Total number of clients who said they increased the TRL of a product 56

Average increase in TRL 3.39

Total number of new technologies that were licensed or brought to market 44

Total number of new clients gained 3,660

Total number of hours spent on entrepreneurship or leadership programs 8,140
Innovative Metrics for Economic Development 38

Conclusions

Our goal for this project was to understand the pathways through which economic development programs have an 

impact on realized outcomes in the long term. We developed a logic model for capturing the impact of economic 

development programs on economic development capacity, and then tested aggregate measures of those 

capacities against long-term economic performance. The figure introduced earlier in the report, and presented 

again in Figure 7, outlines this dual-level approach to assessment. It is the key to understanding how noninfrastructure economic development initiatives can be shown to improve both capacity outcomes and realized 

outcomes at the regional level. 

Figure 7: Dual-level approach to assessing impacts

Our econometric analysis suggested that economic development capacity correlates with desirable long-term 

economic performance in different ways, depending on the metropolitan or micropolitan nature of the region. In 

metropolitan regions, establishment churn, the number of high-tech non-employer firms, and financing events are, 

per our analysis, positively correlated with positive long-term economic performance, such as growth in 

employment and earnings, suggesting that these regions can support high rates of creative destruction and benefit 

from freely available capital. Conversely, in micropolitan regions, regional characteristics of innovation and 

creativity (patent activity, high-tech non-employer firms, and creative class share) were correlated with future 

economic performance. Furthermore, networking capacity matters most for smaller regions with limited 

institutional capacity.

SRI discovered that these economic development capacities impact different components of the economy. For 

example, establishment churn was correlated with long-term employment growth, while financing events were 

correlated with long-term earnings growth. This refined understanding of how economic development capacities 

influence specific, region-dependent components of the economy provides a baseline to guide economic 
Innovative Metrics for Economic Development 39

development strategies to ensure that a specific strategy is well-suited to accomplish its regional goals. It also helps 

to elucidate ways in which economic development grants aimed at expanding capacities have influenced regional 

long-term economic performance. These results serve as a guidepost to improving program evaluation and design 

in the economic development field more broadly. 

Ultimately, this project advanced the field of program evaluation in economic development by thoroughly exploring 

the chain of connections that drive long-term regional outcomes, from organizational activities, to program and 

regional level capacities, and ultimately to improved economic performance. Guided by a logic model, this project 

examined two sets of critical relationships: those between EDA grant activity and organizational capacities, and 

those between aggregate, regional level capacities and long-term economic performance. The tools developed 

through this project have generated evidence that non-infrastructure economic development initiatives produce 

improvements in regional capacities – capacities that our econometric analyses suggest are correlated with 

improved long-term regional outcomes. Taken together, this work provides a holistic, evidence-based view of how 

regional investments in economic development influence economic performance, and how those investments can 

be better designed and evaluated to improve impact. 
Innovative Metrics for Economic Development 40

Appendix A: Summary Statistics

Table A- 1: All regions

All Regions 

 Variable Obs Mean Std.Dev Min Max

Regional Control Variables

Earnings, 2000 929

 

4.49 

 

1.86 

 

1.70 

 

17.50 

Manufacturing Share, 2000 903

 

0.18 

 

0.11 

 

0.00 

 

0.58 

% Bachelors Plus, 2000 917

 

18.89 

 

7.56 

 

6.20 

 

60.50 

Population 917

 

304,609.20 

 

1,032,601.00 

 

12,747.00 

 

19,200,000.00 

Earnings, 2000 917

 

0.43 

 

0.36 

 

0.00 

 

2.40 

Employed Share of Population, 2000 917

 

0.40 

 

0.08 

 

0.11 

 

0.74 

Capacity Variables

Patents, 2005-2007, Normalized 917 0.0043938 0.008486 0 0.1250342

New Product Introductions, 2005-2007, 

Normalized 917 0.0001271 0.0002034 0 0.0013514

Establishment Churn 917 0.1847068 0.0341519 0.0884527 0.3439444

Non-employer Firms, Tech Intensive, 

Normalized, 2006 917 0.0858478 0.042675 0 0.3391464

SBIR and STTR Grants, 2005-2007, 

Normalized 917 0.0004813 0.0015821 0 0.0184481

Financing Events, 2006 917 0.0003959 0.000392 0 0.0029552

Normalized Membership Associations 

and Organizations, 2006 917 0.0046288 0.0031861 0.0004172 0.0257339

Normalized Economic Development 

Orgs, 2006 917 0.0003599 0.0003022 0 0.0017939

Creative Class Share, 2006 917 0.2011185 0.0510088 0.0911917 0.4975423

EDA Grants, 2005-2007 929 1.487621 2.486192 0 22

Networking Capacity 929 1.012917 1.837658 0 18
Innovative Metrics for Economic Development 41

Table A- 2: Metropolitan regions

Metropolitan Regions

 Variable Obs Mean Std.Dev Min Max

Regional Control Variables

Unemployment Rate, 2000 388

 

4.30 

 

1.94 

 

1.70 

 

17.50 

Manufacturing Share, 2000 380

 

0.14 

 

0.08 

 

0.01 

 

0.51 

% Bachelors Plus, 2000 381

 

22.29 

 

7.26 

 

9.90 

 

52.60 

Population 381

 

663,101.40 

 

1,532,579.00 

 

55,410.00 

 

19,200,000.00 

Earnings, 2000 381

 

0.14 

 

0.10 

 

0.00 

 

0.57 

Employed Share of Population, 2000 381

 

0.42 

 

0.08 

 

0.11 

 

0.71 

Capacity Variables

Patents, 2005-2007, Normalized 381 0.0066063 0.0114694 0.000133 0.1250342

New Product Introductions, 2005-2007, 

Normalized 381 0.0001676 0.0001856 0 0.0010548

Establishment Churn 381 0.196585 0.0317035 0.1252007 0.3439444

Non-employer Firms, Tech Intensive, 

Normalized, 2006 381 0.0986057 0.0407425 0.0306678 0.3391464

SBIR and STTR Grants, 2005-2007, 

Normalized 381 0.0006543 0.0015359 0 0.0124541

Financing Events, 2006 381 0.0004874 0.0003649 0 0.0029552

Normalized Membership Associations 

and Organizations, 2006 381 0.0045322 0.0026031 0.0011367 0.0192076

Normalized Economic Development 

Orgs, 2006 381 0.0003908 0.0002161 0 0.0013555

Creative Class Share, 2006 381 0.2297933 0.0477422 0.1277058 0.4261947

EDA Grants, 2005-2007 388 2.489691 3.111602 0 22

Networking Capacity 388 1.458763 2.124266 0 18
Innovative Metrics for Economic Development 42

Table A- 3: Micropolitan regions

Micropolitan Regions

 Variable Obs Mean Std.Dev Min Max

Regional Control Variables

Unemployment Rate, 2000 541

 

4.62 

 

1.78 

 

1.90 

 

16.60 

Manufacturing Share, 2000 523

 

0.20 

 

0.12 

 

0.00 

 

0.58 

% Bachelors Plus, 2000 536

 

16.47 

 

6.80 

 

6.20 

 

60.50 

Population 536

 

49,785.47 

 

26,840.53 

 

12,747.00 

 

217,065.00 

Earnings, 2000 536

 

0.64 

 

0.33 

 

0.13 

 

2.40 

Employed Share of Population, 2000 536

 

0.39 

 

0.09 

 

0.16 

 

0.74 

Capacity Variables

Patents, 2005-2007, Normalized 536 0.0028211 0.0048883 0 0.0712159

New Product Introductions, 2005-2007, 

Normalized 536 0.0000983 0.0002106 0 0.0013514

Establishment Churn 536 0.1762636 0.0333341 0.0884527 0.3363229

Non-employer Firms, Tech Intensive, 

Normalized, 2006 536 0.0767792 0.0417224 0 0.3182011

SBIR and STTR Grants, 2005-2007, 

Normalized 536 0.0003583 0.0016042 0 0.0184481

Financing Events, 2006 536 0.0003309 0.0003978 0 0.0026284

Normalized Membership Associations 

and Organizations, 2006 536 0.0046975 0.0035434 0.0004172 0.0257339

Normalized Economic Development 

Orgs, 2006 536 0.0003379 0.0003493 0 0.0017939

Creative Class Share, 2006 536 0.1807358 0.0428262 0.0911917 0.4975423

EDA Grants, 2005-2007 541 0.7689464 1.563542 0 14

Networking Capacity 541 0.6931608 1.524531 0 12
Innovative Metrics for Economic Development 43

Appendix B: Robustness Checks

Table B- 1: Correlation matrix of normalized capacity variables

Patents, 

2005-2007

New 

Products, 

2005-2007

Establishment 

Churn

Non-employer Firms, 

Tech Intensive

SBIR+STTR, 

2005-2007

Financin

g Events

Associations and 

Organizations

Economic 

Development 

Organizations

Creative 

Class Share

Patents, 2005-2007 1.00

New Products, 2005-

2007

0.32 1.00

Establishment Churn 0.13 -0.04 1.00

Non-employer Firms, 

Tech Intensive

0.42 0.29 0.42 1.00

SBIR+STTR, 2005-2007 0.20 0.13 0.14 0.26 1.00

Financing Events 0.34 0.28 0.11 0.44 0.11 1.00

Associations and 

Organizations

0.10 0.09 -0.11 0.39 0.07 0.07 1.00

Economic 

Development 

Organizations

0.05 0.16 -0.08 0.22 0.06 0.07 0.35 1.00

Creative Class Share 0.50 0.34 0.41 0.78 0.29 0.41 0.26 0.22 1.00
Innovative Metrics for Economic Development 44

Table B- 2: Correlation matrix of non-normalized capacity variables

Patents, 

2005-2007

New Products, 

2005-2007

Establishme

nt Churn

Non-employer Firms, 

Tech Intensive

SBIR+STTR, 

2005-2007

Financing 

Events

Associations and 

Organizations

Economic 

Development 

Organizations

Creative 

Class Share

Patents, 2005-2007 1

New Products, 2005-

2007

0.8012 1

Establishment Churn 0.1808 0.1503 1

Non-employer Firms, 

Tech Intensive

0.7775 0.9146 0.2085 1

SBIR+STTR, 2005-

2007

0.6281 0.6345 0.1538 0.5448 1

Financing Events 0.8425 0.9265 0.2049 0.9722 0.5936 1

Associations and 

Organizations

0.7001 0.8913 0.1662 0.939 0.5631 0.8874 1

Economic 

Development 

Organizations

0.7011 0.8994 0.1679 0.9325 0.6066 0.9067 0.9617 1

Creative Class Share 0.3989 0.3562 0.4081 0.3416 0.3933 0.368 0.3317 0.3437 1
Innovative Metrics for Economic Development 45

Table B- 3: Non-normalized capacity indicators on outcome variables

(1) (2) (3) (4) (5) (6) (7) (8) (9)

All, Earnings All, 

Employment

All, Income Metro, Earnings Metro, 

Employment

Metro, Income Micro, 

Earnings

Micro, 

Employment

Micro, 

Income

Earnings, 2000 -

0.00000142***

-0.00000133* 0.000000544 -0.00000169** -0.00000113 0.00000272 -0.00000214*** -0.000000138 0.00000203

(0.000000510) (0.000000761) (0.00000262) (0.000000672) (0.00000102) (0.00000238) (0.000000763) (0.00000116) (0.00000446)

Employment 2000 2.74e-08** -1.32e-08 6.05e-08 -1.56e-09 -2.48e-08 -9.45e-08 0.000000798* 0.000000547 -0.00000350

(1.38e-08) (2.37e-08) (6.29e-08) (1.19e-08) (2.32e-08) (6.21e-08) (0.000000424) (0.000000620) (0.00000301)

Per Capita Inc 2000 0.000000860* 0.00000143** 0.00000514*** 0.000000923 0.00000133 0.00000352 4.02e-08 -3.75e-08 0.00000219

(0.000000443) (0.000000588) (0.00000186) (0.000000564) (0.000000965) (0.00000222) (0.000000609) (0.000000942) (0.00000172)

Log Pop, 2000 -0.00330* 0.00410 -0.0150** 0.00192 0.00657 0.0345 -0.0392*** -0.0295 -0.0454*

(0.00180) (0.00314) (0.00590) (0.00231) (0.00458) (0.0223) (0.00966) (0.0179) (0.0272)

Unemp Rate, 2000 -0.00238** -0.00335* 0.0129* -0.00205** 0.00128 0.0244* -0.00186 -0.00589** 0.000421

(0.000928) (0.00189) (0.00733) (0.00101) (0.00166) (0.0137) (0.00142) (0.00285) (0.00692)

Mfg Share, 2000 -0.00394 0.108*** -0.0274 0.00271 0.179*** -0.0428 -0.000290 0.0820** 0.0558

(0.0187) (0.0324) (0.134) (0.0264) (0.0527) (0.114) (0.0238) (0.0406) (0.154)

% Bachelors Plus, 2000 0.000706* -0.000897 0.00453** 0.000733 0.00124 0.0108* 0.000264 -0.00218** 0.00205

(0.000409) (0.000647) (0.00227) (0.000517) (0.000915) (0.00576) (0.000570) (0.000967) (0.00197)

Patents, 05-07 0.00000487*** 9.61e-08 0.000000164 0.00000477*** 0.000000445 -0.00000144 0.000230*** -0.000215* 0.000259

(0.00000101) (0.00000165) (0.00000441) (0.00000113) (0.00000173) (0.00000497) (0.0000820) (0.000114) (0.000194)

New Products, Sum 05-

07

-0.000251*** -0.000198* 0.000978*** -0.000169** -0.0000970 0.00143*** -0.000199 -0.00150 -0.00290

(0.0000698) (0.000114) (0.000286) (0.0000680) (0.000105) (0.000383) (0.00132) (0.00224) (0.00348)

Establishment Churn -0.208*** 0.518*** -0.138 -0.252*** 0.895*** 0.132 -0.160** 0.234** -0.414

(0.0461) (0.0803) (0.123) (0.0730) (0.124) (0.256) (0.0657) (0.102) (0.270)

Non-employer Firms -

0.00000165***

-0.000000275 -0.00000125 -0.00000104*** -0.000000447 0.000000580 -0.0000145* 0.0000317** 0.000129*

(0.000000296) (0.000000556) (0.00000123) (0.000000240) (0.000000449) (0.00000112) (0.00000833) (0.0000128) (0.0000704)

SBIR + STTR, 05-07 0.0000126 -0.0000204 -0.000160*** -0.000000677 -0.0000208 -0.000196*** 0.000397* -0.000637** -0.000852

(0.00000831) (0.0000176) (0.0000611) (0.00000723) (0.0000169) (0.0000626) (0.000213) (0.000257) (0.00142)

Financing Events 0.000229*** 0.000236*** 0.000249 0.000217*** 0.000235*** 0.000303 0.00202 0.00441** -0.00645

(0.0000477) (0.0000734) (0.000214) (0.0000399) (0.0000651) (0.000198) (0.00129) (0.00200) (0.00674)
Innovative Metrics for Economic Development 46

(1) (2) (3) (4) (5) (6) (7) (8) (9)

All, Earnings All, 

Employment

All, Income Metro, Earnings Metro, 

Employment

Metro, Income Micro, 

Earnings

Micro, 

Employment

Micro, 

Income

Assoc./Orgs (NAICS 

813)

0.0000219*** 0.0000196** 0.0000425** 0.0000172*** 0.0000184*** 0.0000322 0.000341*** 0.000179 0.000730*

(0.00000525) (0.00000848) (0.0000195) (0.00000354) (0.00000670) (0.0000219) (0.0000892) (0.000137) (0.000414)

EDO -0.000203*** -0.000274*** -0.00132*** -0.000149*** -0.000189*** -0.00108*** -0.00134 -0.00638*** -0.00434

(0.0000371) (0.0000774) (0.000199) (0.0000294) (0.0000610) (0.000231) (0.000989) (0.00166) (0.00307)

Creative Class Share -0.0376 0.334*** -0.522* -0.0158 -0.0482 -1.752** -0.0491 0.508*** -0.257

(0.0650) (0.106) (0.300) (0.0942) (0.167) (0.879) (0.0828) (0.139) (0.223)

Constant 0.201*** -0.149*** 0.155* 0.147*** -0.243*** -0.429 0.597*** 0.251 0.614**

(0.0226) (0.0431) (0.0901) (0.0306) (0.0513) (0.337) (0.102) (0.202) (0.238)

Observations 869 869 866 380 380 378 489 489 488

Adjusted R2 0.096 0.180 0.016 0.183 0.309 0.033 0.122 0.138 0.030

Standard errors clustered at the CBSA level.

* p < 0.10, ** p < 0.05, *** p < 0.01
Innovative Metrics for Economic Development 47

Table B- 4: Networking capacity and capacity variable interaction on long-term earnings growth tests

(1) (2) (3) (4) (5)

Patents Interaction New Product Interaction Establishment Churn 

Interaction

Non-employer Interaction Financing Event 

Interaction

2000 Earnings per Capita 0.0205*** 0.0205*** 0.0200*** 0.0207*** 0.0204***

(0.00457) (0.00457) (0.00456) (0.00460) (0.00456)

Employed Share of Pop 0.0222 0.0223 0.0245 0.0219 0.0225

(0.0212) (0.0212) (0.0209) (0.0212) (0.0211)

Unemployment Rate, 2000 -0.00207** -0.00209** -0.00198** -0.00206** -0.00208**

(0.000871) (0.000872) (0.000861) (0.000872) (0.000871)

Mfg Share of Emp, 2000 -0.0188 -0.0194 -0.0172 -0.0180 -0.0195

(0.0182) (0.0182) (0.0182) (0.0183) (0.0183)

% Bachelors or >, 2000 -0.0000689 -0.0000706 -0.0000524 -0.0000652 -0.0000677

(0.000324) (0.000324) (0.000322) (0.000324) (0.000325)

Patents, 05-07, Norm 0.370 0.266 0.267 0.264 0.262

(0.237) (0.186) (0.184) (0.186) (0.187)

New Prod, 05-07, Norm -0.373 1.444 -1.113 -0.429 -0.249

(5.848) (6.447) (5.815) (5.843) (5.851)

Churn, Select Naics -0.147*** -0.148*** -0.117** -0.147*** -0.148***

(0.0468) (0.0468) (0.0508) (0.0467) (0.0473)

Normalized Non-Emp Firms, HT, 2006 -0.00346 -0.00185 0.00417 0.0131 -0.00165

(0.0466) (0.0464) (0.0459) (0.0494) (0.0464)

SBIR and STTR Grants Combined, Sum 

05-07, Normalized

0.433 0.404 0.387 0.436 0.363

(0.705) (0.706) (0.703) (0.703) (0.705)

Normalized Financing Events, 2006 16.00*** 15.88*** 16.13*** 15.81*** 15.75***

(4.795) (4.783) (4.796) (4.767) (4.938)

Networking Capacity 0.00227*** 0.00205** 0.00931* 0.00316* 0.00173

(0.000829) (0.000905) (0.00479) (0.00173) (0.00114)

Networked Patents -0.0695

(0.0612)
Innovative Metrics for Economic Development 48

Networked New Products -1.350

(2.291)

Networked Churn -0.0411*

(0.0246)

Networked Non-employer Firms -0.0114

(0.0116)

Networked Financing 0.132

(1.690)

Constant 0.118*** 0.119*** 0.111*** 0.117*** 0.119***

(0.0159) (0.0159) (0.0163) (0.0161) (0.0161)

Observations 903 903 903 903 903

Adjusted R2 0.109 0.108 0.111 0.109 0.108
SRI International | page 49

Table B- 5: Capacity variables on earnings growth with different definitions of the networking variable

(1) (2) (3)

Orgs*EDO Orgs*EDA EDA*EDO

2000 Earnings per Capita 0.0189*** 0.0213*** 0.0203***

(0.00452) (0.00464) (0.00464)

Employed Share of Pop 0.0275 0.0204 0.0250

(0.0215) (0.0211) (0.0212)

Unemployment Rate, 2000 -0.00196** -0.00218** -0.00200**

(0.000866) (0.000879) (0.000871)

Mfg Share of Emp, 2000 -0.0245 -0.0173 -0.0223

(0.0187) (0.0181) (0.0183)

% Bachelors or >, 2000 -0.0000194 -0.0000748 -0.0000628

(0.000325) (0.000325) (0.000324)

Patents, 05-07, Norm 0.00262 0.00248 0.00262

(0.00187) (0.00187) (0.00185)

New Prod, 05-07, Norm 0.00217 -0.00201 0.00110

(0.0589) (0.0589) (0.0586)

Churn, Select Naics -0.160*** -0.148*** -0.160***

(0.0489) (0.0466) (0.0466)

Normalized Non-Emp Firms, HT, 

2006

0.0000761 0.00000151 0.000111

(0.000485) (0.000459) (0.000458)

SBIR and STTR Grants Combined, 

Sum 05-07, Normalized

0.00401 0.00368 0.00376

(0.00697) (0.00715) (0.00704)

Normalized Financing Events, 2006 0.154*** 0.160*** 0.154***

(0.0480) (0.0476) (0.0476)

Networking Indicator: Orgs*EDO 0.000351

(0.000908)

Networking Indicator: Orgs*EDA 0.00364***

(0.00114)

Networking Indicator: EDA*EDO 0.00192

(0.00117)

Constant 0.120*** 0.119*** 0.120***

(0.0163) (0.0159) (0.0160)

Observations 903 903 903

Adjusted R2 0.103 0.112 0.105
SRI International | page 50

Appendix C: Selected Literature 

Literature for Capacity Outcomes

Cohen, Susan, 2013, “What Do Accelerators Do? Insights from Incubators and Angels,” Innovations, 8:3/4.

Colbert, C., Adkins, D., Wolfe, C., and LaPan, K. (2010). Best Practices in Action: Guidelines for Implementing FirstClass Business Incubation Programs, Revised 2nd Edition. Athens, OH: National Business Incubation 

Association.

Fehder, D, Y. V. Hochberg, 2014, “Accelerators and the Regional Supply of Venture Capital Investment” (Social 

Science Research Network) http://dx.doi.org/10.2139/ssrn.2518668.

Feld, Brad, 2012, Startup Communities: Building an Entrepreneurial Ecosystem in Your City, (Wiley).

Grimaldi, R., & Grandi, A., 2005, Business incubators and new venture creation: an assessment of incubating models. 

Technovation, 25(2), 111-121.

Hackett, S. & Dilts, D. M., 2004, A systematic review of business incubation research. Journal of Technology Transfer, 

29(1), 55-82.

Hansen, M.T., Chesbrough, H., Norhoa, N., & Sull, D. Networked incubators: Hothouses of the new economy. 

Harvard Business Review, September-October 2000, 74-84.

Lewis, David A., Elsie Harper-Anderson, and Lawrence A. Molnar, “Incubating Success: Incubation Best Practices 

That Lead to Successful New Ventures” (2011, Institute for Research on Labor, Employment, and the 

Economy, Ann Arbor, Michigan).

SRI International, 2015, “Measuring the Impact of USTAR’s Technology Outreach Innovation Program: Baseline 

Impact and Methodology”.

SRI International, 2014, “The EDA i6 Challenge Program: Assessment & Metrics”.

Tornatzky, L., Sherman, H., & Adkins, D., 2002, “A national benchmarking analysis of technology business

incubator performance and practices” (Athens, OH: National Business Incubation Association).
SRI International | page 51

Literature for Realized Outcomes

Acs, Zoltan, 2006, "How is entrepreneurship good for economic growth?" Innovations 1.1: 97-107.

Glaeser, E.L. and Kerr, W., 2009, “Local Industrial Conditions and Entrepreneurship: How Much of the Spatial 

Distribution Can We Explain?” Journal of Economics and Management Strategy, 18.3: 623-663.

Bauer, P.W., Schweitzer, M.E., and Shane, S.A., 2012, “Knowledge Matters: The Long Run determinants of State 

Income Growth.” Journal of Regional Science, 52: 240-255.

Feldman, M.P. and Zoller, T., 2012, “Dealmakers in Place: Social Capital Connections in Regional Entrepreneurial 

Economies” Regional Studies, 46.1: 23-37.

Mueller, Pamela, 2007, "Exploiting entrepreneurial opportunities: The impact of entrepreneurship on growth." 

Small Business Economics 28.4: 355-362.

Fazio, Catherine, Jorge Guzman, Fiona Murray and Scott Stern, 2016, “A New View of the Skew: A Quantitative 

Assessment of the Quality of American Entrepreneurship”.

Motoyama, Yasuyuki, and Brian Danley, 2012, "The ascent of America’s high-growth companies: An analysis of the 

geography of entrepreneurship." (Kauffman Foundation Research Series).

Fritsch, Michael, 2002, "Measuring the quality of regional innovation systems: a knowledge production function 

approach." International Regional Science Review 25.1.

Mueller, Pamela, 2007, "Exploiting entrepreneurial opportunities: The impact of entrepreneurship on growth." 

Small Business Economics 28.4.

Lever, William F. 2002 "Correlating the knowledge-base of cities with economic growth." Urban studies 39.5-6: 

859-870.

Audretsch and Feldman, 1996, “R&D Spillovers and the Geography of Innovation and Production.” The American 

Economic Review, 86(3), 630-640.
SRI International | page 52

Appendix D: Survey Results

Presented here are summary statistics from the surveys SRI fielded. The i6 Grantee and Client Surveys were fielded 

in February and March 2017, and the SFS Grantee survey was fielded in April and May 2017. All surveys were fielded 

using the LimeSurvey online platform.

Table D-1: Summarized results from the i6 grantee program outputs survey

i6 Grantee Survey

Question Aggregated 

response

Total number of clients or beneficiaries served over the past year through i6 program work 1952

Number of grants that supported events, networking, and referrals activity 21

Total number of educational, training, and/or networking events 713

Number of participants 8773

Total number of Conferences, Showcases, and/or Exhibitions and Participants 73

Number of Participants 6431

Total number of client referrals to a technical expert, business contact, investor, etc. for 

required services and/or support 

2465

Number of grants that supported mentoring, coaching, and technical assistance activities 25

Total number of people who attended boot camps, accelerators, or mentoring and coaching 

programs 

2616

Total number of clients who received product development, supply chain, or other 

operational assistance

521

Total number of clients who were provided with marketing, sales, or market research or 

advice

620

Total number of clients who were provided with exporting assistance or advice 81

Number of grants that supported the acquisition of facilities and/or equipment 4

Total amount of new space (in sq. ft.) that was developed, purchased, or leased as a result 

of grants 

19200
SRI International | page 53

Total value ($USD) of new equipment that was purchased or leased as a result of grants $527,300

Number of grants that supported R&D and technology development activities 17

Total number of new joint research projects facilitated between clients or beneficiaries and 

university partners 

50

Total number of clients or beneficiaries assisted with technology commercialization, 

licensing, patenting, or other regulatory or government 

193

Number of grants that supported financing activities 7

Total number of clients assisted in obtaining angel, seed, or venture capital funding 93

Total number of clients assisted with a grant proposal or award application 103

Total number of clients assisted with obtaining other types of funding (e.g. contracts, loans, 

etc.)

70

Number of angel, venture capital, seed funding competitions or events supported 24

Number of grants that supported capacity-building activities 16

Total number of hours of outside expertise or professional services, relevant either to your 

organization's work or your clients' work obtained as a result of support from grants

7821
SRI International | page 54

Table D- 2: Summarized results from the i6 client capacity outcomes survey

i6 Client Survey

Question Aggregated 

Response

Total number of respondents 129

Total number of clients who said grant was intended to improve innovation, tech transfer, 

or commercialization capabilities

75

Average rating of improvement of innovation, tech transfer or commercialization on a scale 

from 1 to 5

3.99

Total number of clients who said they increased the TRL of a product 56

Average increase in TRL 3.39

Total number of new technologies that were licensed or brought to market as a result of 

the services received from grantee

44

Total number of patents or trademarks obtained as a result of the grant services 18

Total number of FDA or other government approvals that were obtained as a result of the 

services received

4

Total number of clients who reported that services were intended to improve products and

production processes and/or business capacities

60

Average client rating of improvement of product and production processes and/or business 

capacities on a scale from 1 to 5

4.02

Total number of days spent working on new or improved designs for products, processes, 

or services as a result of the services received

4,096

Total number of cost reductions, operational efficiencies, and/or quality improvements 

made as a result of the services

20,309

Total cost savings ($USD) from these improvements 335,920

Total number of environmental and/or energy efficiency improvements made as a result of 

the services

7

Total number of clients reporting that the services were intended to improve your human 

capital and/or workforce

29

Average rating of improvement to human capital and/or workforce capacities on a scale 

from 1 to 5

3.52

Total hours spent by employees or trainees on skill development as a result of the services 3,111

Total hours spent by employees or trainees on entrepreneurship or leadership programs as 

a result of the services

8,140

Total number of employees who completed technology or cluster-relevant degrees or 

certificates as a result of the services 

111

Total number of new employees (full-time, part-time, and/or 12-month contract 

employees) hired as a result of the services 

32
SRI International | page 55

Total number of clients reporting that the services received were intended to improve 

access to new markets and business networks

65

Average rating of improvement of access to markets and business networks on a scale from 

1 to 5

3.77

Total number of new customers gained as a result of the services 3,660

Total number of new professional or business relationships formed as a result of the 

services 

767

Total number of clients reporting that new sales, marketing, or branding strategies or 

materials were developed as a result of the services 

46

Total number of clients reporting that new export strategies were developed as a result of 

the services 

15

Total number of clients reporting that the services received were intended to improve your 

access to financing and investment

51

Average rating of improvement of access to financing and investment on a scale from 1 to 5 3.53

Total number of clients reporting that they received funds from seed, angel, or venture 

capital deals as a result of the services

13

Total amount of funding received by clients from seed, angel, venture capital deals $337,000

Total number of clients reporting that they obtained loan funding as a result of the services 2

Total amount of loan funding obtained by clients $410,000

Total number of clients reporting that they received government funding (grants, contracts, 

SBIR, STTR)

12

Total amount of funding received by clients from government funding $257,000

Table D- 3: Average amount of funding obtained by i6 clients, by source

Average amount of funding obtained by clients

Angel, seed, or venture $25,923

Loan $205,000

Government funding $21,417
SRI International | page 56

Table D- 4: Summarized results from the SFS grantee program outputs survey

SFS Grantee Survey

Question Aggregated 

response

Total number of clients or beneficiaries served over the past year through i6 program 

work.

1667

Number of grants that supported events, networking, and referrals activity 13

Total number of conferences, showcases, exhibitions, networking, or outreach events 95

Number of participants 1698

Total number of client referrals to a technical expert, business contact, investor, etc. for 

required services and/or support 

713

Number of grants that supported mentoring, coaching, and training activities 8

Total number of training or skill assistance sessions 94

Total number of people who attended training or skill assistance 2543

Number of grants that supported the acquisition of facilities and/or equipment 1

Total amount of new space (in sq. ft.) that was developed, purchased, or leased as a 

result of grants 

1000

Total value ($USD) of new equipment that was purchased or leased as a result of grants $0

Number of grants that supported R&D and technology development activities 3

Total number of new joint research projects facilitated between clients or beneficiaries 

and university partners 

1

Total number of clients or beneficiaries assisted with technology commercialization, 

licensing, patenting, or other regulatory or government 

15

Number of grants that supported financing activities 10

Total number of clients assisted in obtaining angel funding 37

Total amount of angel funding obtained by clients $6,270,000

Total number of clients assisted in obtaining seed funding 28

Total amount of seed funding obtained by clients $14,950,450

Total number of clients assisted in obtaining venture capital funding 10
SRI International | page 57

Total amount of venture capital funding obtained by clients $200,925,000

Total number of clients assisted in obtaining loan funding 13

Total amount of loan funding obtained by clients $2,725,000

Total number of clients assisted with a grant or award proposal 9

Total amount of grant or award funding obtained by clients $800,000

Total number of clients assisted with obtaining other types of funding (e.g. contracts, 

loans, etc.)

4

Number of angel, venture capital, seed or loan funding competitions or events supported 16

Number of grants that supported planning & institutional development activities 6

Total number of hours of outside expertise or professional services, relevant either to 

your organization's work or your clients' work obtained as a result of support from grants

2050

Total number of hours spent researching funding resources and developing a funding 

plan

4370

Total number of hours spent on conducting grant-related research, producing plans, 

reports or tools, and/or on other coordination activities over the past year

1335

Table D- 5: Average amount of funding obtained by SFS grantee clients, by source

Average amount of funding obtained by SFS grantee clients

Angel $169,459

Seed $533,945

Venture $20,092,500

Loan $209,615

Grant or other $88,889
SRI International | page 58

Table D- 6: Summary of combined SFS and i6 program output survey responses

SFS and i6 Grantee Surveys, combined

Question Aggregated 

response

Total number of clients or beneficiaries served over the past year through i6 program

work

3,619

Number of grants that supported events, networking, and referrals activity 34

Total number of educational, training, and/or networking events 881

Number of participants 16,902

Total number of client referrals to a technical expert, business contact, investor, etc. for 

required services and/or support 

3,178

Number of grants that supported mentoring, coaching, and technical assistance 

activities

33

Total number of people who attended boot camps, accelerators, or mentoring and 

coaching programs 

5159

Number of grants that supported the acquisition of facilities and/or equipment 5

Total amount of new space (in sq. ft.) that was developed, purchased, or leased as a 

result of grants 

20,200

Total value ($USD) of new equipment that was purchased or leased as a result of grants $527,300

Number of grants that supported R&D and technology development activities 20

Total number of new joint research projects facilitated between clients or beneficiaries 

and university partners 

51

Total number of clients or beneficiaries assisted with technology commercialization, 

licensing, patenting, or other regulatory or government 

208

Number of grants that supported financing activities 17

Total number of clients assisted in obtaining angel, seed, or venture capital funding 168

Total number of clients assisted with a grant proposal or award application 112

Number of angel, venture capital, seed funding competitions or events supported 40

Number of grants that supported capacity-building activities 22

Total number of hours of outside expertise or professional services, relevant either to 

your organization's work or your clients' work obtained as a result of support from 

grants

9,871
SRI International | page 59

Appendix E: Cited References

Bauer, P.W., Schweitzer, M.E., and Shane, S.A. 2012, “Knowledge Matters: The Long Run determinants of State 

Income Growth.” Journal of Regional Science, 52: 240-255.

Crescenzi, R. 2005. “Innovation and Regional Growth in the Enlarged Europe: The Role of Local Innovative 

Capabilities, Peripherality, and Education.” Growth and Change, Vol. 36 No. 4: 471–507.

Evans, P., and Karras, G. 1994. “Are Government Activities Productive? Evidence from a Panel of U.S. States.” The 

Review of Economics and Statistics, Vol. 76, No. 1: 1-11.

Feldman, M.P. and Zoller, T. 2012. “Dealmakers in Place: Social Capital Connections in Regional Entrepreneurial 

Economies.” Regional Studies, Vol. 46, No. 1: 23-37.

Florida, Richard. 2012. The Rise of the Creative Class. Basic Books.

Florida, R, Mellander, C., Stolarik, and Ross, A. 2010. “Cities, Skills and Wages.” Institute for Competitiveness & 

Prosperity, Working Paper. 

Gicheva, Link. 2016. “On the Economic Performance of Nascent Entrepreneurs.” European Economic Review Vol. 

86: 109-117.

Glaeser, E.L. and Kerr, W. 2009. “Local Industrial Conditions and Entrepreneurship: How Much of the Spatial 

Distribution Can We Explain?” Journal of Economics and Management Strategy, Vol. 18, No. 3: 623-663.

Glaeser, E.L. and Saiz, A. 2003. “The Rise of the Skilled City.” Federal Reserve Bank of Philadelphia, Working Paper 

04-2. 

Hackett, S. & Dilts, D. M. 2004. “A systematic review of business incubation research.” Journal of Technology 

Transfer, Vol. 29, No. 1: 55-82.

Haltiwanger, J., Hyatt, H., McEntarfer, E., Sousa, L. 2012. “Job Creation, Working Churning, and Wages at Young 

Businesses.” Business Dynamics Statistics Breifing, Kauffman Foundation. 

Information Technology and Innovation Foundation’s 2014 State New Economy.

Lewis, David A., Elsie Harper-Anderson, and Lawrence A. Molnar. 2011. “Incubating Success: Incubation Best 

Practices That Lead to Successful New Ventures” Institute for Research on Labor, Employment, and the 

Economy, Ann Arbor, Michigan.

Mueller, Pamela, 2007, "Exploiting entrepreneurial opportunities: The impact of entrepreneurship on growth." 

Small Business Economics, Vol. 28, No. 4: 355-362.

National Bureau of Economic Research. 2012. “US Business Cycle Expansions and Contractions.” 

http://www.nber.org/cycles.html. Accessed September 2017. 

Office of Management and Budget, Memorandum to the Heads of Departments and Agencies, July 26 2013, “Next 

Steps in the Evidence and Innovation Agenda.”

Sterlacchini, A. 2008. “R&D, Higher education and regional growth: Uneven linkages among European regions.” 

Research Policy Vol. 37, No. 6-7: 1096–1107.

SRI International. 2014. The EDA JIAC Program: Assessment & Metrics.
SRI International | page 60

SRI International. 2014. The EDA i6 Challenge Program: Assessment & Metrics.

SRI International and UNC-Chapel Hill. 2014. The 21st Century EDA TAAF Program: Assessment & Metrics.

SRI International. 2014. Building and Using a New EDA Evaluation System: Final Report.

U.S. Census Bureau. 2012. “Geographic Terms and Concepts – Core Based Statistical Areas and Related Statistical 

Areas.” https://www.census.gov/geo/reference/gtc/gtc_cbsa.html. Accessed September 2017. 

Weissbourd, R & C. Berry. 2004. The Changing Dynamics of Urban America, R.W. Ventures.

Wooldrige, Jeffery. 2006. Introductory Econometrics: A Modern Approach. 3rd Edition, Thomson South-Western. 

Wooldrige, Jeffery. 2006. Introductory Econometrics: A Modern Approach. 3rd Edition, Thomson South-Western. 
SRI International | page 61

Appendix F: Survey Challenges and 

Lessons Learned for EDA

This appendix is specifically intended for EDA staff.

While ultimately SRI obtained approximately a 76% response rate for both grantee surveys, which is a very 

respectable response rate, we had hoped to collect more data, both from grantees and particularly from their 

clients. We faced several challenges in fielding this survey, due largely to the dual-level design and to our position 

as a third-party entity removed from the EDA-grantee ecosystem. The two primary challenges we faced are detailed 

below, along with recommendations for how EDA can mitigate these challenges in their implementation of the 

surveys.

Survey Fatigue

SRI received feedback from several clients voicing concerns about the apparent redundancy of our survey, as they 

had recently completed EDA’s i6-specific required reporting. Some clients wondered why they had to complete a 

survey that asked very similar questions to the one they had just completed for EDA, while others said they simply 

did not have the time to complete another survey. Some grantees also asked to delay distributing the client survey, 

as they had recently fielded their own client surveys, and did not want to overburden or cause survey fatigue among 

their clientele. Ultimately, this survey fatigue contributed to a handful of grantees not completing the survey.

EDA Mitigation:

Coordination with other EDA reporting: One way EDA will be better positioned to avoid or mitigate the risk of survey 

fatigue is by ensuring they coordinate the distribution of this survey with other EDA surveys and required reporting. 

By working internally to identify all of the necessary reporting they will be asking grantees to complete, EDA staff 

can determine a reporting schedule that obtains all of the necessary data while spreading out reporting deadlines 

such that grantees won’t feel overburdened or fatigued. 

Providing advance notice: While SRI worked with EDA staff to notify grantees that the survey was soon to be fielded, 

we were unable to provide grantees with substantial advance notice about the survey, and many had not budgeted 

time for it in their schedules. Since EDA will be building the survey into all of their grants and informing grantees 

about the reporting requirement through the special terms and conditions of the awards, they will be able to 

highlight to clients the need to budget appropriate time to respond to the survey. EDA staff should provide very 

clear timelines for the survey distribution to grantees, and reiterate the need for grantees to provide similar 

guidance to their clients, in order to facilitate appropriate planning. 

Difficulty Reaching Clients

A second challenge encountered by SRI was difficulty in accessing clients of grantees to distribute surveys and follow 

up to encourage responses. As was mentioned earlier, for the piloting of the survey instrument, SRI distributed the 

client survey and an accompanying email text to i6 grantee organizations and asked them to forward it to anyone 

whom they had served over the past year. This inevitably caused some complications in the fielding of the survey, 

and the inability to directly access clients prevented SRI from following up with them to encourage a better response 

rate. Additionally, it appears that many grantees did not forward the survey on to their clients, so SRI received a 
SRI International | page 62

limited (and possibly biased) sample of client responses. However, EDA is in a better position than was SRI to 

communicate to grantees the importance of forwarding the surveys on to clients, and to hold them accountable for 

doing so. Additionally, as the survey is fielded year over year, the process will become more familiar to grantees 

and the likelihood that they will distribute the surveys as required will increase. 

EDA Mitigation

Modify implementation to distribute surveys directly to clients: Should EDA desire, they may decide to alter the 

implementation of the client capacity outcomes survey to eliminate the need to rely on grantees to distribute the 

surveys. This could be done by either collecting client contact information from grantees and distributing the survey 

directly to the clients, or by sending the capacity outcomes survey to grantees and requiring them to collect the 

necessary information from their clients (however they choose to do so). Either one of these approaches could 

reduce the complexity of the survey fielding, and hopefully improve response rate. 

Providing advance notice and setting clear expectations: SRI was at a disadvantage in trying to distribute the surveys 

to clients, as a) we were an unknown entity to most grantees, and b) we were coming in largely unannounced, and 

asking grantees to take on the responsibility of distributing a survey to their clients. EDA, as the provider of funds 

to grantees, has more credibility and authority to encourage grantees to forward surveys (or collect the required 

metrics) to their clients. To ensure the most effective survey implementation possible, it is critical that EDA provide 

substantial advance notice to grantees of the reporting process and timelines, so that they can plan accordingly. It 

is also critical for EDA staff to be exceptionally clear about their expectations that grantees fully participate in the 

process and assist with the client capacity outcomes survey as requested. Setting this expectation early and 

reiterating it throughout the life of the grant will help grantees to accept and embrace their role in the data 

collection process.