Established Program to Stimulate Competitive Research (EPSCoR): Background and Selected Issues
Released: 2017-01-12T00:00:00

Introduction

The Established Program to Stimulate Competitive Research (EPSCoR)—originally named the E xperimental Program to Stimulate Competitive Research (EPSCoR) —is a set-aside funding program that began as an effort to avoid an "undue concentration" of federal research funds by providing competitive grant opportunities to states that historically received little federal research and development (R&D) funding. The National Science Foundation's (NSF's) governing board first established the program in 1978 by resolution, and EPSCoR was formally established in statute in 1988. The program was created to increase research and infrastructure capacity, thereby improving the ability of institutions in EPSCoR states to compete for non-set-aside federal R&D funding. EPSCoR is distinct from other NSF programs in this geographically targeted approach.

Since the first program began at NSF, EPSCoR and EPSCoR-like programs have been established at other federal agencies, with active programs at the Department of Energy (DOE), National Aeronautics and Space Administration (NASA), U.S. Department of Agriculture (USDA), and National Institutes of Health (NIH). In 1992, the EPSCoR Interagency Coordinating Committee (EICC) was established through a Memorandum of Understanding to improve coordination among the programs. NSF serves as the Chair and Executive Secretary of the EICC, which was formally authorized in statute by the America COMPETES Reauthorization Act of 2010 ( P.L. 111-358 ).

EPSCoR has long been a program of interest for Congress. Stakeholders and researchers have noted that EPSCoR—as established within NSF—was initially intended to be a limited-duration catalyst to support the development of research capacity and has evolved into a long-term program that also supports education, diversity, and economic development goals. Some Members have expressed interest in supporting EPSCoR as a long-term program, while others have expressed interest in discontinuing its funding.

This report provides an overview of the active EPSCoR programs, including eligibility criteria, operations and funding information, findings and recommendations from program assessments, programmatic and policy issues, and recent congressional activity. This report focuses on the EPSCoR program at NSF, but also provides information about, and comparisons to and among, the EPSCoR programs at other federal agencies.

Eligibility Criteria and Operations

EPSCoR eligibility criteria and determinations vary by agency. EPSCoR programs at DOE and NASA follow the NSF's eligibility determinations, while USDA and NIH have established their own criteria. Table 1 lists FY2016 eligible jurisdictions (including eligible states, territories, and the District of Columbia) for agencies with active programs. The following sections describe agency-specific operations and eligibility criteria in further detail.

National Science Foundation

NSF's EPSCoR program was established in 1978 by a resolution of the National Science Board (NSB). Congress directed initial appropriations for the program in FY1979. NSF's EPSCoR was formally established in statute in 1988 to assist states that "historically have received relatively little Federal [R&D] funding" and have "demonstrated a commitment to develop their research bases and improve science and engineering research and education." The America COMPETES Reauthorization Act of 2010 directed NSF to continue the program "with the objective of helping eligible States to develop the research infrastructure that will make them more competitive for Foundation and other Federal research funding." According to NSF, the overall program mission is

to advance excellence in science and engineering research and education in order to achieve sustainable increases in research, education, and training capacity and competitiveness that will enable EPSCoR jurisdictions to have increased engagement in areas supported by the NSF.

NSF states that EPSCoR's goals are

to provide strategic programs and opportunities for EPSCoR participants that stimulate sustainable improvements in their R&D capacity and competitiveness; [and] to advance science and engineering capabilities in EPSCoR jurisdictions for discovery, innovation and overall knowledge-based prosperity.

EPSCoR program objectives further include broadening participation of diverse groups in EPSCoR projects and establishing science, technology, engineering, and math (STEM) education, training, and professional development opportunities that advance research and workforce development needs identified by each jurisdiction.

NSF's EPSCoR program is broadly set up as federal-jurisdiction partnerships. To participate in the EPSCoR program, an eligible jurisdiction is required to form its own EPSCoR steering committee and to develop a science and technology (S&T) plan specific to the jurisdiction's needs and goals. Each steering committee is expected to undertake "a recent comprehensive analysis of the strengths, barriers, and opportunities for further development of its institutions in support of overall objectives in research, education, and innovation." Through these activities, steering committees work closely with partners in academia, government, and the private sector to build statewide networks.

The funding awards structure for the NSF EPSCoR program has changed over time. In 1980, NSF awarded the first EPSCoR planning grants to seven states to support establishment of state steering committees, identify barriers to research competitiveness, and suggest possible remedies. Building from that work, five states subsequently developed successful research grant proposals and received five years of initial funding, primarily to support individual investigators. In the 1990s, award levels increased, and the grant focus changed to support for research clusters and statewide proposals for infrastructure development. In the 2000s, NSF's EPSCoR program was moved to the Office of Integrative Activities (OIA) and developed to include the three current investment components: Research Infrastructure Improvement (RII) awards, co-funding, and outreach/workshops.

Research Infrastructure Improvement . In FY2016, the RII component included three tracks:

RII Track-1 awards provide up to $20 million total over a five-year period to support physical, human, and cyberinfrastructure improvements critical to a jurisdiction's science and technology (S&T) plan (e.g., acquisition of research equipment, establishment of university-private sector partnerships). RII Track-2 (Focused EPSCoR Collaborations [FEC]) awards provide up to $1.5 million per year for up to four years to support the establishment of interjurisdictional teams of investigators from at least two RII-eligible jurisdictions working collaboratively on NSF priority research topics. FY2016 awards supported EPSCoR proposals on two topics: (1) Understanding the Brain, and (2) Sustainable, Food, Energy, and Water Systems. RII Track-3 (Building Diverse Communities) awards provide up to $750,000 per award for an award period of up to five years, with projects serving as a "testbed" for building approaches to broaden participation of underrepresented groups in STEM. Track-3 proposals were first called for in FY2013 as a pilot program. Initial funding supported five proposals including projects to develop and implement chemistry coursework in tribal colleges and test cyber-learning methods for STEM education in rural middle schools.

For FY2017, NSF announced an RII Track-4 (EPSCoR Research Fellows) component. These awards provide up to $300,000 over a two-year period for non-tenured investigators to "further develop their individual research potential through extended collaborative visits to the nation's premier private, governmental, or academic research centers." According to the program solicitation, through activities such as accessing unique equipment and developing research collaborations, fellows are expected to improve their individual research competitiveness, and, in turn, to enhance the research capacity of their institutions and jurisdictions.

Specific criteria and proposal requirements vary by track. For example, for RII Track-1 awards, cost-sharing is required and an eligible jurisdiction may only submit one proposal. For RII Track-2 and -3 awards, no cost-sharing requirement is listed in the program solicitation and limits on proposal submissions are less restrictive. EPSCoR proposal evaluations follow the merit-based, peer-review process used for the vast majority of NSF's competitive awards.

A jurisdiction becomes eligible to participate in the EPSCoR program "if their most recent 3-year funding level of NSF research support is equal to or less than 0.75% of the total NSF Research and Related Activities (RRA) budget." This criterion is commonly used when referring to a jurisdiction as EPSCoR-eligible. Figure 1 shows EPSCoR-eligible jurisdictions for FY2016, as well as year of entry into the program. Table A-1 shows NSF RRA funding received by each jurisdiction for FY2013-FY2015, the three-year total by jurisdiction, and the percentage each three-year total accounts for of NSF's total RRA funding for that period.

Co-funding . The co-funding component is not a grant program, but rather an internal, joint funding mechanism, in which project funding is provided by both EPSCoR and another NSF directorate or office. Co-funding awards provide combined support for proposals submitted to NSF's non-EPSCoR awards competitions by investigators in EPSCoR jurisdictions. Proposals deemed meritorious and recommended for funding—but not able to be funded otherwise due to budget limitations—may be evaluated for EPSCoR co-funding consideration. States that have lost eligibility for the RII component remain eligible for three years for the co-funding component.

Outreach/Workshops . The outreach/workshop component provides funding for workshops, conferences, and other community-based activities designed to explore new scientific areas, share best practices, and build capacity. Program administrators expect workshops to be multijurisdictional and of interest to the broad EPSCoR community. This funding component also supports travel costs associated with connecting NSF staff across directorates and offices with research and education communities in EPSCoR jurisdictions. States that have lost eligibility for the RII component remain eligible for three years for the outreach/workshop component.

Department of Energy

The Energy Policy Act of 1992 directed DOE to operate an EPSCoR program and established objectives for the program. Located within its Office of Science (OS), DOE's EPSCoR is a merit-based program that works to support basic and applied energy-related research and development across many interdisciplinary program areas. Program priorities include increasing the number of professionals in energy-related fields and building relationships between professionals in EPSCoR jurisdictions and scientists from DOE facilities.

The DOE EPSCoR program follows NSF's EPSCoR eligibility determinations; states that are eligible for the NSF EPSCoR RII component are eligible for DOE's program. In other words, eligib ility for the DOE EPSCoR is based on recent funding received by each state from NSF, rather than from DOE.

The program offers three types of funding opportunities: EPSCoR Implementation Grants, EPSCoR-State/National Laboratory Partnership Grants, and DOE OS Early Career Awards. DOE award components are similar in structure and function to those at NSF, with some differences. For example, DOE Implementation Grants—which support research infrastructure improvement similar to NSF RII awards—do not require matching funds, nor is there a limit on the number of active grants per jurisdiction.

National Aeronautics and Space Administration

In 1992, the Experimental Program to Stimulate Competitive Research on Space and Aeronautics Act established the NASA EPSCoR program, directing NASA to conduct a merit-based grant competition among states eligible for NSF's EPSCoR program. The program goals focus on enabling jurisdictions to develop long-term, self-sustaining, nationally competitive aerospace and related research capabilities to, in turn, contribute to local economic viability and the expansion of the nation's aerospace R&D base.

NASA's EPSCoR program includes three award components, each based on a performance period of up to three years: EPSCoR Research Infrastructure Development (RID) awards of up to $125,000 per year to help jurisdictions build relationships with NASA researchers; EPSCoR Research awards of up to $750,000 for topic-specific projects on high-priority NASA research and technology development needs; and EPSCoR International Space Station (ISS) Flight Opportunity awards of up to $100,000.

U.S. Department of Agriculture

At USDA, the National Institute of Food and Agriculture's (NIFA's) Agriculture and Food Research Initiative (AFRI) operates the Food and Agricultural Science Enhancement (FASE) grants program. Broadly, FASE grants are designed to help institutions develop competitive projects and attract new scientists and educators to work in national high-priority research areas. The FASE program includes three types of grants: New Investigator Grants, Pre- and Post-doctoral Fellowships, and Strengthening Grants. The Strengthening Grants component includes USDA's EPSCoR-like program funding.

Strengthening Grants are awarded with the goal of enhancing institutional capacity, leading to future funding within the supported project area, and strengthening the competitiveness of research, education, and/or extension activities. This subset of the FASE grants can be awarded to

(1) small and mid-sized or minority-serving degree-granting institutions that previously had limited institutional success for receiving Federal funds, or (2) State Agricultural Experiment Stations or degree-granting institutions eligible for USDA Experimental Program to Stimulate Competitive Research (EPSCoR) funding .

While Strengthening Grants comprise 11.25% of AFRI FASE funding overall, there is no specific set-aside amount for grants to EPSCoR-eligible states. Furthermore, all AFRI grant applicants are first considered together regardless of EPSCoR status; those applicants that do not receive funding due to budget constraints but were ranked highly and identified as EPSCoR-eligible may then be considered for a Strengthening Grant award. Awards are provided directly to individual investigators. Eligibility for Strengthening Grants is determined by NIFA each year and includes states that have a "funding level no higher than the 38 th percentile of all states based on a three-year rolling average of AFRI funding levels, excluding FASE Strengthening Grant funds granted to EPSCoR states and small, mid-sized, and minority-serving degree-granting institutions."

National Institutes of Health

The NIH Institutional Development Award (IDeA) program is also an EPSCoR-like program. IDeA is administered by the National Institute of General Medical Sciences (NIGMS). The National Institutes of Health Revitalization Act of 1993 directed NIH to establish a program to provide research institute funding in states with historically low success rates. The program has two main components: Centers of Biomedical Research Excellence (COBRE) and IDeA Networks of Biomedical Research Excellence (INBRE).

The COBRE program supports thematic, multidisciplinary centers with related projects run by junior faculty and overseen by senior mentors. COBRE focuses on developing both infrastructure and scientific investigator capacity by providing up to 15 years of funding in three five-year phases. Centers must apply for each five-year phase, with funding in the second and third phases contingent upon previous success. An additional COBRE activity is the IDeA Program Infrastructure for Clinical and Translational Research (IDeA-CTR) initiative. IDEA-CTR funding supports the development of regional infrastructure and capacity to address the "tremendous challenges and opportunities [that] remain for pursuing activities aligned with NIH's growing interest and focus on clinical and translational research."

The INBRE program supports the establishment of statewide systems of biomedical institutions and efforts to improve student access to biomedical training and resources. Similar to NSF's EPSCoR program, the IDeA program also has a co-funding component for meritorious awards from other NIH institutes and centers in IDeA-eligible states.

In NSF EPSCoR states, IDeA program staff work informally with the EPSCoR committees and have made efforts to ensure that the committees include members with knowledge of biomedical research and medicine. An eligible jurisdiction may only have one active INBRE grant at a time —similar to the NSF RII restriction—but may have up to three active COBRE grants per institution. Cost-sharing is not required, though applicants are encouraged to provide matching funds. Strong support from the institution and a commitment to building and sustaining the research program are important considerations in the proposal review process.

By statute, IDeA eligibility is to be based on historic proposal success rates of institutions in each state. The most recent NIH eligibility requirement was set at a proposal success rate of less than 20% or a three-year average of total NIH awards of under $120 million per year. In 2008, a reassessment of these criteria and a nationwide decline in NIH proposal success rates led NIH to freeze program eligibility; the list of eligible states has not changed since that time. If eligibility had not been frozen, 47 states and territories would currently be IDeA-eligible. In a report to the Senate and House Appropriations committees, IDeA administrators have proposed a revision in the program's statutory language that would change the basis for eligibility to "aggregate NIH funding received by entities in the state at or below the median of all states."

Funding

Total EPSCoR funding across agencies began to grow significantly starting around FY1999, largely due to increased funding for EPSCoR programs at NIH and NSF. In FY2015, total EPSCoR program funding across agencies was $500.8 million, with NIH and NSF programs together accounting for close to 88% of the funding: nearly 55% at NIH ($273.3 million) and 33% at NSF ($165.5 million). Table 2 shows the funding amounts for prior and current EPSCoR programs across agencies from FY1997 through the FY2017 budget requests. Figure 2 illustrates funding trends for EPSCoR programs by agency from FY1997 through the FY2017 budget requests.

In FY2015, NSF EPSCoR funding comprised about 2.3% of the agency's $7.3 billion overall budget. NSF's EPSCoR program is currently administered by the Research and Related Activities (RRA) account. The RII component has historically accounted for the largest proportion of the program funding. In FY2016, estimated funding for the RII component is $128 million, which is 80% of the total NSF EPSCoR funding; the co-funding component accounts for 19% ($30 million), and the outreach/workshop component accounts for about 1% ($2 million).

Interagency Coordination

In 1992, with direction from the Senate Committee on Appropriations, federal agencies with EPSCoR or EPSCoR-like programs formed the EPSCoR Interagency Coordinating Committee (EICC) "to integrate all EPSCoR programs into a single unified effort to maximize taxpayer investment." Member agencies initially worked together under the guidance of a Memorandum of Understanding (MOU). Statutory authority was subsequently provided to the EICC by the America COMPETES Reauthorization Act of 2010, which directed NSF to continue to carry out its EPSCoR program, to coordinate federal EPSCoR and EPSCoR-like programs through the EICC, and to report to Congress annually on activities, among other provisions. Current EICC membership includes NSF—serving as Chair and Executive Secretary—DOE, NASA, USDA, and NIH.

The EICC meets regularly to improve federal agency coordination and implementation of active EPSCoR and EPSCoR-like programs. The committee's goals include the following:

To coordinate federal EPSCoR and EPSCoR-like programs to maximize the impact of federal support while eliminating duplication in states receiving EPSCoR support from more than one agency; To coordinate agency objectives with state and institutional goals, to obtain continued nonfederal support of science and technology (S&T) research and training; To coordinate the development of metrics to assess gains in academic research quality and competitiveness and in S&T human resource development; and To exchange information on pending legislation, as appropriate, agency policies, and relevant programs related to S&T research and training, and to provide responses on issues of common concern.

As an example of its work, the EICC's FY2015 report to Congress highlighted discussions from its two meetings. Topics included EICC and NSF responses to the National Academy of Sciences EPSCoR assessment report (discussed in the " Program Assessments " section); NSF eligibility criteria and the "graduation" of Iowa, Tennessee, and Utah from the NSF, NASA, and DOE EPSCoR programs; and the role of, and need for, jurisdictional EPSCoR steering committees.

Program Assessments

As the EPSCoR program has grown and developed, Congress and others have expressed interest in determining how successfully it has achieved its mission of helping states with less developed R&D capacity to improve their ability to compete for federal R&D funding. To this end, a number of institutions have conducted assessments of EPSCOR. Many assessments have focused largely or solely on the NSF and NIH programs, while some recent assessments have examined the programs across agencies.

In 1999, NSF released the results of an external evaluation of the EPSCoR program's initial years, from 1980 to 1994. The evaluation was intended to assess the program's influence on the geographical concentration of federal R&D funds and to identify successful strategies. According to the report, the EPSCoR states' share of federal R&D increased modestly from 0.25% to 0.4% ($10.1 million to $50.5 million) per state on average during that period, though the EPSCoR program's impact on this change was not able to be quantified. Evaluators stated that a "plausible argument" could be made that EPSCoR contributed to increased R&D competitiveness—citing program features that reportedly helped build research infrastructure, investigator capabilities, and intra- and inter-university collaborations—though the program's impacts on university-wide actions and policies were limited. The report recommended periodic reassessments of eligibility criteria and continued support of research clusters to promote long-term, sustainable research efforts.

In 2008, NIH sponsored an external process evaluation of its COBRE program, which was initiated in FY2000. The study included 19 centers and 107 junior investigators. While there was considerable variation in baseline characteristics and program activity emphasis among centers, the report concluded that the program was effective in strengthening institutional research infrastructure and training investigators. The report further stated that, as a group, the centers were successful in achieving program goals. Recommendations from evaluators included conducting rigorous assessments of research progress, placing a strong emphasis on mentorship of junior investigators, and increasing involvement from external assessment committees.

Through the America COMPETES Reauthorization Act of 2010, Congress directed the NSF to contract with the National Academies of Sciences (NAS) to conduct a study of "all Federal agencies that administer an [EPSCoR program] or a program similar to [EPSCOR]." The resulting study was released in 2013. Therein, the NAS committee reported that it could not adequately assess EPSCoR effectiveness as charged, citing program goals that have broadened over time and vary by agency, and a "scarcity of rigorous data and scholarly assessment literature." Focusing rather on understanding overall program structure and policies and evaluating core concerns, the committee findings and recommendations included the following:

Challenges . Per the report, EPSCoR programs across agencies have competing objectives and policy directives, as well as inconsistent and incomplete evaluation metrics, making assessment difficult. Participating states reportedly have uneven commitments to improving research capacity, and the aggregate share of federal R&D funding to eligible states has not changed significantly over the course of the EPSCoR program. Successes . Evaluators found that EPSCoR programs have enhanced infrastructure, training, and human capital in participating states. The study notes that nearly all states reported positive changes in cultural attitudes, programs, and policies regarding science, engineering, and research. Recommendations . Broadly, the committee recommended that the federal government continue promoting the development of research capacity in all states, asserting that "students in all parts of the country must have the chance to participate in high-quality research, and it is in the national interest that federal funding be provided to universities in every state to ensure that these research opportunities are available." To improve the focus and impact of the EPSCoR program, recommendations included an enhancement of research and STEM training capacity; more rigorous proposal and project evaluation processes, including effective third-party evaluation; a matching funds requirement for all research awards; and, working through the EICC, development of a new framework for eligibility and graduation.

In 2014, the Institute for Defense Analyses (IDA) Science and Technology Policy Institute (STPI) released a report commissioned by NSF in 2011 to conduct a life-of-program assessment of NSF-specific EPSCoR activities, outputs, and outcomes. The analyses focused on program goals and funding levels, competitiveness for research funding, S&T research base enhancements, eligibility indicators, and the concentration of NSF research funding.

The IDA/STPI report found that NSF's EPSCoR program has meaningfully contributed to increased competitiveness for NSF funding overall; however, competitiveness increased for state cohorts entering the program prior to 2000, but not for those entering after. Furthermore, evaluators stated that the geographic concentration of NSF R&D funding was shown to have decreased slightly since 1980, though the decrease could not be attributed to EPSCoR. Evaluators also noted that jurisdictions have developed their research bases, improved university policies promoting research, and sustained their EPSCoR activities over the long term. While the EPSCoR statute states that the purpose of the program is to assist states that "historically have received relatively little Federal research and development funding," it does not define "relatively little" with respect to determining program eligibility nor does it define the units to be considered (e.g., dollars, share of total federal research funding). The report notes that the choice of such criteria can have important implications for determining eligibility.

To help ensure that EPSCoR funding has the greatest impact, the IDA/STPI report concluded that NSF should define "undue concentration"; encourage jurisdictions to use experimental strategies to enhance their research capacity; and improve eligibility calculations, program-level evaluations, and research competitiveness data analysis.

In 2015, NSF provided an official response to the NAS and STPI reports. In the two-page response, the agency proposed strengthening program-level evaluations; encouraging jurisdictions to experiment with flexible and competitive mechanisms to support research, faculty, and STEM education; and working with other agencies in the EICC to explore the use of specific indicators of "undue concentration" and to reexamine eligibility criteria.

In addition to the assessments, NSF has previously held two workshops with a variety of stakeholders from state and federal government, business, and EPSCoR administrators and faculty to recommend long-term program goals and strategies—the EPSCoR 2020 workshop in 2006 and the EPSCoR 2030 workshop in 2012. Broadly, the EPSCoR 2030 workshop recommended effectively coordinating across federal agencies, maintaining a focus on increasing research capacity, using EPSCoR institutions as test beds for new agency initiatives, and developing robust cyberinfrastructure at institutions to help them stay competitive.

Further, as part of NSF's ongoing internal review processes, the NSF Committee of Visitors (COV) completed a detailed EPSCoR program review in FY2015. The review included program processes and a sample of 74 proposals spanning the three EPSCoR award components. Reviewers provided recommendations for improving proposal review and post-award data collection, expanding certain award components, and establishing a standing advisory committee for the program. NSF's response document included plans to convene an advisory panel in FY2017, building on the EPSCoR 2020 and EPSCoR 2030 workshops.

In summary, assessment activities over the years have included repeated recommendations in certain areas: reevaluating eligibility and graduation criteria, improving data collection and program evaluation processes, and focusing on flexible and sustainable program strategies.

Selected Issues

An overarching challenge for the EPSCoR program since its inception has been crafting a balance between supporting sustainable research capacity development equitably across states while also supporting high-quality science through the merit review process. As EPSCoR has grown among agencies, with variations in policy and program structures, a number of issues have emerged.

Expanding Duration and Focus

EPSCoR's lengthening duration and evolving program focus have been considered by program supporters and critics, and addressed in program reviews and academic analyses. As the program's time frame has continued beyond the initial limit of five years, the goals and objectives have both evolved and expanded.

In planning the creation of EPSCoR, then-Director of NSF Richard Atkinson stated that:

It would be clearly understood from the beginning that no support would be provided beyond five years through [EPSCoR], as scientists in the funded states should then be able to compete more successfully for support from NSF and other agencies.

Some critics contend that the existence of an experimental program 30 years after its establishment demonstrates that the program is ineffective and instead is a "redistribution of wealth." One academic analyst who conducts research on federal decisionmaking regarding integration of science with policy and technology transfer asserted that EPSCoR "was not intended as an entitlement, but rather as a catalyst."

On the other hand, EPSCoR supporters have argued for the necessity of the program's long tenure. One NSF staff member during EPSCoR's initial years has asserted that he and others realized that it was not feasible to achieve the program's objectives in such a short time. A recent study asserted that building research capacity and competitiveness through investments in physical infrastructure and human capital "is not an easily achievable target because capacity building generally takes a long time." Just how long remains uncertain for EPSCoR programs and states and is a topic of debate among stakeholders.

As EPSCoR's duration has increased, program focus has evolved and expanded. Initially, EPSCoR was termed "experimental" in its approach to building research capacity in certain states. More recently, the program has been referred to as "established," with "experimental" referring to support for innovative methods to achieve program goals. In 2009, Arden Bement, then Director of NSF, called it a "program of experimentation" as opposed to an experimental program. Some have compared EPSCoR state projects to the work of small businesses—in regards to purportedly being more nimble than large entities—and have asserted that EPSCoR states are able to develop "unique S&T related abilities and expertise to contribute to the national agenda."

Furthermore, some researchers have noted that EPSCoR has become an important component in developing states' economies. One analyst contends that EPSCoR somewhat inadvertently "evolved into a program that fosters science-based economic development, an extension of the best science paradigm on which NSF and EPSCoR were founded," during which time there was also "a general national trend toward partnering academic research with economic development." A former state EPSCoR administrator asserted that EPSCoR has "put [S&T] on the forefront in discussions of the role and importance of universities in states. Not only from the perspective of education and research, but particularly from the perspective of technology transfer and economic development."

In its RII Track-1 program solicitation, NSF has emphasized the importance of linking an EPSCoR jurisdiction's S&T plan with its economic development plan:

[A jurisdiction's] S&T plan should be informed by the jurisdiction's Economic Development Plan (if applicable) and should describe innovation pathways for bringing outputs and outcomes of the proposed RII Track-1 research to the marketplace.

In addition, the 2015-2016 NSF EPSCoR Workshop Series on Innovation, Entrepreneurship, and Translational Research led to a guide to help EPSCoR jurisdictions build such connections.

NIH's program has a focus on innovation and economic development, as well. In its FY2017 budget request, NIH stated its intention to initiate small business and technology transfer activities in IDeA states. NIH also expressed its intention to use funds from the Small Business Innovation Research (SBIR) and Small Business Technology Transfer (STTR) programs to set up biotechnology incubators in each of the four IDeA regions. The Senate Committee on Appropriations has expressed support for such plans: "The Committee supports the initiative to direct small business research funding to IDeA States to foster the development of products to advance public health."

This focus on innovation is part of a growth of the EPSCoR objectives, which include supporting research projects, developing statewide and regional networks, building infrastructure and human capacity, broadening diversity in STEM, and connecting research to technology transfer, state economic development, and national priorities.

Reevaluating Eligibility Criteria

The threshold for NSF EPSCoR eligibility has changed multiple times over the program's duration. Recently, the 2014 IDA/STPI assessment of NSF's EPSCoR program noted—based on a literature review—that "a range of indicators ... could be used to identify jurisdictions with 'relatively little' Federal funding," but found "no consensus that one or more of those indicators is the preferred way to select EPSCoR-eligible jurisdictions." In addition to the IDA/STPI report, the 2013 NAS assessment also suggested a number of EPSCoR eligibility factors for consideration, including per capita federal research funding; state commitment; proposal success rates of research university faculty; total research funding; progress to date; future opportunities for progress; and financial need.

Responding to the reports, NSF noted that it would "consider models for eligibility and graduation, including the option of using median funding level" and "explore the use of specific indicators of 'undue concentration' in collaboration with other federal agencies ... and ensure that EPSCoR's program design, funding levels, and eligibility criteria reflect the indicators." The NSF's FY2017 budget request stated that this issue was being explored by the EICC: "In FY2016, the [EICC] was tasked with examining the eligibility criteria across the five agencies with active programs ... and determining if there should be a common federal EPSCoR eligibility criterion." The EICC subsequently provided recommendations to NSF's Office of Integrative Activities (OIA); as of October 2016, the recommendations remained under review by NSF leadership. Developing a common federal eligibility criterion (or criteria) might help address concerns voiced by some Members of Congress that eligibility for EPSCoR funding at DOE is based on NSF research award expenditures rather than on DOE Office of Science award expenditures (as discussed in the " Recent Congressional Activity " section).

NIH also reports that, since freezing eligibility in 2008, the agency has developed a new proposed criterion under which eligible states would include those for which the aggregate NIH funding received by entities in the state is at or below the median of all states. Such a criterion raises the question of whether or not state "graduation" from EPSCoR remains a goal. It is uncertain yet if EPSCoR agencies will develop common eligibility and graduation criteria.

Determining Success

Program assessments, peer-reviewed literature, and Members of Congress have all raised broad questions about how to determine EPSCoR's success. Among the types of questions raised: What metrics should be used to determine success (e.g., increases in agency-specific R&D funding, all federal R&D funding, grant proposal success rates, research collaborations)? At which level should these metrics be evaluated—institution, state, region? How can federal agencies with active EPSCoR programs best coordinate data collection and evaluation of comprehensive program success in light of the differences in agency-specific missions?

Several academic studies have been conducted to assess EPSCoR's impact on university research, competitiveness in R&D awards funding, and economic development. A primary measure of program effectiveness has been increased shares of federal funding in EPSCoR states. Some research has shown that states become more competitive for federal science and engineering (S&E) funding the longer they stay in the EPSCoR program. This finding was based on faster growth of federal S&E obligations in EPSCoR states compared to non-EPSCoR states; however, the effect was small, suggesting a long and slow process of addressing changes in concentrations of research funding. A 2009 study concluded that, for some states, greater success in securing federal R&D funding may have led to reduced state contributions to academic institutions. Another evaluation found that the percentage of university-performed R&D funded by industry—used in the study as a rough indicator of the intensity of university-industry research partnerships—was comparable in EPSCoR and non-EPSCoR states between 1985 and 2000.

In addition to funding, researchers have suggested that including a more nuanced evaluation of capacity building—looking at both the "hard infrastructure and the human-based infrastructure and resources"—is needed to fully understand capacity and competitiveness improvements in EPSCoR states. For example, one study showed that EPSCoR had a significant positive impact on publication quality at universities in participating states. A 2009 study noted that "a social capital approach to the assessment of EPSCoR would address capacity building at the micro level, and be targeted to state characteristics"; the study found that—within EPSCoR states—scientists who participated in program projects performed better in capacity-related characteristics (e.g., larger collaborative networks) than their colleagues who did not. Notably, this and other studies highlight a common challenge—demonstrating that EPSCoR is not only associated with measures of success, but directly contributes to them. Nevertheless, some may caution against focusing too heavily on directly quantifiable outcomes, which might not adequately assess impacts from EPSCoR support. For example, research investments, particularly investments in basic research, may have impacts on future competitiveness that are difficult to measure or anticipate, such as advancing fundamental knowledge and spurring new and innovative research directions.

Broadly, researchers and program assessors have recommended more thorough data collection procedures with standardization across programs to allow for improved comparisons in future EPSCoR evaluations. NSF EPSCoR has stated that it "strongly agrees that detailed data, captured in a uniform fashion over time, is essential to assessing the outputs and outcomes of EPSCoR investments, and effective program management overall," highlighting recent efforts to standardize data reporting for RII awards.

Recent Congressional Activity

On several occasions, the 114 th Congress addressed aspects of the EPSCoR program.

The American Innovation and Competitiveness Act (AICA, P.L. 114-329 ), enacted on January 6, 2017, revises program requirements and renames EPSCoR as the Established Program to Stimulate Competitive Research, among other provisions. The act directs agencies with EPSCoR programs to consider modifications to award structures and evaluations, including an emphasis on harmonization of EPSCoR metrics across agencies, long-term investments, and support for innovative and experimental research and funding models. Further, the AICA requires the EICC to brief the appropriate committees of Congress on any such modifications one year after enactment.

In 2015, H.Amdt. 317 would have amended H.R. 2578 , the Commerce, Justice, Science, and Related Agencies Appropriations Act, 2016, to prohibit the use of funds for any EPSCoR program funded by the bill; this would have included EPSCoR programs at NSF and NASA. The amendment failed on a floor vote of 195-232. During floor debate, some Members expressed concerns with the program, asserting that many EPSCoR states have small populations and receive more in federal spending than they pay in taxes; program monies are determined on a per state basis, rather than a per capita basis; and what was originally intended as temporary assistance funding to develop research infrastructure has grown into a permanent program. Opposition to the amendment asserted that EPSCoR has supported areas of strategic importance and helped to stabilize an imbalance in funding—cited as 28 EPSCoR jurisdictions accounting for about 10% of NSF funding but 20% of the U.S. population—and should continue to do so.

In 2016, H.Amdt. 1122 , a similarly focused amendment to H.Amdt. 317 , would have amended H.R. 5055 , the Energy and Water Development and Related Agencies Appropriations Act, 2017, to prohibit the use of funds for DOE's EPSCoR program. The amendment failed on a floor vote of 206-213. During House floor debate, many of the same objections to the program were raised as for H.Amdt. 317 with additional concerns that the program at DOE is based on NSF's EPSCoR program eligibility and research grant expenditures, rather than DOE research expenditures. Supporters of the amendment asserted that "a rational program would ... collect all research funding in all areas and base the set-asides on that [and] would do it on a per capita basis." Opponents of the amendment stated a willingness to debate program details—such as potential modifications to program formulas—and emphasized that eliminating the program would be a mistake.

The introduced version of the 2015 Senate Labor, Health and Human Services, and Education, and Related Agencies Appropriations bill ( S. 1695 ) would have revised eligibility criteria for the IDeA program, including allowing NSF EPSCoR-eligible entities to apply for inclusion in NIH INBRE awards. That program revision was not included in the FY2016 omnibus, but the accompanying explanatory statement included recognition of the IDeA program's success and requests to update and incorporate EPSCoR eligible states into IDeA program eligibility criteria.

Concluding Observations

The EPSCoR program funding and objectives have expanded and evolved since its establishment more than three decades ago. Assessments and workshops involving a wide range of stakeholders have helped to inform these changes, as has some academic research. As the program has grown, some stakeholders have also raised program-specific concerns due to EPSCoR's geographically targeted structure and variations among agency approaches. Further, some of the questions raised about the program are similar to questions raised about the broader role of the NSF, such as: What is the federal government's role in scientific research? How do Congress and agencies balance supporting innovative, independent scientific research while remaining accountable for the use of public funds?

Going forward, Congress may consider whether or not to provide additional direction and oversight regarding EPSCoR's longevity and focus, and the development of new eligibility and graduation criteria. Should graduation remain a desired goal for EPSCoR programs, Congress may further explore how new eligibility criteria—such as setting eligibility at a median funding level—would affect the potential for jurisdictions to "graduate" from a program. Congress may also consider whether or not to require additional data collection and comparative program evaluations across agencies to inform future program direction and congressional action.

Appendix. NSF EPSCoR Eligibility Table