Department of Energy: Advanced Research Projects Agency-Energy Could Benefit from Information on Applicants' Prior Funding
Released: 2012-01-24T00:00:00


Highlights: section_title: Why GAO Did This Study
paragraphs: The Department of Energy’s (DOE) Advanced Research Projects Agency-Energy's (ARPA-E) purpose is to overcome long-term and high-risk technological barriers in the development of energy technologies. Since 2009, ARPA-E has awarded $521.7 million to universities, public and private companies, and national laboratories to fund energy research projects.

GAO was asked to examine (1) ARPA-E’s use of criteria and other considerations for making awards and the extent to which applicants identify and explain other private funding information, (2) the extent to which ARPA-E-type projects could have been funded through the private sector, and (3) the extent to which ARPA-E coordinates with other DOE program offices to avoid duplicating efforts. GAO interviewed ARPA-E program directors, award winners, and nonwinners with characteristics similar to those of award winners. GAO also analyzed private venture capital funding data and spoke with venture capital firms.

section_title: What GAO Found
paragraphs: ARPA-E uses four selection criteria, such as the potential impact of the proposed technology relative to the state of the art, and other considerations in awarding funds. Other considerations include balancing a variety of technology approaches and the likelihood the technology would be brought to market. GAO identified 18 out of 121 award winners through ARPA-E’s first three funding rounds that had received some prior private sector investment, and ARPA-E took steps to identify and understand how this funding was related to proposed projects. Beginning with the third funding round, ARPA-E began requiring that applicants explain why private investors were not willing to fund proposed projects. However, ARPA-E did not provide applicants with guidance, such as a sample response, to assist them in completing this requirement, and responses were generally limited. Some applicants provided general information about prior research but did not specifically explain why private investors would not support their projects. When applicants provided little prior funding information, ARPA-E’s program directors spent time and resources to determine the extent of such funding for proposed ARPA-E projects. One applicant included a letter from its venture capital investor to explain why the investor was not willing to fund the work proposed to ARPA-E, an approach the National Institute of Standards and Technology uses as a check in its funding applications for advanced research but that ARPA-E currently does not use. Also, ARPA-E officials said that they have considered but have not used venture capital data to identify applicants with prior private investors. Examining such data allowed GAO to quickly cross-check applicants’ prior private funding.

GAO’s review suggests that most ARPA-E projects could not have been funded solely by private investors. Private venture capital firms told GAO that, among other considerations, they generally do not fund projects that rely on unproven technologies and tend to invest in projects that can be commercialized in less than 3 years. Data from ARPA-E on award winners show that 91 out of 121 ARPA-E projects from the first three funding rounds had technological concepts that had not yet been proven in a laboratory setting. Also, nearly all of the ARPA-E award winners and applicants GAO spoke with estimated that their projects were at least 3 years away from potential commercialization. In addition, GAO found that few eligible applicants that were not selected for an award later secured private funding.

ARPA-E officials have taken steps to coordinate with other DOE offices to avoid duplication. For example, ARPA-E program directors told GAO they engage in outreach with officials from related DOE offices in advance of funding announcements to identify funding gaps in research. In addition, program directors have recruited officials from other DOE offices and the Department of Defense (DOD) to review ARPA-E applications. This cross-agency interaction may also reduce the potential for overlap in funding.

section_title: What GAO Recommends
paragraphs: GAO recommends that ARPA-E consider providing applicants guidance with a sample response explaining prior sources of funding, requiring applicants to provide letters from investors explaining why they are not willing to fund proposed projects, and using third-party venture capital data to identify applicants' prior funding. ARPA-E commented on a draft of this report and concurred with key findings and recommendations.

section_title: Letter
paragraphs: U.S. energy consumption has increased by 20 percent over the past 20  years and is projected to continue to grow. Volatile prices, global supply  disruptions, and the impacts of energy use on climate and the  environment have driven interest in reducing energy demand, improving  energy efficiency, and expanding supplies with both renewable and  traditional energy sources. In 2005, members of Congress asked the  National Academies what actions federal policymakers could take to  enhance the nation’s science and technology enterprise so that the  United States could successfully compete, prosper, and be secure in the  global community of the 21st century. The National Academies compiled  their findings and recommendations in a report that identified two key  challenges: (1) creating high-quality jobs for Americans, and   (2) responding to the nation’s need for clean, affordable, and reliable  energy. The report also highlighted the idea that scientific and technical  innovations are key drivers of economic growth in the United States.  Among the report’s recommendations was the creation of an organization  within DOE to sponsor energy research that industry by itself cannot or  will not support and in which risk may be high but success would provide  dramatic benefits for the nation in meeting long-term energy challenges.

In 2007, the America Creating Opportunities to Meaningfully Promote  Excellence in Technology, Education, and Science Act (America  COMPETES) established the Advanced Research Projects Agency- Energy (ARPA-E) within the Department of Energy (DOE) to overcome  the long-term and high-risk technological barriers in the development of  energy technologies. ARPA-E borrows from the Defense Advanced  Research Projects Agency (DARPA) model, an agency created within the  Department of Defense (DOD) in 1958 to direct and perform advanced  research and development projects. As specified in statute, ARPA-E’s  program goals are to enhance U.S. economic and energy security  through the development of certain energy technologies and to ensure  that the United States maintains a technological lead in developing and  deploying advanced energy technologies.

Since first receiving an appropriation in 2009 in the American Recovery  and Reinvestment Act of 2009, ARPA-E has awarded $521.7 million to  universities, public and private companies, and national laboratories to  fund 181 projects that attempt to make transformational—rather than  incremental––advances to a variety of energy technologies, including   Award winners must meet  high-energy batteries and renewable fuels. cost share requirements, through either in-kind contributions or outside  funding sources.

ARPA-E is required by statute to achieve its goals through energy  technology projects that, among other things, accelerate transformational  technological advances in areas that industry by itself is not likely to  undertake because of technical and financial uncertainty. At the same  time, the Director of ARPA-E is required to ensure, to the maximum  extent practicable, that ARPA-E’s activities are coordinated with, and do  not duplicate the efforts of, programs and laboratories within DOE and  other relevant research agencies. This report responds to your request  that we examine ARPA-E awards. Our objectives were to examine (1)  ARPA-E’s use of criteria and other considerations for making awards, and  the extent to which applicants identify and explain other private funding  information; (2) the extent to which ARPA-E-type projects could have  been funded through the private sector; and (3) the extent to which  ARPA-E coordinates with other DOE program offices to avoid duplicating  efforts.

To address these three objectives, we reviewed ARPA-E applications and  conducted interviews with applicants, award winners, representatives of  venture capital firms and other experts, and DOE and other federal  agency officials. ARPA-E has released a total of four funding  announcements—meaning the agency was accepting project proposals  for a set period of time—in April 2009, December 2009, March 2010, and  April 2011. Our review focused on ARPA-E’s first three funding rounds,  which had closed prior to the start of our review. The fourth funding round  did not close until September 2011. ARPA-E awarded funds for 121  projects out of 4,788 applicants across the three funding rounds we  examined. To address these three objectives, we reviewed ARPA-E  applications and conducted interviews with applicants, award winners,  DOE officials, and representatives of venture capital firms, among other  activities. Specifically,

To examine ARPA-E’s use of criteria and other considerations for  making awards, we selected a nonprobability sample of 20  applications from the 4,788 applications in the first three funding  rounds and reviewed these 20 applications. To examine the extent to  which applicants identify and explain other private funding information,  we searched for evidence of prior private funding for all 121 award  winners in VentureDeal, a venture capital database. As a result of  our search, we identified 18 award winners that had some prior  private venture capital funding from the 121 award winners. We then  reviewed the applications of these 18 award winners and interviewed  their representatives.

To analyze the extent to which ARPA-E projects could have been  funded through the private sector, we analyzed data on the state of  technology and potential time to commercialization for the 121 award  winners from ARPA-E’s first three funding rounds. In addition to  analyzing data for the 121 award winners, we conducted structured  interviews with 22 of 33 “contingently selected” applicants chosen by  ARPA-E during its second and third funding rounds. Contingently  selected applicants are those applicants that met ARPA-E’s selection  criteria but were ultimately not awarded funds. We also conducted  structured interviews with a nonprobability sample of 13 award  winners selected from ARPA-E’s first three funding rounds and we  spoke with the 18 ARPA-E award winners mentioned above that we  identified through the VentureDeal database to discuss key  differences between their prior research and their ARPA-E-funded  projects. We also conducted interviews with a variety of companies  and individuals knowledgeable about research associated with ARPA- E-type projects, including six venture capital firms.

To examine the extent to which ARPA-E coordinates with other DOE  programs to avoid duplicating efforts, we spoke with the ARPA-E  program directors as well as officials from other DOE program offices,  DARPA, and the DOE Office of Inspector General. We also asked  award winners and contingently selected applicants to discuss their  understanding of other potential sources of DOE funding for their  projects.

We provide a more in-depth discussion of our methods in appendix I.

We conducted this performance audit from November 2010 to December  2011 in accordance with generally accepted government auditing  standards. Those standards require that we plan and perform the audit to  obtain sufficient, appropriate evidence to provide a reasonable basis for  our findings and conclusions based on our audit objectives. We believe  that the evidence obtained provides a reasonable basis for our findings  and conclusions based on our audit objectives.
subsections: section_title: Background
paragraphs: In 2005, the National Academies recommended to Congress the creation  of an organization within DOE like DARPA. In 2007, the America  COMPETES Act created a new agency within DOE called ARPA-E. In  line with the National Academies’ recommendation, the America  COMPETES Act as amended directs ARPA-E to achieve its goals by  identifying and promoting revolutionary advances in fundamental and  applied sciences, translating scientific discoveries and cutting-edge  inventions into technological innovations, and accelerating  transformational technological advances in areas that industry by itself is  not likely to undertake because of technical and financial uncertainty. As  such, ARPA-E officials told us that ARPA-E was designed to sponsor  research beyond basic science, yet riskier than what the private sector   (See fig. 1.) The National  alone or DOE’s applied offices would support.Academies recommended that ARPA-E should not perform research and  development itself, but should fund it to be conducted by universities and  others in the private sector. In 2009, the American Recovery and  Reinvestment Act of 2009 appropriated $400 million for ARPA-E.

ARPA-E is an agency with fewer than 30 federal employees, and its eight  program directors, who are generally scientists and engineers, create and  manage funding programs for the agency. ARPA-E’s program development  and award selection process takes 6 to 8 months from start to finish,  beginning when the agency hires a program director for a 3-year term and  tasks the program director with identifying a gap in energy technology  research and developing a program to fill that gap. For example, ARPA-E’s  batteries for transportation program, called the Batteries for Electrical Energy  Storage in Transportation (BEEST) program, was established to fill a gap in  existing federal research programs on batteries for electric vehicles.  Identifying these gaps and designing the program involves research;  consultation with scientific experts, including a workshop with outside  experts; and internal discussion at ARPA-E. From this process, program  directors develop funding announcements that describe the technical  requirements specific to each program’s technology area that applicants   have to meet and the four criteria that ARPA-E uses in its selection  process. The four criteria are the  Impact of the proposed technology relative to the state of the art. The  applicant must demonstrate the potential for a transformational—not  incremental––advancement over current technologies. (See fig. 2.)  More specifically, the applicant must demonstrate an awareness of  competing commercial and emerging technologies and identify how its  proposed concept/technology provides significant improvement over  these other solutions.

Overall scientific and technical merit. The applicant must demonstrate  that the work is unique and innovative. The applicant must also  demonstrate a sound technical approach to accomplish the proposed  research and development objectives. The outcome and deliverables  of the program, if successful, should be clearly defined. Specific  technical requirements that are unique to each individual ARPA-E  program funding announcement must also be addressed.

Qualifications, experience, and capabilities. The applicant must  demonstrate that it has the expertise and experience to accomplish the  proposed project. In addition, the applicant must have access to all  facilities required to accomplish the research and development effort.

Sound management plan. The applicant must have a workable plan to  manage people and resources. Major technical research and  development risks should be identified. The schedule and budget  should be reasonable.

ARPA-E employs the following three-stage application process:

Concept paper. Applicants initially submit a 5- to 7-page abstract of  their projects. Scientific experts from industry, government, and  academia serve as reviewers.

Full application. After reviews of the concept paper, ARPA-E encourages  some applicants to submit full applications using ARPA-E’s online  application system. ARPA-E’s current instructions request that applicants  provide, among other things, information about other prior, current, and  pending public and private sources of funding, as well as why other  funding sources are not willing to fund the projects. Full applications are  then reviewed by leading scientific experts in the field, who evaluate  them against the four criteria and assign numerical scores.

Reply to reviewer comments. After assessing the full applications,  reviewers provide comments and questions to the applicants, who  then have the opportunity to respond.

The applications with the reviewers’ comments are forwarded to a three- person panel beginning the next three phases of ARPA-E’s award funding  process, which are as follows:

Selection. The three-person panel, usually chaired by the relevant  program director, considers the reviewers’ comments and numerical  scores, and recommends applications to award. The final decisions  on which applicants to select are made by the selecting official, which  is usually the ARPA-E Director.

Award negotiations. Negotiations proceed for approximately 2  months. Program directors work closely with the award winners to set  up a project plan with technical milestones that are to be met during  the funding of the award, which are planned to last between 2 and 3  years. Funds are awarded following the negotiations.

Monitoring. ARPA-E monitors and supports the project through  quarterly reviews and site visits. After about 1 year, the agency  decides whether to continue or terminate the project if the agreed-to  milestones are not met.

In April 2009, ARPA-E started its funding award process by releasing a  funding announcement soliciting proposals for all energy ideas and  technologies. Following the review process, 36 projects were awarded  funds after being selected from 3,700 applications that spanned the  technology areas of 10 programs. ARPA-E released additional funding  announcements in December 2009, March 2010, and April 2011. (See  table 1.) Money appropriated by the American Recovery and  Reinvestment Act of 2009 funded ARPA-E’s first three funding rounds.  After receiving an appropriation in DOE’s fiscal year 2011 appropriations  act, ARPA-E announced a fourth round of funding in April 2011.

section_title: In Addition to Its Selection Criteria, ARPA-E Also Considers Applicants’ Prior Sources of Private Funding; However, Most Award Winners We Reviewed Did Not Explain This Information
paragraphs: In addition to applying its four criteria, ARPA-E gives program directors  discretion to use additional considerations to award funds to projects,  including whether ARPA-E applicants received private funding. Most  ARPA-E award winners did not receive prior private funding, but for those  that did, most award winners we reviewed did not explain these funds.
subsections: section_title: ARPA-E Program Directors Use the Agency’s Four Selection Criteria and Other Considerations to Select Projects
paragraphs: Of the 20 applications we reviewed for award selection criteria, all  contained supporting information addressing the agency’s four criteria. In  our analysis of the ARPA-E reviewers’ evaluations from these 20  applications, we noted regular assignment of numerical scores rating  applicants on the extent to which they met the criteria. All eight ARPA-E  program directors told us they considered or, if they were recently hired,  will consider all four criteria, but several focused more heavily on two  criteria—the impact of the proposed technology relative to the state of the  art and its overall scientific and technical merit.

In addition to basing the numerical scores applicants receive on the  extent to which they meet the four selection criteria, program directors  told us the agency gives them the ability to take other qualitative  considerations into account when awarding funds. One of those  considerations is to fund a broad range of potential technological  solutions with varying levels of risk in solving a given technical problem.  Two program directors selected projects to reflect a variety of  technologies, and they told us they believe that this approach increases  their programs’ overall chances of success. Specifically, one program  director told us he chose projects that employed a variety of new battery  technologies, a strategy that should increase the likelihood that at least  one of them will work. This program director also chose some battery  projects with much higher potential storage capacity but with a lower  probability of success in achieving project milestones and in ultimately  being brought to market. In those cases, ARPA-E provided smaller  awards to the projects with the lower probability of success.

Several program directors also told us that during the selection process,  they considered the applicants’ projects’ proposed project scope and  duration, requested funding levels, and technical milestones and  negotiated to revise these, if necessary, to better align applicants’  projects with ARPA-E’s program goals. According to our review of ARPA- E data from the first three rounds of funding, the agency reduced  requested award amounts by 5 percent or more on 31 out of 121 projects,  for a total of $59 million below total requested award amounts for these  rounds. When ARPA-E makes these kinds of adjustments, the agency  may also reduce the proposed project scope to fund only what the  program directors consider to be the transformational part of the project  and to avoid funding applied research or development work that would be  outside ARPA-E’s program goals. For example, the agency reduced the  award amount and proposed project scope for an energy storage  technology project designed to improve energy storage on the electrical  grid. The project proposal initially requested nearly $5 million to  demonstrate the technology at nearly full scale. During award  negotiations, ARPA-E reduced this amount to $750,000 to focus the  project only on smaller-scale development and testing of the technology.  ARPA-E officials told us the larger-scale demonstration could likely be  funded by the private sector.

section_title: ARPA-E Also Considers Applicants’ Prior Sources of Private Funding, but Most Award Winners We Reviewed Did Not Explain This Information
paragraphs: When making award decisions or adjusting the scope of proposed  projects, ARPA-E program directors may also consider the identification  in applications of sources of private funding and the extent to which that  funding might support the proposed projects. This information can help  provide program directors with assurance that ARPA-E funds do not  overlap with private investment. During the first two funding rounds,  ARPA-E required that applicants identify relevant private investors if the  applicant believed these funds were related to the proposed project. Of  the 18 applications we reviewed from award winners that we identified as  having received private venture capital, 14 applied during ARPA-E’s first  and second funding rounds. Most of these award winners did not explain  why investors were not willing to fund proposed work. ARPA-E program  directors and an ARPA-E official, speaking on behalf of the agency, told  us they took additional steps to clarify outstanding prior funding questions  when ARPA-E was aware that applicants had received private sector  funding. For example, one applicant we reviewed from the first funding  round had previously received substantial private funding for work that  appeared very similar to its proposed ARPA-E project. ARPA-E officials  told us they were initially unable to determine why the private investor  was not willing to also fund the proposed ARPA-E project and that the  company’s application did not include an explanation. ARPA-E officials  told us that getting this information required them to draft a series of direct  and detailed questions that elicited several pages of written responses  from the applicant. ARPA-E officials also told us they conducted multiple  rounds of written and oral follow-up with the applicant and the private  investor. Through these efforts, ARPA-E determined that the  technological risks of key parts of the project were too high for the private  investor and therefore decided to fund the research.

Because ARPA-E officials recognized the need for applicants to provide  better prior funding explanations, beginning with its third funding  announcement, the agency required applicants to explain why proposed  work was not sponsored internally if the applicant was a large company,  or why private investors were not willing to support the project if the  applicant was a small business or start-up company. ARPA-E did not  provide guidance on how applicants should respond to this additional  requirement by, for example, providing a sample response. Of the 18  ARPA-E award-winning companies, 4 applied during the third funding  round, and these companies provided a range of information in response  to this new requirement in their funding applications. Two explained how  ARPA-E funds would allow them to go beyond currently funded work but  did not provide reasons why investors were not willing to support the  proposed work. Another wrote only that the ARPA-E research was too  risky for the company’s private investors. One application contained an  explanation outlining the specific research its private investors were and  were not willing to fund. This applicant explained that private funds were  directed toward lower-risk and higher-cost technologies. This application  also included a letter from the company’s venture capital investors that  explained which parts of its research the investors were planning to  continue funding and which research was too risky for them, although not  requested by ARPA-E. This letter provided additional third-party support  for the funding information in the application. Officials from the National  Institute of Standards and Technology’s (NIST) Technology Innovation  Program told us they request that applicants provide letters from private  investors to document why applicants’ projects could not be privately  funded.

When we followed up with the 18 companies, they were generally able to  explain to us why their private investors were not willing to undertake the  additional risk and uncertainty associated with the proposed projects.  When we examined the data in the VentureDeal database for a number of  applicants, the data allowed us to quickly cross-check the names of prior  private investors that applicants reported to ARPA-E. ARPA-E officials  said that they have not used such data for these purposes but that they  have considered doing so. Without an examination of outside venture  capital data on its applicants, the agency may be missing a time-saving  opportunity to check information on private funding provided in  applications, especially in instances where applicants may not have been  thorough in their explanations. We found a number of readily available  subscription-based venture capital data services that provided company  names, transaction amounts, and funding purposes. We found that the  web-based VentureDeal database matched formats and data available  from other venture capital data services.

section_title: Most ARPA-E Projects Likely Could Not Have Been Funded Solely by Private Investors
paragraphs: Our review suggests that most ARPA-E-type projects could not be funded  solely by private investors. Private venture capital firms told us that,  among other considerations, they generally do not invest in projects that  cannot be commercialized in less than 3 years. Nearly all of the 13  ARPA-E award winners and most of the 22 of the contingently selected  applicants we spoke with estimated that their projects were 3 or more  years away from a potential market-ready product (i.e.,  commercialization). In addition, we found that only 2 of the 22  contingently selected applicants we spoke with that met ARPA-E’s  selection criteria but were not selected for an award subsequently  secured private funding.
subsections: section_title: Venture Capitalists Generally Do Not Fund Projects That ARPA-E Looks to Fund
paragraphs: The representatives we spoke with from six venture capital firms identified  three factors that limit the general availability of venture capital funding for  new energy technologies. These factors were consistent with data we  analyzed for the 121 award winners from ARPA-E’s first three funding  rounds, the sample of 13 award winners we interviewed from these  funding rounds, and the 22 contingently selected applicants we  interviewed.

First, venture capital firms generally do not fund projects that rely on  unproven technological concepts or lack working prototypes  demonstrating the technology. A number of venture capital firm  representatives told us that they are generally not willing to fund the  applied scientific research sometimes required by ARPA-E-type projects.  Projects they fund generally focus on developing technologies based on  known scientific principles. Data from ARPA-E on award winners show  that 91 out of 121 ARPA-E projects from the first three funding rounds  had technological concepts that had not yet been demonstrated in a   According to a recent report from the American  laboratory setting.Energy Innovation Council, private investors consider these projects too  high risk for investment, even for concepts with promising technological  potential. Most of the contingently selected applicants we spoke with–– 17 out of the 22––told us they were unlikely to receive funding from other  sources for their proposed projects because of high levels of scientific  uncertainty, an unavailable or undeveloped market, or a lack of a working  prototype. For example, one such applicant said that he only had a  computer model suggesting that his high-efficiency air conditioning device  would work, which was insufficient to convince potential private investors.  In addition, many of the ARPA-E award winners we surveyed also  recognized the inherent uncertainty in their research; 5 of the 13 told us  that their projects had a fairly low probability of success.

Second, venture capital firms seek more rapid returns on investment and  closely analyze a project’s potential return on investment over time, a  factor that influences their decisions to invest in projects that are in later  stages of development and closer to commercialization. Venture capital  firm officials told us that they focused closely on the timeliness of  investment returns, with one firm noting that the industry tended to invest  in technologies that could be commercialized in less than 3 years and that  would potentially exhibit exponential market growth in approximately 5 to  7 years. However, we found that nearly all of the ARPA-E award winners  and most contingently selected applicants we spoke with estimated that  their projects were 3 or more years away from potential  commercialization. estimated that it would take at least 3 years for their ARPA-E projects to  For example, 12 out of 13 ARPA-E award winners  reach the commercialization stage with ARPA-E funding. Had they not  received ARPA-E funding, most of these award winners––10 out of 13–– told us they either would not have pursued their ARPA-E project or that  they would not have been able to develop a commercial product in less  than 10 years. At the same time, 18 out of 22 ARPA-E contingently  selected applicants estimated it would take at least 3 years for their  projects to reach commercialization if they had been able to secure  funding for the proposal they submitted to ARPA-E.

Third, venture capital firms may not be comfortable investing in new  energy technologies, noting the historical lack of successful venture  capital investments in these types of projects. Venture representatives  said that venture firms were more comfortable investing in software  companies or other businesses with higher potential profit margins and  less costly product development than new energy technologies. One  venture representative noted that his firm looked to invest in products with  potential gross profit margins of 50 percent or more. In addition, these  representatives noted that it is difficult for new advanced energy  technologies to compete with well-established and low-margin traditional  sources of energy like natural gas. Venture representatives also noted  that venture firms had become more risk averse and reluctant to fund new  energy technologies after lackluster investment returns have made the  venture industry more aware of the challenges associated with investing  in unproven energy technologies.

While venture capital firms generally do not fund projects that ARPA-E  looks to fund, our work suggests that receiving ARPA-E project funding  may have a positive effect on some award winners’ ability to attract  follow-on funding from the private sector for their ARPA-E work. For  example, ARPA-E’s data indicate that 18 out of 121 ARPA-E award  winners from ARPA-E’s first three rounds of funding had received private  sector funding totaling $318 million after receiving ARPA-E funding. In  some cases, award winners received private follow-on funding  immediately after receiving ARPA-E funding. A number of the award  winners we spoke with stated that, given the highly competitive nature of  the program, receiving ARPA-E funding served as a “stamp of approval”  to venture capital or other private firms. These award winners told us  that an ARPA-E award served as a signal of scientific and financial  approval for potential investors. Economists call this rapid follow-on  private funding a certification effect, which may explain the experiences of  some of these award winners. This effect suggests that public awards  address information gaps that might have otherwise precluded private  investment. Some award winners and economists we spoke with told us  that the government was suited to identifying technical risks because of  its ability to draw on the expertise of many scientific reviewers, while  venture firms may not have the scientific expertise on hand to fully  understand potential investments. Furthermore, economic literature  suggests that the certification effect may be particularly relevant in the  high-technology industries, where the venture capital community plays an  important role and in which traditional financial measures of risk and  returns on investments may prove insufficient.information on the difference between research funded by selected award  winners’ prior investors and ARPA-E funded work.

section_title: Few Contingently Selected Applicants Found Funding from Private Investors or Public Sources
paragraphs: Eighteen of the 22 ARPA-E contingently selected applicants we  interviewed sought funding after being turned down for ARPA-E funds. Of  the 18 that sought funding elsewhere, 13 submitted project proposals to  government sources, such as other DOE offices, the National Science  Foundation, or nonprofit academic research institutes, and the remaining  5 submitted proposals to private investors such as venture capital firms.  As of September 2011, we found that 2 out of the 22 contingently  selected applicants secured funding from venture capital firms for work  that was very similar to their ARPA-E project proposals.that 4 contingently selected applicants secured funding from a   We also found  government or nonprofit source for their projects. In addition, we found  that most contingently selected applicants modified their ARPA-E project  proposals to attract subsequent funding for their projects by reducing the  scope of their proposals or by focusing on more basic science research.  For example, the 4 contingently selected applicants that secured funding  from a government or nonprofit source modified their ARPA-E proposals  to be more focused on basic science research, rather than on developing  a commercial technology. In addition, 1 of these applicants told us that  the funding will allow it to continue exploring fundamental materials  science rather than developing a product. Also, many contingently  selected applicants and award winners said that other government  sources were limited. Some noted that Small Business Innovation  Research (SBIR) grants would not allow them to make as much progress  as larger ARPA-E awards.military funding agencies were not as focused on developing low-cost  technologies with broader market appeal, because aerospace or military  applications do not need to achieve the same low costs and market  appeal as consumer or commercial applications.

section_title: ARPA-E Officials Have Taken Steps to Coordinate with Other Department of Energy Offices in Advance of Awarding Funds
paragraphs: According to ARPA-E officials and documents, agency officials have  taken steps to coordinate with other DOE offices in advance of awarding  ARPA-E funds to help avoid duplication of efforts. These coordination  efforts can be categorized into three areas:  Prefunding coordination. ARPA-E officials told us that program directors  engage with officials from related DOE offices in advance of announcing  the availability of ARPA-E funds. ARPA-E program directors told us that  early in the development of a funding announcement, they conduct  outreach with industry, academic, and government officials both inside  and outside of DOE in an attempt to identify funding gaps related to the  technology they wish to develop. For example, by doing such outreach,  one program director determined that there had been little funding at DOE  or elsewhere for lithium air or lithium sulfur batteries, which have the  potential to last significantly longer than existing lithium ion batteries.  Program directors also hold workshops and invite relevant participants,  including those from other DOE offices and from other federal agencies,  to identify technologies that have little to no existing research funding but  that have transformational potential. ARPA-E officials told us that  directors use the workshops and other meetings to identify research  areas that other DOE offices are not working on, and the other DOE  officials provide insights on funding areas where they are not active. For  example, one of these ARPA-E program directors told us that he met with  officials from DOE’s Office of Electricity Delivery and Energy Reliability  and the Solar Energy Technologies Program within the Office of Energy  Efficiency and Renewable Energy (EERE) before announcing available  funds for the electrical power electronics funding announcement.  According to this program director, this coordination helped him identify  that there had been little funding for the development of magnetic devices  for use in electrical power electronics. He ultimately designed the ARPA- E electrical power electronics funding announcement to focus, in part, on  the development of improved magnetic devices because of the lack of  funding elsewhere.

In addition to inviting officials from other DOE offices to ARPA-E  workshops, program directors told us they also engage with other DOE  officials in other ways, both formally and informally. The program director  responsible for ARPA-E’s work on advanced batteries said that he was a  member of DOE’s Energy Storage Technology Development Team and  regularly met with other officials who are engaged in applied battery  research. This director said that it had become clear that DOE’s Vehicle  Technologies Program will continue to focus on incremental  improvements to existing lithium ion battery technologies that are  currently on the market, while ARPA-E will fund newer, alternative battery  technologies. Other program directors told us that they have regular  discussions with counterparts within DOE to avoid duplicating efforts,  although through other means than a formal committee.

Coordination of application reviews. Some ARPA-E program directors told  us that they have recruited officials from other DOE offices to review  applications submitted to ARPA-E and that these officials made up as  many as one-third of the reviewers for one director. These application  reviewers rate and recommend proposals for potential ARPA-E funding.  ARPA-E program directors told us that these DOE reviewers help them  stay aware of the types of projects that other DOE offices are funding. For  example, according to one program director, DOE reviewers indicated on  a number of occasions that an ARPA-E advanced battery applicant would  be better suited for funding under DOE’s Vehicle Technologies Program  because it was for a more developed technology. ARPA-E has also used  application reviewers from other federal agencies, such as the  Department of Defense. One program director told us that these  reviewers have also helped avoid funding projects similar to those  potentially funded elsewhere.

Official DOE coordination groups. ARPA-E is also a participant in DOE’s  SunShot Initiative within the Solar Energy Technologies Program. The  SunShot Initiative is an effort to coordinate solar energy research across  DOE’s Office of Science, four national laboratories, the National Science  Foundation, and ARPA-E, with the goal of achieving costs of $1 per watt  for solar-generated electricity. One ARPA-E program director is a  member of the SunShot Initiative advisory board and therefore able to  coordinate ARPA-E solar-related activities with other SunShot Initiative  members. SunShot Initiative officials told us that DOE plans to make it a  model for DOE’s internal coordination efforts and that DOE hopes to  expand the approach to other research areas.

Additionally, the ARPA-E Director created the Panel of Senior Technical  Advisors (PASTA), which is a group of high-level DOE managers that  meet periodically to discuss current and future DOE research efforts.  ARPA-E officials told us that PASTA is an attempt to avoid duplicating  efforts within DOE. PASTA meeting attendees have included officials  from DOE’s applied and basic science offices.

We were not able to directly evaluate the effectiveness of ARPA-E’s  efforts to coordinate with other DOE offices. Nevertheless, we found that  on the basis of our interviews with ARPA-E award winners and  contingently selected applicants, four award winners and two contingently selected applicants had received prior funding from other DOE offices.According to these award winners and contingently selected applicants,  the prior funding was either for more proven technologies or was focused  on more basic or foundational research than was the ARPA-E funded  project.

section_title: Conclusions
paragraphs: ARPA-E recognizes the need to ensure that the agency is not funding  projects that would be otherwise funded by the private sector, and has  taken steps to get information from applicants on their other sources of  funding. The agency has also taken steps to coordinate with other DOE  offices in advance of awarding ARPA-E funds. However, for the  applications we reviewed, we found that ARPA-E’s current funding  announcements have generally yielded limited information from  applicants that had prior sources of private funding. Where applicants  provided little information, ARPA-E’s program directors spent time and  resources to determine the extent of such funding for projects related to  or similar to the applicants’ proposed ARPA-E projects. The agency’s  requirements for information on private sector funding could be improved.  For example, ARPA-E does not provide guidance to applicants, such as a  sample response, on how to meet its information requirement on prior  private funding. An approach used by another federal program that funds  advanced research is for applicants to provide letters from private  investors to document why their projects could not be privately funded.  This approach was used by one ARPA-E award winner, who included a  letter from the company’s venture capital investors to explain why the  investors were not willing to fund the project proposed to ARPA-E. Also,  ARPA-E officials said that they have not used venture capital data to  identify applicants with prior private investors and to check information  applicants provide to them, but that they have considered doing so.  Examining such data allowed us to quickly cross-check applicants’ self- reported prior private funding. Without additional tools to better  understand prior private funding, ARPA-E program directors will continue  to spend time and agency resources taking additional steps to clarify prior  private funding and may miss opportunities to avoid duplication with  private investors.

section_title: Recommendations for Executive Action
paragraphs: To ensure that ARPA-E uses a more complete range of methods to  ensure that limited federal funds are targeted appropriately, we  recommend that the Secretary of Energy consider taking the following  three actions: provide guidance with a sample response to assist applicants in  providing information on sources of private funding for proposed  ARPA-E projects,  require that applicants provide letters or other forms of documentation  from private investors that explain why investors are not willing to fund  the projects proposed to ARPA-E, and use venture capital funding databases to help identify applicants with  prior private investors and to help check information applicants  provide on their applications.

section_title: Agency Comments and Our Evaluation
paragraphs: We provided a copy of our draft report to ARPA-E for review and  comment. ARPA-E concurred with key findings and our recommendations  in its written comments, which are reproduced in appendix IV. In its  comments, ARPA-E outlined the steps that the agency plans to take to  address our recommendations. ARPA-E also provided additional  clarifying comments, which we incorporated.

As agreed with your offices, unless you publicly announce the contents of  this report earlier, we plan no further distribution until 30 days from the  report date. At that time, we will send copies of this report to the  appropriate congressional committees, the Secretary of Energy, and  other interested parties. In addition, the report will be available at no  charge on the GAO website at http://www.gao.gov.

If you or your staffs have any questions about this report, please contact  me at (202) 512-3841 or ruscof@gao.gov. Contact points for our Offices  of Congressional Relations and Public Affairs may be found on the last  page of this report. GAO staff who made major contributions to this report  are listed in appendix V.

section_title: Appendix I: Scope and Methodology
paragraphs: To examine the Advanced Research Projects Agency-Energy’s (ARPA-E)  use of criteria and other considerations for making awards and the extent  to which applicants identify and explain other private funding information,  we reviewed 20 applications drawn from a nonprobability sample of the  4,788 applicants ARPA-E received during its first three funding rounds.  We selected applications from a range of ARPA-E technology program  areas to which the applications were submitted and applicant institution  types (e.g., small company or university). Because we selected a  nonprobability sample of applications to review, information we collected  cannot be generalized to all applicants; however, it provided us with an  understanding of ARPA-E’s criteria and other considerations for making  an award. We also interviewed and reviewed the applications from our  sample of 18 award winners, which were private companies that we  identified as having received funding from private investors prior to  receiving an ARPA-E award. We identified these 18 companies by  searching for evidence of prior private funding for the 121 award winners  in the VentureDeal venture capital database. In our review of these  applications, we focused on the extent to which applicants disclosed prior  private funding. We also spoke with all eight ARPA-E program directors  to discuss ARPA-E’s process for making awards and managing projects  of award winners.

To analyze the extent to which ARPA-E projects could have been funded  through the private sector, we conducted three sets of interviews with  ARPA-E applicants and award winners. Specifically,

We conducted structured interviews with 22 of the 33 contingently  selected applicants that ARPA-E encouraged to submit full  applications during its second and third funding rounds.

Each of the  contingently selected applicants fulfilled ARPA-E’s selection criteria,  had the same characteristics as ARPA-E award winners, and,  according to ARPA-E officials with whom we spoke, would have been  selected for an award had additional funds been available. This  approach allowed us to consider the potential of ARPA-E-type  projects to receive private funding.

We conducted structured interviews with a nonprobability sample of  13 award winners selected from ARPA-E’s first three funding rounds.  We selected subjects for this sample across a range of ARPA-E  award winner characteristics, including the technology program area  for which an award winner received funding, the stage of development  of an award winner’s project, and an award winner’s type of institution  (e.g., small company or university). Because this was a nonprobability  sample, the information from these structured interviews cannot be  generalized to all award winners but can provide examples about  award winners’ experiences. We conducted content analyses of the  award winners’ and contingently selected applicants’ interview  responses to quantify issues such as the ability of each group to  secure private sector funding for ARPA-E-type projects.

Third, we spoke with the 18 ARPA-E award winners we identified  through the VentureDeal database to discuss key differences between  their prior research and their ARPA-E-funded projects.

We also conducted interviews with a variety of companies and individuals  knowledgeable about research associated with ARPA-E-type projects,  including six venture capital firms and the National Venture Capital  Association (NVCA), a trade association, to determine the availability of  private capital for ARPA-E-type projects and the criteria venture capital  firms apply in making their investment decisions; two additional public  companies that were awarded ARPA-E funding to discuss the ability of a  public company to internally fund research; and three economists to  discuss the role and effectiveness of government-funded research and  development of technology.

To examine the extent to which ARPA-E coordinates with other DOE  programs to avoid duplicating efforts, we spoke with the ARPA-E program  directors as well as officials from other DOE program offices including the  Office for Energy Efficiency and Renewable Energy (EERE) and the  Office of Science. In addition, we met with officials from the SunShot  Initiative, which is a collaboration among EERE, the Office of Science,  and ARPA-E to make solar energy technologies cost-competitive with  other forms of energy. We also spoke with officials from the Defense  Advanced Research Projects Agency (DARPA) and the Department of  Energy (DOE) Office of Inspector General. During our interviews with the  award winners and contingently selected applicants previously  mentioned, we asked them to discuss their understanding of other  potential sources of DOE funding for their projects.

To assess the reliability of data from ARPA-E and VentureDeal, we  reviewed relevant documentation and interviewed key data system  officials at ARPA-E and VentureDeal and determined that the data were  sufficiently reliable for the purposes of this report.

We conducted this performance audit from November 2010 to December  2011 in accordance with generally accepted government auditing  standards. Those standards require that we plan and perform the audit to  obtain sufficient, appropriate evidence to provide a reasonable basis for  our findings and conclusions based on our audit objectives. We believe  that the evidence obtained provides a reasonable basis for our findings  and conclusions based on our audit objectives.

section_title: Appendix II: ARPA-E Program Technology Areas
paragraphs: Biomass energy. Biomass energy projects focus on means to convert  crops, along with plant waste from other industrial processes, into  energy through chemical, biological, or thermal techniques.

Building efficiency. Building efficiency projects focus on technologies  that heat, power, and maintain buildings.

Carbon capture. Carbon capture and sequestration projects seek to  create new methods to prevent the release of carbon dioxide into the  atmosphere from traditional fossil fuel sources such as coal, natural  gas, and petroleum.

Conventional energy. Conventional energy projects seek to  significantly increase the efficiency of traditional fossil fuel power  production and reduce waste generated from this use.

Direct solar fuels. Direct solar fuel projects seek to utilize  photosynthetic microorganisms to produce liquid fuels and fuel  precursors directly from solar energy.

Energy storage. Energy storage projects seek to revolutionize battery,  capacitor, and other energy storage methods for significantly  improved efficiency.

Energy-efficient water purification. Water technology projects seek to  reduce the water intensity of the electricity and fuel sectors and,  reciprocally, to reduce the energy intensity of the water sector.

Renewable power. Renewable power projects focus on innovative  technologies in several sustainable energy areas such as extremely  efficient photovoltaic solar collectors, wind turbines, and geothermal  energy.

Vehicle technologies. Vehicle technology projects seek to advance  efficiency in vehicles through technologies like new hybrid engines to  those that convert on-board waste heat to electricity.

Waste heat capture. Waste heat capture projects seek to use thermal  energy expelled by traditional industrial processes, such as coal  smokestacks, and efficiently convert that heat into electricity.

Batteries for transportation. Batteries for Electrical Energy Storage in  Transportation (BEEST) projects seek to develop batteries for plug-in  hybrid electric vehicles (PHEV) and electric vehicles (EV) that can  make a 300- to 500-mile-range electric car.

Materials for carbon capture. Innovative Materials and Processes for  Advanced Carbon Capture Technologies (IMPACCT) projects seek to  reduce the cost of carbon capture significantly through a combination  of new materials, improvements to existing processes, and  demonstration of new capture processes.

Electrofuels. Electrofuels projects intend to explore new paradigms for  the production of renewable liquid fuels that are compatible with  today’s infrastructure. They seek to use microorganisms to harness  chemical or electrical energy to convert carbon dioxide into liquid fuels  without using petroleum or biomass.

Grid-scale electricity storage. Grid-Scale Rampable Intermittent  Dispatchable Storage (GRIDS) projects seek to develop new energy  storage technologies that are comparable in reliability and cost to  pumped hydropower and that are modular and can be deployed in  any location in the country.

Building efficiency. Building Energy Efficiency Through Innovative  Thermodevices (BEETIT) projects focus on developing new  approaches and technologies for cooling equipment used in heating,  ventilating, and air conditioning (HVAC) systems in buildings, as well  as in refrigeration.

Electrical power electronics. Agile Delivery of Electrical Power  Technology (ADEPT) projects strive to reinvent the basic building  blocks of circuits from transistors, inductors, and transformers to  capacitors for a broad spectrum of power applications. ADEPT  focuses on two areas: (1) creating the world’s first kilovolt-scale  integrated circuits, and (2) developing transistor switches operating at  grid-level voltages that would exceed 13 kilovolts.

Advanced thermal storage. High Energy Advanced Thermal Storage  (HEATS) projects seek to develop revolutionary cost-effective thermal  energy storage technologies. HEATS focuses on three areas: (1)  high-temperature storage systems to deliver solar electricity more  efficiently around the clock to allow nuclear and fossil base load  resources the flexibility to meet peak demand, (2) fuel produced from  the sun’s heat, and (3) HVAC systems that use thermal storage to  dramatically improve the driving range of electric vehicles.

Electricity network integration. Green Electricity Network Integration  (GENI) projects focus on innovative control software and high-voltage  hardware to reliably control the grid network. GENI focuses on two  areas: (1) cost-optimizing controls to manage sporadically available  sources, such as wind and solar, alongside coal and nuclear, and (2)  resilient power flow control hardware to enable automated, real-time  control of grid components.

Plants Engineered To Replace Oil (PETRO). PETRO projects seek to  advance technologies that optimize the biochemical processes of  energy capture and conversion in plants to develop farm-ready crops  that deliver more energy per acre with less processing.

Rare earth alternatives for energy technologies. Rare Earth  Alternatives in Critical Technologies for Energy (REACT) projects  work on early-stage technology alternatives that reduce or eliminate  dependence on rare earth materials that may jeopardize the  widespread adoption of many critical energy solutions by developing  substitutes in two key areas: electric vehicle motors and wind  generators.

Solar electrical power technology. Solar Agile Delivery of Electrical  Power Technology (Solar ADEPT) projects focus on integrating  advanced power electronics into solar panels to extract and deliver  energy more efficiently. Solar ADEPT projects are centered on  advances in magnetics, semiconductor switches, and charge storage.

section_title: Appendix III: Description of ARPA-E Award Winners with Prior Private Investment
paragraphs: The 18 award winners we identified as having received prior private  venture capital told us that with the ARPA-E funding, they were generally  able to pursue the development of energy technologies with greater  scientific or technical uncertainty than they had when they were working  with their private funding. About two-thirds of these award winners told us  that the ARPA-E funding is allowing them to develop prototypes or to  prove basic technology concepts on more advanced ideas than their prior  work—6 of these award winners said this was for completely new  research and 7 said it was for major advancements to prior research. A  few of these award winners also told us they were able to work on  projects with outstanding scientific research questions that private  investors would not have funded. Five of these award winners reported  that they would likely have been able to pursue some research similar to  their ARPA-E projects, but it would have taken years longer without  ARPA-E funding.

The following three examples reflect in more detail much of what we  heard from these 18 award winners regarding the distinction between  research funded by their prior investors and ARPA-E funded work:

Sun Catalytix. Sun Catalytix was founded by a professor at the  Massachusetts Institute of Technology to commercialize a set of  catalysts to split water into hydrogen and oxygen gases. This reaction  allows these gases to be cheaply produced for a variety of purposes,  including renewable energy. Sun Catalytix was initially funded by a  Boston area venture capital firm to develop a product based on these  catalysts. According to a representative from this firm, the venture  capital funding allows Sun Catalytix to attempt to develop a product  that would potentially earn the venture firm a return on investment in a  reasonable amount of time. At the time of the ARPA-E award, Sun  Catalytix representatives told us they were still some years away from  a commercial product using this new technology. According to these  representatives, had the firm not won an ARPA-E award, further  venture capital might not have been available to develop an initial  version of their products. Sun Catalytix representatives told us that  ARPA-E funds allowed the company to conduct additional applied  scientific research that led to their discovery of a new platinum-free  and therefore lower-cost catalyst with much wider market potential,  including renewable energy applications. A representative from the  venture firm told us that the firm would not have funded the additional  advanced scientific research needed to develop the new, cheaper  catalyst.

Agrivida. This small biotechnology company based in the Boston area  is developing genetically modified sorghum, corn, and switchgrass  crops for use in biofuel production. Agrivida representatives explained  that the goal of their ARPA-E project is to generate crops capable of  producing enzymes within the plant itself to internally break down the  plant’s own cellulose after harvest. This technology would significantly  lower the costs of cellulosic biofuel production, because enzyme  treatments are currently a large part of the costs of current production  methods. Before winning an award from ARPA-E, Agrivida had  received venture capital funding to develop the technology. We spoke  with a representative of the venture firm that funded Agrivida, who told  us that this venture funding was only for research on the corn crop  enzymes; the firm was not willing to fund additional research on other  crops because the amount of funding it could provide to any one  company in the early stages of research was limited. Agrivida officials  told us that the ARPA-E award allowed them to expand the scope of  their work and conduct additional research on switchgrass, which may  have potential to become a major biofuel crop. They said that the  ARPA-E funds have enabled rapid progress, allowing them to  complete laboratory work in 1 year that would have otherwise taken 5  years. Officials from Agrivida said they hope to have made enough  progress by the time they complete their ARPA-E research to be able  to attract additional investors and secure commercialization partners.

24M. This is a startup company that is developing flow batteries for  use in transportation and electrical grid applications. Unlike normal  batteries, flow batteries generate electrical current by internally  circulating electrically active liquids, which allows for much lower costs  than traditional batteries. However, flow batteries do not exist for use  in tight spaces like cars where their cost advantages could allow for  significant improvements to electric vehicles. In 2010, concurrent with  its ARPA-E award, 24M received $10 million from two venture capital  firms to develop flow batteries for consumer and commercial  applications. A representative from one of these venture firms told us  that his firm would not have been confident in funding the 24M project  without ARPA-E involvement. Representatives from 24M said that the  ARPA-E award was critical to their ability to secure private investment  and to launch the company and that they now expect to have a  working prototype by the end of their ARPA-E project.

In addition, the two public companies we spoke with that were awarded  ARPA-E money told us that although their companies had internal  resources devoted to research and development, they were not able to  internally fund the projects they proposed to ARPA-E. They told us there  were two reasons for this. First, the companies said that existing product  lines placed heavy demands on their internal research and development  budgets, and that there is continuous pressure from existing customers  and competitors to improve existing products; since ARPA-E projects  were still a number of years away from a return on investment, these  investments could not be justified. Second, these companies told us that  internal investments had to meet minimum investment return thresholds,  and that ARPA-E-type projects were not able to meet these thresholds.  Officials from one company told us that the rate of return on investment  required by its management was at least 20 percent per year.

section_title: Appendix IV: Comments from ARPA-E

section_title: Appendix V: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact

section_title: Staff Acknowledgments
paragraphs: In addition to the individual named above, Tim Minelli (Assistant Director),  Paola Bobadilla, Cindy Gilbert, Robert Marek, Justin Mausel, Alison  O’Neill, Jeanette Soares, and Franklyn Yao made important contributions  to this report.


Question-summary annotations: questions: question: Why is ARPA-E working with other DOE offices?
answer: ARPA-E officials have taken steps to coordinate with other DOE offices to avoid duplication.
child_questions: question: How can this collaboration help to address funding gaps?
answer: For example, ARPA-E program directors told GAO they engage in outreach with officials from related DOE offices in advance of funding announcements to identify funding gaps in research.

question: How do officials from other offices and agencies contribute to the ARPA-E application review process?
answer: In addition, program directors have recruited officials from other DOE offices and the Department of Defense (DOD) to review ARPA-E applications.
child_questions: question: What is a potential benefit to this type of cross-agency interaction?
answer: This cross-agency interaction may also reduce the potential for overlap in funding.