Department of Energy: Advanced Research Projects Agency-Energy Could Benefit from Information on Applicants' Prior Funding
Released: 2012-01-24T00:00:00

Letter

Background

In 2005, the National Academies recommended to Congress the creation of an organization within DOE like DARPA. In 2007, the America COMPETES Act created a new agency within DOE called ARPA-E. In line with the National Academies’ recommendation, the America COMPETES Act as amended directs ARPA-E to achieve its goals by identifying and promoting revolutionary advances in fundamental and applied sciences, translating scientific discoveries and cutting-edge inventions into technological innovations, and accelerating transformational technological advances in areas that industry by itself is not likely to undertake because of technical and financial uncertainty. As such, ARPA-E officials told us that ARPA-E was designed to sponsor research beyond basic science, yet riskier than what the private sector (See fig. 1.) The National alone or DOE’s applied offices would support.Academies recommended that ARPA-E should not perform research and development itself, but should fund it to be conducted by universities and others in the private sector. In 2009, the American Recovery and Reinvestment Act of 2009 appropriated $400 million for ARPA-E.

ARPA-E is an agency with fewer than 30 federal employees, and its eight program directors, who are generally scientists and engineers, create and manage funding programs for the agency. ARPA-E’s program development and award selection process takes 6 to 8 months from start to finish, beginning when the agency hires a program director for a 3-year term and tasks the program director with identifying a gap in energy technology research and developing a program to fill that gap. For example, ARPA-E’s batteries for transportation program, called the Batteries for Electrical Energy Storage in Transportation (BEEST) program, was established to fill a gap in existing federal research programs on batteries for electric vehicles. Identifying these gaps and designing the program involves research; consultation with scientific experts, including a workshop with outside experts; and internal discussion at ARPA-E. From this process, program directors develop funding announcements that describe the technical requirements specific to each program’s technology area that applicants have to meet and the four criteria that ARPA-E uses in its selection process. The four criteria are the Impact of the proposed technology relative to the state of the art. The applicant must demonstrate the potential for a transformational—not incremental––advancement over current technologies. (See fig. 2.) More specifically, the applicant must demonstrate an awareness of competing commercial and emerging technologies and identify how its proposed concept/technology provides significant improvement over these other solutions.

Overall scientific and technical merit. The applicant must demonstrate that the work is unique and innovative. The applicant must also demonstrate a sound technical approach to accomplish the proposed research and development objectives. The outcome and deliverables of the program, if successful, should be clearly defined. Specific technical requirements that are unique to each individual ARPA-E program funding announcement must also be addressed.

Qualifications, experience, and capabilities. The applicant must demonstrate that it has the expertise and experience to accomplish the proposed project. In addition, the applicant must have access to all facilities required to accomplish the research and development effort.

Sound management plan. The applicant must have a workable plan to manage people and resources. Major technical research and development risks should be identified. The schedule and budget should be reasonable.

ARPA-E employs the following three-stage application process:

Concept paper. Applicants initially submit a 5- to 7-page abstract of their projects. Scientific experts from industry, government, and academia serve as reviewers.

Full application. After reviews of the concept paper, ARPA-E encourages some applicants to submit full applications using ARPA-E’s online application system. ARPA-E’s current instructions request that applicants provide, among other things, information about other prior, current, and pending public and private sources of funding, as well as why other funding sources are not willing to fund the projects. Full applications are then reviewed by leading scientific experts in the field, who evaluate them against the four criteria and assign numerical scores.

Reply to reviewer comments. After assessing the full applications, reviewers provide comments and questions to the applicants, who then have the opportunity to respond.

The applications with the reviewers’ comments are forwarded to a three- person panel beginning the next three phases of ARPA-E’s award funding process, which are as follows:

Selection. The three-person panel, usually chaired by the relevant program director, considers the reviewers’ comments and numerical scores, and recommends applications to award. The final decisions on which applicants to select are made by the selecting official, which is usually the ARPA-E Director.

Award negotiations. Negotiations proceed for approximately 2 months. Program directors work closely with the award winners to set up a project plan with technical milestones that are to be met during the funding of the award, which are planned to last between 2 and 3 years. Funds are awarded following the negotiations.

Monitoring. ARPA-E monitors and supports the project through quarterly reviews and site visits. After about 1 year, the agency decides whether to continue or terminate the project if the agreed-to milestones are not met.

In April 2009, ARPA-E started its funding award process by releasing a funding announcement soliciting proposals for all energy ideas and technologies. Following the review process, 36 projects were awarded funds after being selected from 3,700 applications that spanned the technology areas of 10 programs. ARPA-E released additional funding announcements in December 2009, March 2010, and April 2011. (See table 1.) Money appropriated by the American Recovery and Reinvestment Act of 2009 funded ARPA-E’s first three funding rounds. After receiving an appropriation in DOE’s fiscal year 2011 appropriations act, ARPA-E announced a fourth round of funding in April 2011.

In Addition to Its Selection Criteria, ARPA-E Also Considers Applicants’ Prior Sources of Private Funding; However, Most Award Winners We Reviewed Did Not Explain This Information

In addition to applying its four criteria, ARPA-E gives program directors discretion to use additional considerations to award funds to projects, including whether ARPA-E applicants received private funding. Most ARPA-E award winners did not receive prior private funding, but for those that did, most award winners we reviewed did not explain these funds.

ARPA-E Program Directors Use the Agency’s Four Selection Criteria and Other Considerations to Select Projects

Of the 20 applications we reviewed for award selection criteria, all contained supporting information addressing the agency’s four criteria. In our analysis of the ARPA-E reviewers’ evaluations from these 20 applications, we noted regular assignment of numerical scores rating applicants on the extent to which they met the criteria. All eight ARPA-E program directors told us they considered or, if they were recently hired, will consider all four criteria, but several focused more heavily on two criteria—the impact of the proposed technology relative to the state of the art and its overall scientific and technical merit.

In addition to basing the numerical scores applicants receive on the extent to which they meet the four selection criteria, program directors told us the agency gives them the ability to take other qualitative considerations into account when awarding funds. One of those considerations is to fund a broad range of potential technological solutions with varying levels of risk in solving a given technical problem. Two program directors selected projects to reflect a variety of technologies, and they told us they believe that this approach increases their programs’ overall chances of success. Specifically, one program director told us he chose projects that employed a variety of new battery technologies, a strategy that should increase the likelihood that at least one of them will work. This program director also chose some battery projects with much higher potential storage capacity but with a lower probability of success in achieving project milestones and in ultimately being brought to market. In those cases, ARPA-E provided smaller awards to the projects with the lower probability of success.

Several program directors also told us that during the selection process, they considered the applicants’ projects’ proposed project scope and duration, requested funding levels, and technical milestones and negotiated to revise these, if necessary, to better align applicants’ projects with ARPA-E’s program goals. According to our review of ARPA- E data from the first three rounds of funding, the agency reduced requested award amounts by 5 percent or more on 31 out of 121 projects, for a total of $59 million below total requested award amounts for these rounds. When ARPA-E makes these kinds of adjustments, the agency may also reduce the proposed project scope to fund only what the program directors consider to be the transformational part of the project and to avoid funding applied research or development work that would be outside ARPA-E’s program goals. For example, the agency reduced the award amount and proposed project scope for an energy storage technology project designed to improve energy storage on the electrical grid. The project proposal initially requested nearly $5 million to demonstrate the technology at nearly full scale. During award negotiations, ARPA-E reduced this amount to $750,000 to focus the project only on smaller-scale development and testing of the technology. ARPA-E officials told us the larger-scale demonstration could likely be funded by the private sector.

ARPA-E Also Considers Applicants’ Prior Sources of Private Funding, but Most Award Winners We Reviewed Did Not Explain This Information

When making award decisions or adjusting the scope of proposed projects, ARPA-E program directors may also consider the identification in applications of sources of private funding and the extent to which that funding might support the proposed projects. This information can help provide program directors with assurance that ARPA-E funds do not overlap with private investment. During the first two funding rounds, ARPA-E required that applicants identify relevant private investors if the applicant believed these funds were related to the proposed project. Of the 18 applications we reviewed from award winners that we identified as having received private venture capital, 14 applied during ARPA-E’s first and second funding rounds. Most of these award winners did not explain why investors were not willing to fund proposed work. ARPA-E program directors and an ARPA-E official, speaking on behalf of the agency, told us they took additional steps to clarify outstanding prior funding questions when ARPA-E was aware that applicants had received private sector funding. For example, one applicant we reviewed from the first funding round had previously received substantial private funding for work that appeared very similar to its proposed ARPA-E project. ARPA-E officials told us they were initially unable to determine why the private investor was not willing to also fund the proposed ARPA-E project and that the company’s application did not include an explanation. ARPA-E officials told us that getting this information required them to draft a series of direct and detailed questions that elicited several pages of written responses from the applicant. ARPA-E officials also told us they conducted multiple rounds of written and oral follow-up with the applicant and the private investor. Through these efforts, ARPA-E determined that the technological risks of key parts of the project were too high for the private investor and therefore decided to fund the research.

Because ARPA-E officials recognized the need for applicants to provide better prior funding explanations, beginning with its third funding announcement, the agency required applicants to explain why proposed work was not sponsored internally if the applicant was a large company, or why private investors were not willing to support the project if the applicant was a small business or start-up company. ARPA-E did not provide guidance on how applicants should respond to this additional requirement by, for example, providing a sample response. Of the 18 ARPA-E award-winning companies, 4 applied during the third funding round, and these companies provided a range of information in response to this new requirement in their funding applications. Two explained how ARPA-E funds would allow them to go beyond currently funded work but did not provide reasons why investors were not willing to support the proposed work. Another wrote only that the ARPA-E research was too risky for the company’s private investors. One application contained an explanation outlining the specific research its private investors were and were not willing to fund. This applicant explained that private funds were directed toward lower-risk and higher-cost technologies. This application also included a letter from the company’s venture capital investors that explained which parts of its research the investors were planning to continue funding and which research was too risky for them, although not requested by ARPA-E. This letter provided additional third-party support for the funding information in the application. Officials from the National Institute of Standards and Technology’s (NIST) Technology Innovation Program told us they request that applicants provide letters from private investors to document why applicants’ projects could not be privately funded.

When we followed up with the 18 companies, they were generally able to explain to us why their private investors were not willing to undertake the additional risk and uncertainty associated with the proposed projects. When we examined the data in the VentureDeal database for a number of applicants, the data allowed us to quickly cross-check the names of prior private investors that applicants reported to ARPA-E. ARPA-E officials said that they have not used such data for these purposes but that they have considered doing so. Without an examination of outside venture capital data on its applicants, the agency may be missing a time-saving opportunity to check information on private funding provided in applications, especially in instances where applicants may not have been thorough in their explanations. We found a number of readily available subscription-based venture capital data services that provided company names, transaction amounts, and funding purposes. We found that the web-based VentureDeal database matched formats and data available from other venture capital data services.

Most ARPA-E Projects Likely Could Not Have Been Funded Solely by Private Investors

Our review suggests that most ARPA-E-type projects could not be funded solely by private investors. Private venture capital firms told us that, among other considerations, they generally do not invest in projects that cannot be commercialized in less than 3 years. Nearly all of the 13 ARPA-E award winners and most of the 22 of the contingently selected applicants we spoke with estimated that their projects were 3 or more years away from a potential market-ready product (i.e., commercialization). In addition, we found that only 2 of the 22 contingently selected applicants we spoke with that met ARPA-E’s selection criteria but were not selected for an award subsequently secured private funding.

Venture Capitalists Generally Do Not Fund Projects That ARPA-E Looks to Fund

The representatives we spoke with from six venture capital firms identified three factors that limit the general availability of venture capital funding for new energy technologies. These factors were consistent with data we analyzed for the 121 award winners from ARPA-E’s first three funding rounds, the sample of 13 award winners we interviewed from these funding rounds, and the 22 contingently selected applicants we interviewed.

First, venture capital firms generally do not fund projects that rely on unproven technological concepts or lack working prototypes demonstrating the technology. A number of venture capital firm representatives told us that they are generally not willing to fund the applied scientific research sometimes required by ARPA-E-type projects. Projects they fund generally focus on developing technologies based on known scientific principles. Data from ARPA-E on award winners show that 91 out of 121 ARPA-E projects from the first three funding rounds had technological concepts that had not yet been demonstrated in a According to a recent report from the American laboratory setting.Energy Innovation Council, private investors consider these projects too high risk for investment, even for concepts with promising technological potential. Most of the contingently selected applicants we spoke with–– 17 out of the 22––told us they were unlikely to receive funding from other sources for their proposed projects because of high levels of scientific uncertainty, an unavailable or undeveloped market, or a lack of a working prototype. For example, one such applicant said that he only had a computer model suggesting that his high-efficiency air conditioning device would work, which was insufficient to convince potential private investors. In addition, many of the ARPA-E award winners we surveyed also recognized the inherent uncertainty in their research; 5 of the 13 told us that their projects had a fairly low probability of success.

Second, venture capital firms seek more rapid returns on investment and closely analyze a project’s potential return on investment over time, a factor that influences their decisions to invest in projects that are in later stages of development and closer to commercialization. Venture capital firm officials told us that they focused closely on the timeliness of investment returns, with one firm noting that the industry tended to invest in technologies that could be commercialized in less than 3 years and that would potentially exhibit exponential market growth in approximately 5 to 7 years. However, we found that nearly all of the ARPA-E award winners and most contingently selected applicants we spoke with estimated that their projects were 3 or more years away from potential commercialization. estimated that it would take at least 3 years for their ARPA-E projects to For example, 12 out of 13 ARPA-E award winners reach the commercialization stage with ARPA-E funding. Had they not received ARPA-E funding, most of these award winners––10 out of 13–– told us they either would not have pursued their ARPA-E project or that they would not have been able to develop a commercial product in less than 10 years. At the same time, 18 out of 22 ARPA-E contingently selected applicants estimated it would take at least 3 years for their projects to reach commercialization if they had been able to secure funding for the proposal they submitted to ARPA-E.

Third, venture capital firms may not be comfortable investing in new energy technologies, noting the historical lack of successful venture capital investments in these types of projects. Venture representatives said that venture firms were more comfortable investing in software companies or other businesses with higher potential profit margins and less costly product development than new energy technologies. One venture representative noted that his firm looked to invest in products with potential gross profit margins of 50 percent or more. In addition, these representatives noted that it is difficult for new advanced energy technologies to compete with well-established and low-margin traditional sources of energy like natural gas. Venture representatives also noted that venture firms had become more risk averse and reluctant to fund new energy technologies after lackluster investment returns have made the venture industry more aware of the challenges associated with investing in unproven energy technologies.

While venture capital firms generally do not fund projects that ARPA-E looks to fund, our work suggests that receiving ARPA-E project funding may have a positive effect on some award winners’ ability to attract follow-on funding from the private sector for their ARPA-E work. For example, ARPA-E’s data indicate that 18 out of 121 ARPA-E award winners from ARPA-E’s first three rounds of funding had received private sector funding totaling $318 million after receiving ARPA-E funding. In some cases, award winners received private follow-on funding immediately after receiving ARPA-E funding. A number of the award winners we spoke with stated that, given the highly competitive nature of the program, receiving ARPA-E funding served as a “stamp of approval” to venture capital or other private firms. These award winners told us that an ARPA-E award served as a signal of scientific and financial approval for potential investors. Economists call this rapid follow-on private funding a certification effect, which may explain the experiences of some of these award winners. This effect suggests that public awards address information gaps that might have otherwise precluded private investment. Some award winners and economists we spoke with told us that the government was suited to identifying technical risks because of its ability to draw on the expertise of many scientific reviewers, while venture firms may not have the scientific expertise on hand to fully understand potential investments. Furthermore, economic literature suggests that the certification effect may be particularly relevant in the high-technology industries, where the venture capital community plays an important role and in which traditional financial measures of risk and returns on investments may prove insufficient.information on the difference between research funded by selected award winners’ prior investors and ARPA-E funded work.

Few Contingently Selected Applicants Found Funding from Private Investors or Public Sources

Eighteen of the 22 ARPA-E contingently selected applicants we interviewed sought funding after being turned down for ARPA-E funds. Of the 18 that sought funding elsewhere, 13 submitted project proposals to government sources, such as other DOE offices, the National Science Foundation, or nonprofit academic research institutes, and the remaining 5 submitted proposals to private investors such as venture capital firms. As of September 2011, we found that 2 out of the 22 contingently selected applicants secured funding from venture capital firms for work that was very similar to their ARPA-E project proposals.that 4 contingently selected applicants secured funding from a We also found government or nonprofit source for their projects. In addition, we found that most contingently selected applicants modified their ARPA-E project proposals to attract subsequent funding for their projects by reducing the scope of their proposals or by focusing on more basic science research. For example, the 4 contingently selected applicants that secured funding from a government or nonprofit source modified their ARPA-E proposals to be more focused on basic science research, rather than on developing a commercial technology. In addition, 1 of these applicants told us that the funding will allow it to continue exploring fundamental materials science rather than developing a product. Also, many contingently selected applicants and award winners said that other government sources were limited. Some noted that Small Business Innovation Research (SBIR) grants would not allow them to make as much progress as larger ARPA-E awards.military funding agencies were not as focused on developing low-cost technologies with broader market appeal, because aerospace or military applications do not need to achieve the same low costs and market appeal as consumer or commercial applications.

ARPA-E Officials Have Taken Steps to Coordinate with Other Department of Energy Offices in Advance of Awarding Funds

According to ARPA-E officials and documents, agency officials have taken steps to coordinate with other DOE offices in advance of awarding ARPA-E funds to help avoid duplication of efforts. These coordination efforts can be categorized into three areas: Prefunding coordination. ARPA-E officials told us that program directors engage with officials from related DOE offices in advance of announcing the availability of ARPA-E funds. ARPA-E program directors told us that early in the development of a funding announcement, they conduct outreach with industry, academic, and government officials both inside and outside of DOE in an attempt to identify funding gaps related to the technology they wish to develop. For example, by doing such outreach, one program director determined that there had been little funding at DOE or elsewhere for lithium air or lithium sulfur batteries, which have the potential to last significantly longer than existing lithium ion batteries. Program directors also hold workshops and invite relevant participants, including those from other DOE offices and from other federal agencies, to identify technologies that have little to no existing research funding but that have transformational potential. ARPA-E officials told us that directors use the workshops and other meetings to identify research areas that other DOE offices are not working on, and the other DOE officials provide insights on funding areas where they are not active. For example, one of these ARPA-E program directors told us that he met with officials from DOE’s Office of Electricity Delivery and Energy Reliability and the Solar Energy Technologies Program within the Office of Energy Efficiency and Renewable Energy (EERE) before announcing available funds for the electrical power electronics funding announcement. According to this program director, this coordination helped him identify that there had been little funding for the development of magnetic devices for use in electrical power electronics. He ultimately designed the ARPA- E electrical power electronics funding announcement to focus, in part, on the development of improved magnetic devices because of the lack of funding elsewhere.

In addition to inviting officials from other DOE offices to ARPA-E workshops, program directors told us they also engage with other DOE officials in other ways, both formally and informally. The program director responsible for ARPA-E’s work on advanced batteries said that he was a member of DOE’s Energy Storage Technology Development Team and regularly met with other officials who are engaged in applied battery research. This director said that it had become clear that DOE’s Vehicle Technologies Program will continue to focus on incremental improvements to existing lithium ion battery technologies that are currently on the market, while ARPA-E will fund newer, alternative battery technologies. Other program directors told us that they have regular discussions with counterparts within DOE to avoid duplicating efforts, although through other means than a formal committee.

Coordination of application reviews. Some ARPA-E program directors told us that they have recruited officials from other DOE offices to review applications submitted to ARPA-E and that these officials made up as many as one-third of the reviewers for one director. These application reviewers rate and recommend proposals for potential ARPA-E funding. ARPA-E program directors told us that these DOE reviewers help them stay aware of the types of projects that other DOE offices are funding. For example, according to one program director, DOE reviewers indicated on a number of occasions that an ARPA-E advanced battery applicant would be better suited for funding under DOE’s Vehicle Technologies Program because it was for a more developed technology. ARPA-E has also used application reviewers from other federal agencies, such as the Department of Defense. One program director told us that these reviewers have also helped avoid funding projects similar to those potentially funded elsewhere.

Official DOE coordination groups. ARPA-E is also a participant in DOE’s SunShot Initiative within the Solar Energy Technologies Program. The SunShot Initiative is an effort to coordinate solar energy research across DOE’s Office of Science, four national laboratories, the National Science Foundation, and ARPA-E, with the goal of achieving costs of $1 per watt for solar-generated electricity. One ARPA-E program director is a member of the SunShot Initiative advisory board and therefore able to coordinate ARPA-E solar-related activities with other SunShot Initiative members. SunShot Initiative officials told us that DOE plans to make it a model for DOE’s internal coordination efforts and that DOE hopes to expand the approach to other research areas.

Additionally, the ARPA-E Director created the Panel of Senior Technical Advisors (PASTA), which is a group of high-level DOE managers that meet periodically to discuss current and future DOE research efforts. ARPA-E officials told us that PASTA is an attempt to avoid duplicating efforts within DOE. PASTA meeting attendees have included officials from DOE’s applied and basic science offices.

We were not able to directly evaluate the effectiveness of ARPA-E’s efforts to coordinate with other DOE offices. Nevertheless, we found that on the basis of our interviews with ARPA-E award winners and contingently selected applicants, four award winners and two contingently selected applicants had received prior funding from other DOE offices.According to these award winners and contingently selected applicants, the prior funding was either for more proven technologies or was focused on more basic or foundational research than was the ARPA-E funded project.

Conclusions

ARPA-E recognizes the need to ensure that the agency is not funding projects that would be otherwise funded by the private sector, and has taken steps to get information from applicants on their other sources of funding. The agency has also taken steps to coordinate with other DOE offices in advance of awarding ARPA-E funds. However, for the applications we reviewed, we found that ARPA-E’s current funding announcements have generally yielded limited information from applicants that had prior sources of private funding. Where applicants provided little information, ARPA-E’s program directors spent time and resources to determine the extent of such funding for projects related to or similar to the applicants’ proposed ARPA-E projects. The agency’s requirements for information on private sector funding could be improved. For example, ARPA-E does not provide guidance to applicants, such as a sample response, on how to meet its information requirement on prior private funding. An approach used by another federal program that funds advanced research is for applicants to provide letters from private investors to document why their projects could not be privately funded. This approach was used by one ARPA-E award winner, who included a letter from the company’s venture capital investors to explain why the investors were not willing to fund the project proposed to ARPA-E. Also, ARPA-E officials said that they have not used venture capital data to identify applicants with prior private investors and to check information applicants provide to them, but that they have considered doing so. Examining such data allowed us to quickly cross-check applicants’ self- reported prior private funding. Without additional tools to better understand prior private funding, ARPA-E program directors will continue to spend time and agency resources taking additional steps to clarify prior private funding and may miss opportunities to avoid duplication with private investors.

Recommendations for Executive Action

To ensure that ARPA-E uses a more complete range of methods to ensure that limited federal funds are targeted appropriately, we recommend that the Secretary of Energy consider taking the following three actions: provide guidance with a sample response to assist applicants in providing information on sources of private funding for proposed ARPA-E projects, require that applicants provide letters or other forms of documentation from private investors that explain why investors are not willing to fund the projects proposed to ARPA-E, and use venture capital funding databases to help identify applicants with prior private investors and to help check information applicants provide on their applications.

Agency Comments and Our Evaluation

We provided a copy of our draft report to ARPA-E for review and comment. ARPA-E concurred with key findings and our recommendations in its written comments, which are reproduced in appendix IV. In its comments, ARPA-E outlined the steps that the agency plans to take to address our recommendations. ARPA-E also provided additional clarifying comments, which we incorporated.

As agreed with your offices, unless you publicly announce the contents of this report earlier, we plan no further distribution until 30 days from the report date. At that time, we will send copies of this report to the appropriate congressional committees, the Secretary of Energy, and other interested parties. In addition, the report will be available at no charge on the GAO website at http://www.gao.gov.

If you or your staffs have any questions about this report, please contact me at (202) 512-3841 or ruscof@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made major contributions to this report are listed in appendix V.

Appendix I: Scope and Methodology

To examine the Advanced Research Projects Agency-Energy’s (ARPA-E) use of criteria and other considerations for making awards and the extent to which applicants identify and explain other private funding information, we reviewed 20 applications drawn from a nonprobability sample of the 4,788 applicants ARPA-E received during its first three funding rounds. We selected applications from a range of ARPA-E technology program areas to which the applications were submitted and applicant institution types (e.g., small company or university). Because we selected a nonprobability sample of applications to review, information we collected cannot be generalized to all applicants; however, it provided us with an understanding of ARPA-E’s criteria and other considerations for making an award. We also interviewed and reviewed the applications from our sample of 18 award winners, which were private companies that we identified as having received funding from private investors prior to receiving an ARPA-E award. We identified these 18 companies by searching for evidence of prior private funding for the 121 award winners in the VentureDeal venture capital database. In our review of these applications, we focused on the extent to which applicants disclosed prior private funding. We also spoke with all eight ARPA-E program directors to discuss ARPA-E’s process for making awards and managing projects of award winners.

To analyze the extent to which ARPA-E projects could have been funded through the private sector, we conducted three sets of interviews with ARPA-E applicants and award winners. Specifically,

We conducted structured interviews with 22 of the 33 contingently selected applicants that ARPA-E encouraged to submit full applications during its second and third funding rounds.

Each of the contingently selected applicants fulfilled ARPA-E’s selection criteria, had the same characteristics as ARPA-E award winners, and, according to ARPA-E officials with whom we spoke, would have been selected for an award had additional funds been available. This approach allowed us to consider the potential of ARPA-E-type projects to receive private funding.

We conducted structured interviews with a nonprobability sample of 13 award winners selected from ARPA-E’s first three funding rounds. We selected subjects for this sample across a range of ARPA-E award winner characteristics, including the technology program area for which an award winner received funding, the stage of development of an award winner’s project, and an award winner’s type of institution (e.g., small company or university). Because this was a nonprobability sample, the information from these structured interviews cannot be generalized to all award winners but can provide examples about award winners’ experiences. We conducted content analyses of the award winners’ and contingently selected applicants’ interview responses to quantify issues such as the ability of each group to secure private sector funding for ARPA-E-type projects.

Third, we spoke with the 18 ARPA-E award winners we identified through the VentureDeal database to discuss key differences between their prior research and their ARPA-E-funded projects.

We also conducted interviews with a variety of companies and individuals knowledgeable about research associated with ARPA-E-type projects, including six venture capital firms and the National Venture Capital Association (NVCA), a trade association, to determine the availability of private capital for ARPA-E-type projects and the criteria venture capital firms apply in making their investment decisions; two additional public companies that were awarded ARPA-E funding to discuss the ability of a public company to internally fund research; and three economists to discuss the role and effectiveness of government-funded research and development of technology.

To examine the extent to which ARPA-E coordinates with other DOE programs to avoid duplicating efforts, we spoke with the ARPA-E program directors as well as officials from other DOE program offices including the Office for Energy Efficiency and Renewable Energy (EERE) and the Office of Science. In addition, we met with officials from the SunShot Initiative, which is a collaboration among EERE, the Office of Science, and ARPA-E to make solar energy technologies cost-competitive with other forms of energy. We also spoke with officials from the Defense Advanced Research Projects Agency (DARPA) and the Department of Energy (DOE) Office of Inspector General. During our interviews with the award winners and contingently selected applicants previously mentioned, we asked them to discuss their understanding of other potential sources of DOE funding for their projects.

To assess the reliability of data from ARPA-E and VentureDeal, we reviewed relevant documentation and interviewed key data system officials at ARPA-E and VentureDeal and determined that the data were sufficiently reliable for the purposes of this report.

We conducted this performance audit from November 2010 to December 2011 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Appendix II: ARPA-E Program Technology Areas

Biomass energy. Biomass energy projects focus on means to convert crops, along with plant waste from other industrial processes, into energy through chemical, biological, or thermal techniques.

Building efficiency. Building efficiency projects focus on technologies that heat, power, and maintain buildings.

Carbon capture. Carbon capture and sequestration projects seek to create new methods to prevent the release of carbon dioxide into the atmosphere from traditional fossil fuel sources such as coal, natural gas, and petroleum.

Conventional energy. Conventional energy projects seek to significantly increase the efficiency of traditional fossil fuel power production and reduce waste generated from this use.

Direct solar fuels. Direct solar fuel projects seek to utilize photosynthetic microorganisms to produce liquid fuels and fuel precursors directly from solar energy.

Energy storage. Energy storage projects seek to revolutionize battery, capacitor, and other energy storage methods for significantly improved efficiency.

Energy-efficient water purification. Water technology projects seek to reduce the water intensity of the electricity and fuel sectors and, reciprocally, to reduce the energy intensity of the water sector.

Renewable power. Renewable power projects focus on innovative technologies in several sustainable energy areas such as extremely efficient photovoltaic solar collectors, wind turbines, and geothermal energy.

Vehicle technologies. Vehicle technology projects seek to advance efficiency in vehicles through technologies like new hybrid engines to those that convert on-board waste heat to electricity.

Waste heat capture. Waste heat capture projects seek to use thermal energy expelled by traditional industrial processes, such as coal smokestacks, and efficiently convert that heat into electricity.

Batteries for transportation. Batteries for Electrical Energy Storage in Transportation (BEEST) projects seek to develop batteries for plug-in hybrid electric vehicles (PHEV) and electric vehicles (EV) that can make a 300- to 500-mile-range electric car.

Materials for carbon capture. Innovative Materials and Processes for Advanced Carbon Capture Technologies (IMPACCT) projects seek to reduce the cost of carbon capture significantly through a combination of new materials, improvements to existing processes, and demonstration of new capture processes.

Electrofuels. Electrofuels projects intend to explore new paradigms for the production of renewable liquid fuels that are compatible with today’s infrastructure. They seek to use microorganisms to harness chemical or electrical energy to convert carbon dioxide into liquid fuels without using petroleum or biomass.

Grid-scale electricity storage. Grid-Scale Rampable Intermittent Dispatchable Storage (GRIDS) projects seek to develop new energy storage technologies that are comparable in reliability and cost to pumped hydropower and that are modular and can be deployed in any location in the country.

Building efficiency. Building Energy Efficiency Through Innovative Thermodevices (BEETIT) projects focus on developing new approaches and technologies for cooling equipment used in heating, ventilating, and air conditioning (HVAC) systems in buildings, as well as in refrigeration.

Electrical power electronics. Agile Delivery of Electrical Power Technology (ADEPT) projects strive to reinvent the basic building blocks of circuits from transistors, inductors, and transformers to capacitors for a broad spectrum of power applications. ADEPT focuses on two areas: (1) creating the world’s first kilovolt-scale integrated circuits, and (2) developing transistor switches operating at grid-level voltages that would exceed 13 kilovolts.

Advanced thermal storage. High Energy Advanced Thermal Storage (HEATS) projects seek to develop revolutionary cost-effective thermal energy storage technologies. HEATS focuses on three areas: (1) high-temperature storage systems to deliver solar electricity more efficiently around the clock to allow nuclear and fossil base load resources the flexibility to meet peak demand, (2) fuel produced from the sun’s heat, and (3) HVAC systems that use thermal storage to dramatically improve the driving range of electric vehicles.

Electricity network integration. Green Electricity Network Integration (GENI) projects focus on innovative control software and high-voltage hardware to reliably control the grid network. GENI focuses on two areas: (1) cost-optimizing controls to manage sporadically available sources, such as wind and solar, alongside coal and nuclear, and (2) resilient power flow control hardware to enable automated, real-time control of grid components.

Plants Engineered To Replace Oil (PETRO). PETRO projects seek to advance technologies that optimize the biochemical processes of energy capture and conversion in plants to develop farm-ready crops that deliver more energy per acre with less processing.

Rare earth alternatives for energy technologies. Rare Earth Alternatives in Critical Technologies for Energy (REACT) projects work on early-stage technology alternatives that reduce or eliminate dependence on rare earth materials that may jeopardize the widespread adoption of many critical energy solutions by developing substitutes in two key areas: electric vehicle motors and wind generators.

Solar electrical power technology. Solar Agile Delivery of Electrical Power Technology (Solar ADEPT) projects focus on integrating advanced power electronics into solar panels to extract and deliver energy more efficiently. Solar ADEPT projects are centered on advances in magnetics, semiconductor switches, and charge storage.

Appendix III: Description of ARPA-E Award Winners with Prior Private Investment

The 18 award winners we identified as having received prior private venture capital told us that with the ARPA-E funding, they were generally able to pursue the development of energy technologies with greater scientific or technical uncertainty than they had when they were working with their private funding. About two-thirds of these award winners told us that the ARPA-E funding is allowing them to develop prototypes or to prove basic technology concepts on more advanced ideas than their prior work—6 of these award winners said this was for completely new research and 7 said it was for major advancements to prior research. A few of these award winners also told us they were able to work on projects with outstanding scientific research questions that private investors would not have funded. Five of these award winners reported that they would likely have been able to pursue some research similar to their ARPA-E projects, but it would have taken years longer without ARPA-E funding.

The following three examples reflect in more detail much of what we heard from these 18 award winners regarding the distinction between research funded by their prior investors and ARPA-E funded work:

Sun Catalytix. Sun Catalytix was founded by a professor at the Massachusetts Institute of Technology to commercialize a set of catalysts to split water into hydrogen and oxygen gases. This reaction allows these gases to be cheaply produced for a variety of purposes, including renewable energy. Sun Catalytix was initially funded by a Boston area venture capital firm to develop a product based on these catalysts. According to a representative from this firm, the venture capital funding allows Sun Catalytix to attempt to develop a product that would potentially earn the venture firm a return on investment in a reasonable amount of time. At the time of the ARPA-E award, Sun Catalytix representatives told us they were still some years away from a commercial product using this new technology. According to these representatives, had the firm not won an ARPA-E award, further venture capital might not have been available to develop an initial version of their products. Sun Catalytix representatives told us that ARPA-E funds allowed the company to conduct additional applied scientific research that led to their discovery of a new platinum-free and therefore lower-cost catalyst with much wider market potential, including renewable energy applications. A representative from the venture firm told us that the firm would not have funded the additional advanced scientific research needed to develop the new, cheaper catalyst.

Agrivida. This small biotechnology company based in the Boston area is developing genetically modified sorghum, corn, and switchgrass crops for use in biofuel production. Agrivida representatives explained that the goal of their ARPA-E project is to generate crops capable of producing enzymes within the plant itself to internally break down the plant’s own cellulose after harvest. This technology would significantly lower the costs of cellulosic biofuel production, because enzyme treatments are currently a large part of the costs of current production methods. Before winning an award from ARPA-E, Agrivida had received venture capital funding to develop the technology. We spoke with a representative of the venture firm that funded Agrivida, who told us that this venture funding was only for research on the corn crop enzymes; the firm was not willing to fund additional research on other crops because the amount of funding it could provide to any one company in the early stages of research was limited. Agrivida officials told us that the ARPA-E award allowed them to expand the scope of their work and conduct additional research on switchgrass, which may have potential to become a major biofuel crop. They said that the ARPA-E funds have enabled rapid progress, allowing them to complete laboratory work in 1 year that would have otherwise taken 5 years. Officials from Agrivida said they hope to have made enough progress by the time they complete their ARPA-E research to be able to attract additional investors and secure commercialization partners.

24M. This is a startup company that is developing flow batteries for use in transportation and electrical grid applications. Unlike normal batteries, flow batteries generate electrical current by internally circulating electrically active liquids, which allows for much lower costs than traditional batteries. However, flow batteries do not exist for use in tight spaces like cars where their cost advantages could allow for significant improvements to electric vehicles. In 2010, concurrent with its ARPA-E award, 24M received $10 million from two venture capital firms to develop flow batteries for consumer and commercial applications. A representative from one of these venture firms told us that his firm would not have been confident in funding the 24M project without ARPA-E involvement. Representatives from 24M said that the ARPA-E award was critical to their ability to secure private investment and to launch the company and that they now expect to have a working prototype by the end of their ARPA-E project.

In addition, the two public companies we spoke with that were awarded ARPA-E money told us that although their companies had internal resources devoted to research and development, they were not able to internally fund the projects they proposed to ARPA-E. They told us there were two reasons for this. First, the companies said that existing product lines placed heavy demands on their internal research and development budgets, and that there is continuous pressure from existing customers and competitors to improve existing products; since ARPA-E projects were still a number of years away from a return on investment, these investments could not be justified. Second, these companies told us that internal investments had to meet minimum investment return thresholds, and that ARPA-E-type projects were not able to meet these thresholds. Officials from one company told us that the rate of return on investment required by its management was at least 20 percent per year.

Appendix IV: Comments from ARPA-E

Appendix V: GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

In addition to the individual named above, Tim Minelli (Assistant Director), Paola Bobadilla, Cindy Gilbert, Robert Marek, Justin Mausel, Alison O’Neill, Jeanette Soares, and Franklyn Yao made important contributions to this report.