Department of Energy: New Process to Review Financial Assistance for Research Projects Created Uncertainty
Released: 2018-02-28T00:00:00

Letter

Background

ARPA-E’s typical funding announcement and award selection process begins with the agency hiring a program director responsible for identifying a gap in energy technology research and developing a program to fill that gap. ARPA-E is required by statute to achieve its goals through energy technology projects that, among other things, accelerate transformational technological advances in areas that industry on its own is not likely to undertake because of technical and financial uncertainty, while also ensuring that its activities are coordinated with, and do not duplicate the efforts of, programs and laboratories within DOE and other relevant research agencies. ARPA-E’s efforts to identify existing energy technology research gaps and to design a program to address those gaps involve research; consultation with scientific experts, including a workshop with outside experts; and internal discussions within ARPA-E. From this process, program directors develop funding opportunity announcements that describe the technical requirements specific to each program’s technology area that applicants have to meet, as well as the four standard criteria that ARPA-E uses to guide its merit selection process. Following the issuance of a funding opportunity announcement, ARPA-E employs the following multi-stage process to merit review applications, make funding award decisions, and monitor projects:

Concept paper. Applicants initially submit a 4- to 7-page abstract of their projects. Scientific experts from government, industry, and academia serve as reviewers.

Full application. After reviewing concept papers, ARPA-E encourages some applicants to submit full applications. Full applications are generally quite extensive, requesting information on the technical and financial aspects of the proposed project, among other things. ARPA-E officials we interviewed noted that these applications are frequently more than 100 pages and can take 30 to 45 days for the applicant to develop. Full applications are reviewed against the selection criteria by leading scientific experts in the relevant field and assigned numerical scores.

Reply to reviewer comments. After reviewing a full application, reviewers provide comments and questions to the applicants, who then have the opportunity to respond.

Selection. A three- to four-person panel, chaired by the relevant ARPA-E program director, considers the reviewers’ comments and numerical scores and recommends applications for an award. The final decisions on which applicants to select for award negotiations are made by the selecting official, usually the Director of ARPA-E.

Award negotiations. Once selections are made, ARPA-E program directors work closely with selectees to negotiate the terms and conditions of their award. These negotiations include, among other things, developing a project plan with technical milestones that are to be met during the 2 to 3 years that the award is being funded, a budget and management plan, and an intellectual property and data management plan. Funds are awarded once negotiations regarding the terms and conditions of the award are concluded. ARPA-E seeks to complete negotiations regarding the terms and conditions of an award within approximately 100 days of sending a letter notifying an applicant that they have been selected for award negotiations.

Selectees may be allowed to begin spending money to start work on their projects up to 90 days prior to the completion of award negotiations. However, these expenditures are made with the risk that applicants may not be reimbursed if award negotiations are unsuccessful and ARPA-E does not fund the award.

Monitoring. ARPA-E monitors and supports the projects it funds through quarterly reviews and site visits. At any point during the award, ARPA-E may decide whether to continue or terminate the project based on whether agreed-upon project milestones are being met.

DOE Developed and Implemented a New Process to Assess the Department’s Financial Assistance against the Administration’s Priorities

In 2017, DOE developed and implemented a new review process to assess DOE financial assistance for new work against the current administration’s priorities, including financial assistance for which ARPA- E had already made award selections. DOE reviewed and approved ARPA-E’s opportunities for financial assistance on a rolling basis from May to September 2017, and nearly all were approved to proceed.

The formal review of DOE financial assistance officially began on May 4, 2017, when DOE’s Chief of Staff issued a memorandum stating that funding opportunity announcements and determinations of non- competitive financial assistance would be reviewed to ensure consistency with the administration’s priorities. According to the memorandum, DOE agencies that award financial assistance—referred to in this report as funding organizations—were to provide information about each competitively selected funding announcement and determination of non- competitive financial assistance by May 15, 2017. This information included, for example, a brief description of the financial assistance, the number and amount of planned awards, and the technology readiness level of the projects being funded. DOE Office of Management officials told us that the agency’s financial assistance review lasted through September 2017, as some DOE organizations continued to submit new financial assistance for review, but that the review was largely completed by August 10, 2017. However, while the formal review of DOE financial assistance began in May, award negotiations for ARPA-E-funded projects were suspended nearly 1 month earlier. Specifically, according to ARPA- E officials, DOE’s Deputy Chief of Staff verbally directed ARPA-E on April 6, 2017 to stop all ongoing award negotiations. Figure 1 shows the timeline of DOE’s review of ARPA-E financial assistance.

Pursuant to the DOE Chief of Staff’s May 4th memorandum, ARPA-E and other DOE funding organizations submitted the requested information to the DOE review team, which was coordinated and facilitated by the Director of DOE’s Office of Management. Other members of the financial assistance review team included DOE’s acting Chief Financial Officer; deputy assistant secretaries, chiefs of staff, and senior advisors at several DOE funding organizations; and members of the department’s congressional affairs and public affairs staff.

According to DOE Office of Management officials we interviewed, the review team assessed the department’s financial assistance against five criteria:

Whether the financial assistance was statutorily mandated;

Whether the financial assistance was described in congressional

Whether the financial assistance was consistent with administration priorities, as identified in budget documents and other statements from the President and Secretary of Energy, among other things;

What technology readiness level the financial assistance was intended to fund; and

Whether the technology encompassed by the project was already being funded by the private sector or others.

DOE Office of Management officials stated that the review team did not use the above criteria to assign quantitative scores to evaluate the department’s financial assistance; instead, the team collaboratively discussed each opportunity for assistance. In most cases, the review team was able to reach consensus on whether the financial assistance aligned with the administration’s priorities. DOE Office of Management officials also noted that they met with ARPA-E leadership to obtain additional information about ARPA-E financial assistance on three occasions during the course of the review. ARPA-E officials said that, in addition to those three meetings, they provided written information to address questions received from the review team and to provide additional context regarding ARPA-E financial assistance.

In total, DOE’s review team assessed 6 ARPA-E fiscal year 2017 or prior- year funding opportunity announcements for which applicants had been selected for award negotiation, 7 fiscal year 2017 announcements in the earlier stages of the merit review and selection process, 2 fiscal year 2017 announcements that had not yet been released, and 17 opportunities for financial assistance where ARPA-E funded renewals or new work under a determination of noncompetitive financial assistance. According to DOE Office of Management officials, the review team worked as quickly as possible to review all of DOE’s financial assistance to minimize potential disruptions for recipients and DOE’s funding organizations. Once the review team approved an opportunity for financial assistance, DOE funding organizations were allowed to resume work, DOE Office of Management officials told us. Figure 2 shows the total cumulative funds for ARPA-E financial assistance approved by the review committee at various stages in the review. For example, as shown in Figure 2, the review team approved roughly $158.3 million (55.6 percent) of ARPA-E’s proposed financial assistance on May 18, 2017, 3 days after the deadline for DOE funding organizations to submit information to the review team. The remaining proposed financial assistance was approved in several stages from June through August 2017. As of August 25, 2017, all of ARPA-E’s competitively selected funding opportunity announcements, renewals, and determinations for noncompetitive financial assistance, where selectees had been selected for negotiation, were approved by the review team, representing roughly $265 million, or 93.1 percent, of all ARPA-E funding reviewed by the team.

DOE Office of Management officials also stated that the financial assistance review team made a decision early in the course of the review to honor all existing DOE commitments to fund new work. These officials said that this extended to commitments made to entities that had been selected for award negotiations, even though the department does not officially commit to providing funds until such negotiations are completed and the award is finalized. However, according to ARPA-E selectees we interviewed, this message was generally not communicated to them, which led to uncertainty about whether their projects would be funded. In contrast, the review team recommended that the DOE Chief of Staff cancel ARPA-E’s Facsimile Appearance to Create Energy Savings funding announcement, which had accepted full applications but had not selected any applicants for award negotiation. This opportunity would have funded the development of advanced information technology that could allow for three-dimensional digital representation of a person in a room nearly indistinguishable from the person being there in real life, which might allow for increased telecommuting. DOE Office of Management officials told us that the review team reached this recommendation in part because this technology was already being funded by the private sector. As of November 2017, DOE Office of Management officials said the review team had cancelled 3 other DOE funding announcements as a result of the review.

According to information we collected, DOE’s review of ARPA-E financial assistance, as part of the DOE-wide review process, did not require the President to send a special message under the Impoundment Control Act. Specifically, the delay in obligating ARPA-E funds for financial assistance examined through DOE’s review process was for programmatic reasons. DOE officials explained that the purpose of the review was to ensure that the agency’s financial assistance aligned with the priorities of the current administration.

DOE’s Financial Assistance Review Created Uncertainty for ARPA-E Selectees, Which Led to Delayed Project Timelines and Staffing Difficulties, among Other Impacts

According to the 10 ARPA-E selectees we interviewed, DOE’s financial assistance review process created uncertainty, which led to a variety of impacts—the most frequently cited of which were potentially delayed project timelines and difficulties staffing project teams. Selectees told us that they received little communication from ARPA-E during the review process, and they indicated that additional information about review timelines and potential effects on their awards would have helped them manage some of the uncertainty they experienced during the review process. DOE Office of Management officials said the fiscal year 2017 review process helped to better identify and coordinate future financial assistance department-wide on crosscutting issues. DOE is conducting the fiscal year 2018 review process prior to publicly issuing funding announcements. As a result, DOE Office of Management officials said, the delays and uncertainty that selectees experienced in fiscal year 2017 should be reduced.

In our structured interviews with ARPA-E selectees, the most frequently cited impact of the uncertainty caused by DOE’s financial assistance review was the potential need to delay project timelines. All of the ARPA- E selectees we interviewed told us that they might need to extend their project timelines because of uncertainty caused by DOE’s review. Four of these selectees noted that the delay caused by DOE’s review could cause additional, cascading delays in their timelines. For example, 1 selectee we interviewed said that it would need to re-issue a hiring announcement it had publicized prior to the review because the review prevented it from hiring someone. In addition, the selectee would need to resubmit the hiring announcement to the university and state human resources departments for approval, which could take months to process. Another selectee said that it missed 2 months of a 3-month planting season because it could not start project work, and had the delay lasted any longer, the selectee would have missed an entire year of data collection on the project.

Selectees also cited challenges to staffing project teams as a result of the uncertainty caused by the review. Selectees stated that delays caused by the review affected their ability to hire team members they had planned to hire based on their original schedule, as potential members moved on to other projects or took different jobs. For example, 9 selectees told us that they delayed hiring new project team members while DOE’s review was occurring. Four selectees we interviewed said that they had difficulty retaining staff during the review process. For example, 1 selectee had to lay off 2 of the company’s 15 staff members because of the delay in receiving funding, and several other staff members left voluntarily. Furthermore, the selectee said laying off these staff members resulted in an increase in the company’s unemployment taxes, which was expensive for a small-sized company. Four other selectees that we interviewed said they had to assign existing project team members to other funded work or general activities because they could not begin work on their ARPA-E project until they received funding.

Selectees we interviewed cited additional impacts associated with the uncertainty caused by DOE’s financial assistance review. These impacts included:

Delaying equipment purchases. Four selectees reported that they had to delay purchasing important equipment needed to execute their project. One selectee noted that the delay caused by the review was long enough that price quotes it had received from equipment sellers expired, and that prices could increase in later quotes.

Changes to project scope. Two selectees told us that they might need to limit the planned scope of their projects to be able to complete them in the proposed timeline. For example, 1 selectee said its project involves helping to scale up three to four different technologies a year, which it might not be able to do if it has to adhere to its initial timeframes.

Loss of advantage against potential competitors. Four of the selectees we interviewed said that the delay may have caused their technology to fall behind their potential competitors in some way. For example, 1 selectee noted that it was working in a competitive environment for its technology, with ongoing efforts in multiple countries, and reported that its project might have fallen behind others’ efforts as a result of delays associated with DOE’s review. However, 4 other selectees said that the review was not likely to cause any loss of competitiveness.

Impacts on external project partners. Three selectees noted that DOE’s review caused uncertainty for partners on their projects, including partners that provide external funding. For example, 1 selectee told us that private investors in its technology area are most active in the fall and that its project team might not be able to seek a second round of funding if it could not demonstrate the necessary technical results of the project by then.

Impacts on pre-award spending reimbursements. One selectee reported that it had to cease certain pre-award spending. The selectee said that it spent roughly $10,000 on equipment and 150 hours of labor prior to DOE’s financial assistance review, but it could not submit invoices for these expenditures to ARPA-E while the review was ongoing and would not be able to if its project was ultimately not approved. Furthermore, the selectee said that even if the award was approved, the delay might result in expenditures falling outside the 90-day window of allowable pre-award expenditures, which would require obtaining approval from ARPA-E to be reimbursed.

Selectees we interviewed also stated that they received little communication from ARPA-E during the review, which contributed to the uncertainty about the status of their projects. Specifically, 6 of the selectees said that they would have liked additional information from ARPA-E on a variety of topics related to the review. For example, 4 selectees said they would have liked additional information about the review timeline and when it was planned to be completed. One of these selectees told us that a written document from ARPA-E indicating a rough time frame and next steps would have helped facilitate better planning for their project team. Two selectees we interviewed wanted additional information about whether they could renegotiate their timelines once the review was completed. Three selectees told us that they would have liked additional information about whether the review would cause them to lose their funding. ARPA-E officials we interviewed told us that they made three separate requests to DOE’s Deputy Chief of Staff to learn what they could communicate to selectees about the April 6, 2017, verbal order and the review process. ARPA-E officials told us that they were directed by the Deputy Chief of Staff not to communicate with selectees about the verbal order until receiving guidance from his office. ARPA-E developed proposed language to share with selectees but did not receive approval from the Deputy Chief of Staff to distribute it. DOE Office of Management officials we interviewed told us that they did not issue guidance to ARPA- E or other DOE funding organizations about how the organizations should communicate with selectees during the review.

In contrast to its fiscal year 2017 review, DOE began its 2018 financial assistance review in August 2017, prior to publicly issuing funding announcements. On August 10, 2017, DOE’s Office of Management sent an email to DOE funding organizations directing them to submit descriptions of their proposed financial assistance by September 8, 2017. Because the review will occur prior to publicly issuing funding opportunity announcements, and thus before any recipients apply or are selected, DOE Office of Management officials said the delays and uncertainty that selectees experienced in fiscal year 2017 should be reduced.

DOE Office of Management officials told us that—aside from changing the timing of its 2018 financial assistance review—the review team’s membership and evaluation criteria will be largely the same as for the fiscal year 2017 review. The officials said that they discussed the review process with senior leaders in DOE’s funding organizations to help ensure that they understood the priorities, expectations, and steps of the review process. The officials also told us that the review team developed additional guidance to clarify certain issues that arose during the fiscal year 2017 review. This additional guidance included:

On August 17, 2017, funding organization managers were informed that continuation awards—those where the activity is presently being funded—would be exempt from submission to the review team and can continue to move forward.

On August 29, 2017, funding organizations were informed that they should identify financial assistance that falls under one of seven crosscutting research issue areas. According to DOE Office of Management officials we interviewed, a key benefit of the fiscal year 2017 review process was that the review team noticed that DOE had several funding announcements at multiple funding organizations related to these areas. DOE’s funding organizations may be able to coordinate to issue a consolidated funding announcement in these crosscutting research issue areas, to ensure efforts are complementary and not duplicative. Furthermore, DOE Office of Management officials we interviewed said that knowing which funding organizations are funding work in these areas will support DOE meetings on crosscutting issues.

Agency Comments

We provided a draft of this report for review and comment to the Secretary of Energy. DOE provided technical comments, which we incorporated as appropriate.

We are sending copies of this report to the appropriate congressional committees, the Secretary of Energy, and other interested parties. In addition, the report is available at no charge on the GAO website at http://www.gao.gov.

If you or your staff members have any questions about this report, please contact me at (202) 512-3841 or neumannj@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made key contributions to this report are listed in appendix I.

Appendix I: GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

In addition to the individual named above, Chris Murray (Assistant Director), Perry Lusk (Analyst-in-Charge), Antoinette C. Capaccio, John Delicath, Justin Fisher, Kimberly McGatlin, Dan Royer, Tind Shepper Ryen, Lauren G. Sherman, and McKenna Storey made key contributions to the report.