U.S.-China Cooperation: Bilateral Clean Energy Programs Show Some Results but Should Enhance Their Performance Monitoring
Published: 2016-07-05T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: The United States and China lead the world in energy consumption, and both are investing in renewable resources and efforts to increase the efficiency of traditional fossil fuel sources in part to address climate change. In 2014, a congressional commission raised questions about bilateral cooperation between the United States and China on clean energy, including potential IP risks to U.S. participants involved in collaborative research projects.
GAO was asked to review government-led U.S.-China collaborative initiatives on clean energy. This report examines (1) how much funding U.S. agencies obligated for clean energy cooperation with China, (2) what is known about the results of key programs and the extent to which they follow leading practices in performance monitoring, and (3) the extent to which DOE managed risks that CERC participants may face. GAO analyzed funding data, reviewed documents and compiled reported results, interviewed agency officials and participants of key programs, and conducted site visits.
section_title: What GAO Found
paragraphs: In fiscal years 2008–2015, U.S. agencies obligated a total of about $97 million for clean energy cooperation with China. Two-thirds of this money was obligated for three key programs (projects of which are depicted from left to right below):
Department of Energy (DOE) program, the U.S.-China Clean Energy Research Center (CERC), that has focused on research and development in clean coal, clean vehicles, and energy efficiency in buildings;
U.S. Trade and Development Agency (USTDA) program focused on export promotion through projects such as feasibility studies and trade missions; and
Department of State (State) program that includes information sharing and technology demonstration projects across various clean energy technologies.
The key programs have yielded some results and have performance monitoring tools but generally lack targets for their performance, making the significance of their progress unclear. Examples of the programs' results include: for CERC, as of the end of 2015, the launch of 15 products, such as software for enhancing energy efficiency of buildings; and for the USTDA program, through fiscal year 2015, about $230 million in U.S. exports from its clean energy projects. Based on performance monitoring principles in the GPRA Modernization Act of 2010, it is a leading practice for federal programs to link goals to performance measures with established targets. Without targets, it is unclear how results compare with intended performance and what improvements may be needed; this is particularly important as DOE and State officials are planning the next phases of their programs and USTDA emphasizes the role of data in program decisions.
DOE identified intellectual property (IP) risks CERC participants may face, such as participants not having a clear plan for protecting IP, and took steps to manage them. These steps included requiring agreements clarifying IP rights and providing training, in part to encourage participants to share IP created outside of CERC projects. DOE officials said this IP sharing is important for valuable research and development collaboration. CERC participants GAO spoke with reported no significant issues with DOE's management of IP risks but, nonetheless, have been reluctant to share IP. DOE officials acknowledged that participants face a tradeoff between the risks and benefits of sharing IP with Chinese participants and that it is appropriate for companies to assess risks for themselves.
section_title: What GAO Recommends
paragraphs: GAO is making four recommendations to enhance performance monitoring, including that DOE, USTDA, and State each develop targets for program-level performance and track progress against them for the key programs GAO reviewed. The agencies agreed with GAO's recommendations and plan to take actions to address them.

section_title: Letter
paragraphs: The United States and China, the world’s largest economies, lead the  world in energy consumption and depend largely on traditional fossil fuel  sources. Together, China and the United States account for roughly 40  percent of global carbon dioxide emissions annually, according to the  U.S. Energy Information Administration. Concerns over rising emissions  of greenhouse gases, such as carbon dioxide, and their effects on the  climate have led most countries to adopt or consider adopting policies to  reduce these emissions. In part for these reasons, the United States and  China are seen as instrumental in multilateral efforts to address climate  change. Individually, in an attempt to reduce greenhouse gas emissions,  both the United States and China are investing in renewable resources  and also in efforts to increase the efficiency of traditional fossil fuel  sources. In 2015, China led the world in renewable energy investments  with an estimated $102.9 billion in new investments, while the United  States invested approximately $44.1 billion. In addition, the United  States and China have increased bilateral cooperation on clean energy in  an effort to spur progress.
In 2014, the congressional U.S.-China Economic and Security Review  Commission raised concerns about U.S.-China cooperation on clean  energy, including a lack of clear public reporting on results and potential  risks to participants in joint research and development projects due to  concerns related to China's protection of intellectual property (IP) rights.  Because of these issues, the commission recommended that we conduct  an assessment of government-led U.S.-China collaborative initiatives on  clean energy; subsequently, you requested that we do so. In this report,  we examine (1) how much funding U.S. agencies have obligated to clean  energy cooperation with China, (2) what is known about the results of key  U.S.-China cooperation programs and the extent to which these programs  follow leading practices in performance monitoring, and (3) the extent to  which the U.S. Department of Energy (DOE) manages risks that may face  U.S. participants in the U.S.-China Clean Energy Research Center  (CERC).
To address these objectives, we analyzed funding data and documents,  interviewed agency officials, and conducted site visits to locations in  California, Michigan, and West Virginia and phone conferences to  interview program participants, including from the three organizations  leading collaborative research under CERC. The U.S. government has no  uniform definition of clean energy that is applied government-wide.  However, based on consultation with participating agencies and review of  the White House’s June 2013 Climate Action Plan, we determined that  the following types of energy technologies are relevant for this review:  renewable energy (including  solar, wind, hydro, geothermal, and  biofuels); energy efficiency technologies (i.e., technologies that decrease  the intensity of energy usage); nuclear power; natural gas; clean coal  (e.g., coal with carbon capture, utilization, and storage); clean vehicle  technologies; and improved energy infrastructure (e.g., smart grids).
To identify the funding obligated to clean energy cooperation with China  from fiscal years 2008 through 2015, we sent a questionnaire to agencies  that we identified as being involved in these efforts. Among other items,  the questionnaire asked them to identify their U.S.-China clean energy  cooperative activities and the amount of funding obligated to each activity  by fiscal year. We took steps to ensure that agency responses were  complete, such as by comparing the responses against other agency  documentation, and also sent questions to the agencies to determine the  reliability of the sources of the data. We determined that the data  provided were reliable for our purposes.
To describe what is known about the results of U.S.-China clean energy  cooperation, we focused on three key programs to which the largest  amount of funding was obligated by the three agencies that provided the  most funding to U.S.-China clean energy cooperation. These key  programs were identified as DOE’s CERC, the U.S. Trade and  Development Agency’s (USTDA) East Asia Program, and the Department  of State’s (State) U.S.-China Climate Change Working Group (CCWG).  To describe these key programs’ results, we analyzed agency documents  and clarified and assessed the reliability of results data in these  documents through interviews with agency officials, among other steps.  We determined that the data provided by the three programs are  sufficiently reliable for describing their results. To determine the extent to  which these programs follow leading practices in performance monitoring,  we examined agency and program performance documents and  interviewed agency officials.
To determine the extent to which DOE has managed any risks that CERC  participants may face, we first determined what risks those might include  through interviewing DOE officials and a nongeneralizable sample of  CERC participants. We also analyzed relevant documents, such as the  Technology Management Plan for each CERC track, and compared steps  such as these that DOE has taken to manage IP risks with federal internal  control standards for risk assessment. Appendix I provides a more  detailed description of our objectives, scope, and methodology.
We conducted this performance audit from June 2015 to July 2016 in  accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.
subsections: section_title: Background
subsections: section_title: Overview of U.S. Cooperation with China and Other Countries on Clean Energy and the Environment
paragraphs: The United States and China have cooperated for over 35 years on  science and technology initiatives. In 1979, the two countries signed a  bilateral science and technology agreement that has served as an  umbrella agreement for subsequent bilateral environment and energy  initiatives.
In 2008, the countries established the Ten Year Framework for  Cooperation on Energy and Environment. This framework was intended  to facilitate the exchange of information and best practices to develop  solutions to the environment and energy challenges both countries face.  The framework includes some action plans related to clean energy, such  as plans for clean, efficient, and secure electricity; clean and efficient  transportation; and energy efficiency. According to staff from think tanks  and business associations and other individuals knowledgeable about  U.S.-China clean energy cooperation that we interviewed, U.S.  cooperation with China on clean energy could yield benefits such as  building trust between the countries, helping both countries advance their  efforts to meet environmental challenges, and creating opportunities for  U.S. businesses in China. According to these individuals, the sharing of  any IP through this cooperation is a potential risk due to possible IP theft.
In November 2014, the two countries’ presidents issued a U.S.-China  Joint Announcement on Climate Change, which included targets to  reduce greenhouse gas emissions in the United States, and for China to  intend to reach peak carbon dioxide emissions around 2030 and increase  the share of non-fossil fuels in its energy consumption. The  announcement also emphasized the countries’ commitment to a  successful climate agreement at the United Nations Climate Change  Conference in Paris in 2015, and the countries’ presidents reaffirmed this  commitment in a U.S.-China Joint Presidential Statement on Climate  Change in September 2015. In December 2015, more than 190 member  states under the United Nations Framework Convention on Climate  Change came together to adopt the Paris Agreement, which aims to hold  the increase in the global average temperature to well below 2 degrees  Celsius above pre-industrial levels through countries setting their own  nonbinding targets for emissions reductions. In Paris, some of the  countries that adopted the Paris Agreement also committed to initiatives  to substantially increase public and private investment in climate change  mitigation and adaptation activities. For example, through the Mission  Innovation initiative, 20 countries, including the United States and China,  will seek to double their governmental clean energy research and  development investment over 5 years to accelerate clean energy  innovation and make it widely affordable.
In addition to its bilateral cooperation with China, the United States  cooperates bilaterally and multilaterally with other countries on initiatives  related to climate change and clean energy. For example, the United  States has a Clean Energy Dialogue with Canada to encourage the  development of clean energy technologies to reduce greenhouse gases  and combat climate change. Also, in 2009, the United States launched  the Partnership to Advance Clean Energy with India, which is working to  accelerate inclusive, low carbon growth by supporting research and  deployment of clean energy technologies. Both the United States and  China, along with 21 other countries and the European Commission,  participate in the Clean Energy Ministerial, a high-level global forum to  promote policies and programs that advance clean energy technology, to  share lessons learned and best practices, and to encourage the transition  to a global clean energy economy. The Clean Energy Ministerial is  focused on improving energy efficiency worldwide, enhancing clean  energy supply, and expanding clean energy access.
section_title: Concern over China’s Protection of IP Rights
paragraphs: As U.S. cooperation with China on science and technology has expanded  over time, China’s protection of IP rights has been a persistent concern.  Although some IP issues have been addressed through dialogues, such  as the U.S.-China Joint Commission on Commerce and Trade,  according to a 2016 report by the U.S. Trade Representative, the  uncertain IP environment is a leading concern for businesses operating in  China. According to the report, the theft of trade secrets remains a  particular concern, and conditions are unlikely to improve as long as  those committing such theft continue to operate with relative impunity.  The report also identified concerns about reports that Chinese  government policies may have negative impacts on U.S. investors and  their IP rights, including that Chinese regulations, rules, and other  measures appear to require foreign companies to transfer or license their  IP rights to domestic Chinese entities in order to do business in China.
U.S. clean energy companies may face particular IP concerns with regard  to doing business in China. China’s 5-year plan for economic and social  development initiatives for 2016–2020 includes developing its  environmental technology industry as a focus area. The U.S. Trade  Representative has expressed concern that China’s innovation-related  and other industrial policies may have negative impacts on U.S. exports  or IP in particular industries by encouraging actions that pressure foreign  IP rights holders to transfer those rights to domestic Chinese entities.
section_title: U.S. Agencies Obligated About $97 Million in Total for Clean Energy Cooperation with China for Fiscal Years 2008 through 2015
paragraphs: U.S. agencies obligated about $97 million for clean energy cooperation  with China over the 8-year period of fiscal years 2008 through 2015. More  than 90 percent of this money was obligated by three agencies: DOE,  USTDA, and State. Two-thirds of the overall funding went to three key  programs, which are the largest U.S.-China clean energy cooperative  programs at each of these agencies. Almost half of the funding went to  research and development, and the overall funding went to a variety of  types of clean energy, with the majority for energy efficiency, clean coal,  and clean vehicles.
subsections: section_title: DOE, USTDA, and State Obligated Nearly All the Funding, Mostly for Three Programs
paragraphs: In total over the period encompassing fiscal years 2008 through 2015,  U.S. agencies obligated about $96.9 million for U.S. agencies, other  public entities, and private sector participants to cooperate with Chinese  entities related to clean energy. DOE obligated the majority of this  funding (71 percent). USTDA and State obligated another 13 percent and  11 percent, respectively. Two-thirds of the overall funding went to the  largest related programs at each of these agencies (see fig. 1), which are  the three key programs we focused on:
DOE’s CERC program: Through CERC, DOE obligated $47.5 million  for teams of U.S. scientists and engineers to perform research and  development with China on clean energy technologies. This  collaboration is being pursued for reasons beyond attempting to  address climate change, including to improve air quality, to lower  energy costs, and to promote energy security. The work through fiscal  year 2015 was separated into three tracks focused on clean coal,  clean vehicles, and energy efficiency in buildings. DOE funds U.S.  researchers in each of the tracks, while the Chinese government  funds the Chinese researchers, with the intention that U.S. and  Chinese researchers will be working together and learning from each  other on all projects.
USTDA’s East Asia Program: Through its East Asia Program,  USTDA obligated $12.5 million for U.S. companies to engage in  various types of clean energy projects with China, such as feasibility  studies, trade missions, and technical assistance. These projects  have focused on a wide range of clean energy technologies related to  smart grids, clean coal, and shale gas, among others.
State’s CCWG program: Through CCWG, State obligated $5.8  million for U.S. participation in cooperation and dialogue with China on  clean energy. Through fiscal year 2015, CCWG’s clean energy  cooperation has occurred through groups of projects bundled into six  initiatives: (1) heavy-duty and other vehicles; (2) smart grids; (3)  carbon capture, utilization, and storage; (4) energy efficiency in  buildings and industry; (5) climate-smart and low-carbon cities; and  (6) industrial boilers efficiency and fuel switching.
In addition to the U.S. federal funding obligated to these key programs,  U.S. private sector participants also cover a share of the costs of some  projects. For CERC and USTDA’s East Asia Program, such cost-share  increases the overall U.S. funding spent on these projects by  approximately double. Agency officials have pointed out that cost-share  from private companies shows the companies’ confidence in the  programs’ ability to achieve results.
Seven other agencies also engaged in clean energy cooperation with  China during this period.
The U.S. Agency for International Development obligated $5.5 million  for two technical assistance programs in China, one focused on  energy efficiency in buildings and another focused on various forms of  clean energy development, such as financing clean energy projects.
The Departments of Commerce and Transportation and the Federal  Energy Regulatory Commission each obligated between $1,800 and  $32,000 for clean energy cooperation with China during this period,  mostly for travel expenses to attend events or consultations in China  for regulatory cooperation.
The Departments of Agriculture and the Interior and the  Environmental Protection Agency participated in clean energy  cooperation with China using funding provided by DOE or State.
In addition, some agencies provided funding for their own travel expenses  to attend related events or to organize some related activities but were  unable to identify the amounts of such funding specifically related to clean  energy cooperation.
Annual U.S. obligations for U.S.-China clean energy cooperation varied in  fiscal years 2008 through 2015. As figure 2 shows, large increases in  annual obligations occurred in fiscal years 2010 and 2014, which were  within the years following the launches of the CERC and CCWG  programs, respectively.
section_title: Almost Half the Funding Supported Research and Development
paragraphs: As seen in figure 3, U.S. government funding provided to clean energy  cooperation with China supported numerous types of activities.
Research and development: Almost half the funding was obligated  by DOE for research and development to promote clean energy  innovations, with most of that funding for CERC. According to DOE  officials and CERC participants, through research and development  under CERC in particular, U.S. participants gain important benefits,  such as the ability to speed progress in their research through  collaboration with other U.S. researchers and leading Chinese  scientists and engineers and access to unique experimental platforms  unavailable in the United States. In addition, U.S. companies obtain  the opportunity to demonstrate the viability of their products in China’s  large market.
Information exchange: Another 26 percent of the funding supported  different types of information exchange, including forums for technical  discussion and regulatory cooperation. For example, there are annual  meetings between the United States and China organized to discuss  energy efficiency, renewable energy, and clean coal, and there have  been other forums held to discuss topics such as biofuels, smart  grids, and smart cities. According to agency officials, these forums  have multiple benefits for U.S. participants, including opportunities to  highlight U.S. businesses, to work toward harmonizing codes and  standards between China and the United States, and to share  regulatory best practices.
Export promotion: Activities to promote U.S. exports received about  13 percent of the funding, all of which was from USTDA and included  feasibility studies, trade missions, and some technical assistance.  Feasibility studies help U.S. companies demonstrate the viability of  their technologies to prospective Chinese buyers. Through trade  missions, USTDA brings Chinese officials to the United States to  observe the design, manufacture, and operation of U.S. clean energy  technologies. Also for export promotion, USTDA provided technical  assistance to Chinese officials through technical exchange, training,  and standards development programs. USTDA funds all such projects  with the intention to create U.S. exports while supporting China’s  efforts to reduce carbon emissions through the deployment of clean  energy technologies.
Other types of activities: The remaining 12 percent of the funding  went to other types of technical assistance and activities such as  demonstration projects in China using advanced renewable energy  technologies, surveys in northwestern China to identify sites for  demonstrations of carbon capture and storage, a study of the shale  gas potential in one Chinese province, strategy development, and  training efforts to promote IP protection.
section_title: Funding Supported Cooperation on a Wide Range of Clean Energy Technologies
paragraphs: U.S. government funding supported cooperation on a wide range of types  of clean energy technologies. As seen in figure 4, the largest portions of  funding went to energy efficiency, clean coal, and clean vehicle  technologies, which related to the three areas that the CERC program  focused on through fiscal year 2015.
section_title: Key Programs Have Yielded Some Results and Monitor Performance, but They Lack Targets, Making the Extent of Progress toward Their Goals Unclear
paragraphs: All three key programs have yielded some results. For example, CERC  projects had led to the launch of 15 products by the end of 2015,  including software for enhancing energy efficiency in buildings. In  addition, by the end of fiscal year 2015, the 24 USTDA projects from its  East Asia Program included in our review had generated about $230  million in U.S. exports, and the six CCWG initiatives we reviewed had  trained 48 people on global climate change. In addition, the three  programs have tools to monitor performance such as performance reports  and program reviews. Generally, however, the three programs lack  targets for their performance measures and USTDA does not have  agency-wide targets. Agency officials provided various explanations for  why it was difficult for them to set targets, including that CERC was a new  program when it started work in 2011 and that USTDA is a demand- based agency. However, establishing targets for these programs, and for  USTDA agency-wide, could help managers generate and communicate  more meaningful performance information that they could also learn from  to identify performance shortfalls and pinpoint options for improvement.
subsections: section_title: The Three Key Programs Have Yielded Some Results
paragraphs: Our analysis of the measures and documents used by the three programs  to track performance at the program and lower levels shows that all of the  programs have yielded some results, such as the number of products  launched as a result of CERC, the dollar value of exports generated by  USTDA’s projects, and the number of people trained on global climate  change by CCWG initiatives.
DOE’s CERC Program. CERC was announced in 2009 and work began  on projects in 2011. As seen in table 1, at the program level, CERC has  yielded results for select key performance measures through the end of  2015. According to CERC officials, CERC’s key performance measures  are the program’s most important and relevant measures.
Beyond the results tracked for these program-level performance  measures, each track has also achieved significant technical outcomes,  according to DOE. For example, the clean coal track used data from a  Chinese power plant’s carbon dioxide capture process to model that  system in a U.S. power plant and found that it could cost significantly less  to capture carbon dioxide than initially estimated. Also, the energy  efficiency in buildings track developed and commercialized a moisture  and air sealant that reduces energy consumption and is environmentally  friendly; and the clean vehicles track developed techniques to model  hybrid powertrains for vehicles that are now being applied to design a  hybrid light truck.
USTDA’s East Asia Program. According to our analysis of USTDA  project-level results, the 24 USTDA projects included in our review from  the East Asia Program had generated about $230 million in U.S. exports  through the end of fiscal year 2015. The exports generated as a result  of these projects range from $160,000 from a feasibility study funded in  fiscal year 2011 to almost $135 million from a feasibility study funded in  fiscal year 2009. According to USTDA, these exports have supported  about 1,500 U.S. jobs based on a Department of Commerce methodology  for estimating U.S. jobs attributable to U.S. exports.
Chinese officials to the United States to  learn about U.S. shale gas and energy  efficiency technologies (see bottom photo  below), green buildings and city planning,  and vehicle fuel economy standards.  inform Chinese officials, such as through  workshops on U.S. shale gas practices;  developing a model for smart grids in  China; and assisting the development of  Chinese smart grid standards that would  be harmonized with U.S. standards.
Thus far, the completed USTDA clean energy projects included in our  review have resulted in a U.S. export multiplier of about 36—for every  dollar obligated by USTDA the agency identified about $36 in U.S.  exports generated. This compares with USTDA’s overall multiplier, for  fiscal year 2015, of $74 in U.S. exports for every dollar in agency  funding. USTDA projects have also yielded results for project-level  performance measures showing the projects' development impact on  recipient countries. Examples of development impact results for the East  Asia Program’s clean energy projects include projects that, individually,  led to an estimated 200 permanent jobs in China, about 20 people in  China receiving training and skill development, and 50 megawatts of new  energy capacity, according to USTDA.
State’s CCWG Program. CCWG was announced in 2013, and work for  the initiatives covered in our review began in either fiscal year 2014 or  2015. Based on our aggregation of the targets and results of the six  CCWG initiatives we reviewed, through fiscal year 2015, CCWG has  yielded some progress related to seven of eight performance measures  CCWG uses to monitor performance for these six initiatives, as shown in  table 2. Results data are reported to State by DOE, the Department of  Transportation, and the Environmental Protection Agency, which  implement CCWG activities. Generally, any initiative-level targets set for  these performance measures were designed to be met in late fiscal year  2016 or  fiscal year 2017, and State officials said that they expect to see  more results near the end of the initiatives, because that is when more  activities are planned. In addition, one initiative had a change in its scope  of work that has delayed its activities. CCWG initiatives have also  achieved additional outcomes not captured by their initiative-level  performance measures. For example, the energy efficiency in buildings  and industry initiative developed three partnerships between U.S. and  Chinese companies that could reduce their buildings’ energy use by 25 to  51 percent.
Climate-Smart/Low-Carbon Cities: This  industry: This initiative includes sharing  best practices on energy performance  contracting and energy efficiency  upgrades (see right photo below).   Industrial boilers efficiency and fuel  switching: Through this initiative, U.S.  and Chinese researchers conducted an  assessment of China’s coal-fired  industrial boilers and plan to implement  identified strategies to improve their  efficiency.
Performance measure  Amount of investment leveraged in U.S. dollars, from private and public sources, for  climate change  Number of laws, policies, strategies, plans, or regulations addressing climate  change (mitigation or adaptation) and/or biodiversity conservation officially proposed  or adopted  Number of people receiving training in global climate change  Number of person hours of training completed in climate change  Number of days of technical assistance in climate change provided to counterparts  or stakeholders  Projected greenhouse gas emissions reduced or avoided through 2030 from  adopted laws, policies, regulations, or technologies related to clean energy  (measured in metric tonnes carbon dioxide)
In addition, DOE and State officials said that their programs had achieved  results related to the bilateral relationship with China that could not be  quantified. For example, DOE and State officials said that the trust built  between the United States and China on climate issues through the joint  work and dialogue under CERC and CCWG, respectively, helped to  enable the November 2014 U.S. and Chinese Presidents’ Joint  Announcement on Climate Change. These officials said that this  announcement helped catalyze the December 2015 Paris Agreement on  climate change.
section_title: Although the Three Programs Monitor Performance, They Generally Lack Targets, Making It Unclear How Results Compare with Intended Performance
paragraphs: All three programs monitor progress toward their goals through a variety  of tools, such as performance reports and program reviews. Two of the  programs also have performance measures reflecting their goals and  collect data on some of those measures; however, none of the programs  have targets for all their performance measures, which would enable  them to compare the results that they have achieved with the results they  had planned to achieve. To help manage program performance, linking  goals to performance measures that are tracked against established  targets is a leading practice for federal programs. In addition, USTDA  did not have targets for most of its agency-wide performance measures.  The GPRA Modernization Act of 2010 (GPRAMA) requires agencies to  publish a performance plan that, among other things, contains  performance measures with established targets that can be used to  assess progress toward achieving those targets.
subsections: section_title: DOE’s CERC Program
paragraphs: CERC monitors program performance through a combination of routine  reports and specific data requests. DOE requires that each track submit  quarterly reports. Although most information in these reports is provided  at the project level, these reports also contain information on some of the  program’s performance measures, such as measures related to  intellectual property creation. However, DOE officials largely collected  information regarding CERC’s program performance through specific data  requests, such as to prepare for meetings or program-level reports. DOE  officials emphasized that they focus their performance monitoring at the  project level, where there have been more than 80 projects within the  three CERC tracks. Each project follows a 10-point plan describing,  among other elements, the research objective, work schedule with interim  milestones, and deliverables and dates. Officials said that these plans are  the basis for the information in the quarterly reports and are how CERC  holds projects accountable for their performance. At the project level,  performance monitoring also occurs through review meetings, such as  reviews by industrial partners and DOE management, peer review of  projects under one of the tracks by DOE's Office of Energy Efficiency and  Renewable Energy, and other technical reviews by that office as well as  DOE's Office of Fossil Energy for projects under the remaining two tracks.  See appendix III for more information on CERC’s organization and  reporting relationships.
DOE officials monitor CERC’s performance against four overarching  goals that they said have been the objectives of CERC since it was  established. Those goals are to accelerate development and deployment of clean energy technology; expand and strengthen bilateral engagement between the United protect intellectual property, encourage its development, and improve  U.S.-China interactions regarding intellectual property; and facilitate market access to participating businesses to speed  technology deployment.
Officials said that they use 19 key performance measures, each linked to  at least one of the four goals, to indicate progress toward those goals.  However, during the first phase of CERC that ended in fiscal year 2015,  none of these performance measures had targets. According to DOE  officials, setting targets for CERC was difficult because it was a new  program focused on a new model of collaborative research and  development and they did not have enough information to create targets  when it first started work in 2011. In addition, officials said it is difficult to  know what a research and development program will accomplish before it  begins. However, according to Office of Management and Budget  guidance, agencies managing any research and development program  should develop targets to measure progress toward its goals.
CERC is a high-visibility program for U.S-China cooperation on clean  energy, with the Secretary of Energy and his Chinese counterpart  involved in annual program reviews. In addition, CERC is planning to start  its second phase in 2016 and is in the process of developing new work  plans for each track for this phase, according to DOE officials. If CERC  does not have targets, as suggested by leading practices, managers may  not have the information needed to make timely improvements to ensure  that progress toward goals remains on track and to clearly communicate  to DOE leadership how CERC is performing against its intended results.
section_title: USTDA and Its East Asia Program
paragraphs: According to USTDA officials, the agency monitors performance of the  agency’s East Asia Program through annual meetings during which all  levels of USTDA staff review USTDA’s regional programs by sharing  lessons across the programs and discussing program results. USTDA  examines program efficacy by reviewing information on funded activities,  countries and regions, and industry sectors. Officials said that USTDA’s  Office of Program Monitoring and Evaluations provides the East Asia  Program and the agency with data that can be used to examine program  performance and identify areas for improvement.
USTDA assesses its projects while they are ongoing and soon after they  have been completed. These assessments focus on several areas,  including the implementation potential of the project; feedback from  project participants; and project impacts, such as U.S. exports and the  development impact on the recipient country. USTDA also uses an  independent evaluator to evaluate almost all its projects. These  evaluations occur on an annual basis to determine whether the projects  have resulted in additional exports or development impacts until USTDA  determines that no further results are likely to occur, which can take 5  years or longer. USTDA follows these same monitoring practices for all of  its programs throughout the agency. See appendix III for more  information on USTDA’s organization and reporting relationships.
USTDA has agency-wide goals used to evaluate its performance.  Officials further stated that these goals flow down from the agency to the  East Asia Program. The goals are to  create U.S. jobs by supporting exports of U.S. goods and services for  priority development projects in emerging economies, foster opportunities for U.S. small businesses through significant  involvement in USTDA’s programs, and utilize evidence and evaluation data to guide agency programming  decisions.
Each agency-wide goal has associated performance measures. These  same measures are also used to monitor the East Asia Program,  according to USTDA officials. USTDA set a target for one of the agency- wide performance measures—to exceed the Small Business  Administration’s benchmark of 23 percent of federal prime contracts  awarded to U.S. small businesses—although USTDA officials said that  they do not break down this target by program. None of the other  agency-wide or program-level performance measures had targets,  although the agency does set targets at the project level for some  performance measures reflecting certain goals such as potential exports.
USTDA officials said that there are several reasons why they do not have  targets for most of their performance measures at the agency or program  level. Because USTDA is a demand-based agency, with projects  generally proposed by industry, officials said that it is difficult to know  what kinds of projects will be proposed and ultimately approved and  funded. Furthermore, officials said that having a precise target for each  performance measure could produce a perverse incentive by encouraging  them to fund a project in order to meet a given target, even if they did not  think it was the project most worthy of being funded. USTDA officials are  also concerned that targets would reduce their flexibility in allocating  USTDA’s resources. For example, officials said that they have strategic  reasons for investing in certain countries, including responding to U.S.  government policy priorities, even if those projects will not necessarily  produce the most exports, and targets could limit their ability to fund those  projects. However, as GAO has previously reported, if an agency has  measurable, balanced performance measures that cover all an agency’s  priorities, this should prevent an overemphasis on one or two priorities at  the expense of others that may skew an agency’s performance. Without  published agency-wide targets, as required by GPRAMA, it is unclear if  agency managers have the information they need to determine if they are  making sufficient progress toward achieving their goals, to identify  performance shortfalls and options for improvement, and to provide  Congress and the public with information needed to enhance their  oversight and better ensure the agency’s accountability. Furthermore,  without targets at the program level, as suggested by leading practices,  managers risk not being able to use all the information generated from  long-term project evaluations to inform timely improvements, such as in  deciding which types of projects to fund in particular countries or regions.
section_title: State’s CCWG Program
paragraphs: State officials said that they monitor the performance of CCWG as a  program through two reports that focus on initiative-level activities: (1)  internal reports on the status of the CCWG initiatives that are presented  annually to the U.S. Special Envoy for Climate Change and his Chinese  counterpart and (2) public reporting of CCWG’s annual performance by  initiative to the chairs of the U.S.-China Strategic and Economic Dialogue  (S&ED). The reporting to the S&ED is CCWG’s main monitoring  mechanism, according to State officials.
As shown previously in table 2, the six CCWG initiatives within our review  have performance measures that are tracked at the initiative level. These  initiatives are implemented by other federal agencies that are required to  report semiannually to either State’s Bureau of Oceans and International  Environmental and Scientific Affairs (OES), which oversees five of the  CCWG initiatives under our review, or State’s Bureau of Energy  Resources (ENR), which oversees one of the CCWG initiatives. These  reports include information on the results achieved for each relevant State  performance measure as well as a narrative describing the initiative’s key  activities over the reporting period. According to State officials, OES and  ENR use this information as inputs to standard Department of State  reporting of performance by bureau or for the whole agency; however,  CCWG does not use these performance measures to monitor  performance at the program level. See appendix III for more information  on CCWG’s organization and reporting relationships.
CCWG works toward an overarching goal negotiated with China, which is  to facilitate constructive U.S.-China cooperation and dialogue on climate  change, but does not have program-level performance measures or  targets, according to our review of CCWG documents and State officials.
State officials said that CCWG does not have program-level performance  measures or targets because the program is viewed as a cooperative  effort with China, and it would be difficult to negotiate these elements with  the Chinese government. State did not negotiate with the Chinese  government the performance measures that it uses at the initiative level,  nor does it share the resulting performance information with China,  because these are for State’s internal use. In addition, because of the  initiative-level and other reports, program-level performance information  had not seemed necessary to monitor program progress, according to a  State official. These reports show that CCWG is making progress from  year to year, according to the State official who leads CCWG. However,  without program-level performance measures with targets, as suggested  by leading practices, CCWG program managers may lack an apt and  adequate framework to determine the extent to which the results  measured at the initiative level are yielding expected program results and  whether any program improvements are needed. The State official who  leads CCWG agreed that program-level performance measures and  targets could be helpful for learning about CCWG’s performance,  particularly if the performance measures chosen reflected CCWG’s broad  goal of working constructively with China on climate change.
section_title: DOE Has Taken Steps to Manage IP Risks CERC Participants May Face, but Participants Are Reluctant to Share IP
paragraphs: DOE officials identified potential sharing of background IP—IP generated  outside the scope of a research and development collaboration—and  participants not having a clear plan for managing IP as risks to U.S.  companies and researchers participating in CERC. DOE has taken steps  to manage these risks, in part to enable participants to share background  IP, which is important for valuable research and development, according  to DOE officials. Although CERC participants reported no significant  issues with DOE’s approach to managing IP risks, companies  participating in CERC have been reluctant to share background IP as a  part of CERC. As a result, U.S. CERC participants only shared  background IP with Chinese organizations for 3 of the more than 80  projects that took place in the first 5-year phase of CERC. DOE officials  acknowledged that companies participating in CERC face a tradeoff  between the risk of sharing background IP and potential benefits, such as  valuable research and development outcomes and gaining a market  advantage through demonstrating projects in China.
subsections: section_title: DOE Has Identified IP Risks CERC Participants May Face and Has Taken Steps to Manage Them
paragraphs: When CERC was first launched in 2009, DOE officials identified potential  sharing of background IP and participants not having a clear plan for  managing IP as risks to U.S. companies and researchers participating in  CERC. DOE officials said that background IP needs to be protected in  order for participants to bring their most creative ideas forward to facilitate  joint research and development, which is important to the CERC goal of  accelerating development and deployment of clean energy technology.  According to DOE officials, strong protection of IP encourages innovation  by allowing researchers to build on discoveries through lawful means,  which accelerates further innovation and enables collaboration. DOE  officials and almost all of the CERC participants we interviewed, including  the lead organizations of the three tracks and several participants from  each track, did not identify any other risks for CERC participants.
DOE has taken steps to manage IP risk to CERC participants, which is in  accordance with federal internal control standards for risk assessment.
Specifically, DOE has taken steps to manage IP risks to CERC  participants through the following means.
The IP Annex to the CERC Protocol: This part of the CERC  founding agreement attempts to help manage IP risk by defining how  IP may be shared or licensed in each country. The U.S. Patent and  Trademark Office has identified a potential discrepancy between  Chinese law and the bilateral U.S.-China Science and Technology  Agreement upon which the IP Annex to the CERC Protocol is based,  according to U.S. Patent and Trademark Office officials. These  officials stated that the potential discrepancy is related to ownership of  any improvements made to IP licensed between U.S. and Chinese  entities. The U.S. Patent and Trademark Office is discussing the  matter with other agencies, including DOE.  According to DOE,  differences in the laws of the two countries with respect to intellectual  property protection were considered and addressed when drafting the  IP Annex to the CERC Protocol. In that regard, in order to specify IP  rights in greater detail, the IP Annex to the CERC Protocol requires  each CERC track to have a Technology Management Plan in place  before work on projects can begin.
Technology Management Plans: These plans, which are agreed to  by all the participants in a CERC track, are intended to facilitate joint  research and development and encourage information sharing by  specifying IP rights in greater detail than the IP Annex to the CERC  Protocol. According to DOE officials, the Technology Management  Plans encourage sharing of background IP to research and  development partners by setting up an IP framework in advance of  work beginning on projects and making it clear that both governments  have endorsed the Technology Management Plans. In addition, the  Technology Management Plans state that participants shall negotiate  in good faith to provide nonexclusive licenses for IP developed on  joint projects with participants in the other country, as well as with  third parties who are not participants. According to agency officials,  this has not been the case in previous science and technology  agreements between the United States and other countries.  According to DOE officials, this provision was important because U.S.  CERC participants were interested in being able to license IP to have  market access in both countries.
IP training workshops: CERC has conducted five IP training  workshops to help participants understand IP sharing under CERC  and relevant IP practices and laws in the United States and China.  According to DOE officials, these workshops are intended to promote  research through cooperation and to encourage participants to share  IP, and DOE intends to hold more workshops during the second  phase of CERC. DOE also hosted a webcast about IP challenges and  opportunities for U.S. organizations doing business in China that it  posted on the CERC website.
IP guide: CERC developed an IP guide to assist researchers working  on CERC projects. This guide provides a broad overview of IP issues  and information specific to CERC, such as information related to how  to handle the commercial development of inventions that result from  CERC research projects.
IP experts group: DOE encouraged the establishment of an IP  experts group to provide pro bono legal assistance to the CERC  program. As of November 2015, the IP experts group had 19 U.S.  members and 7 Chinese members. Members of the group reviewed  and commented on CERC’s IP guide and are available to answer IP  questions for participants on a limited basis.
DOE officials said these steps have not eliminated all IP risk but that DOE  is focused on preemptive IP protection and education for CERC  participants, so that the participants can best protect their own IP  interests.
section_title: CERC Participants Reported No Significant Issues with DOE’s Approach to IP, but Participants Are Reluctant to Share IP
paragraphs: CERC participants we interviewed did not report any significant issues  with steps DOE has taken to address IP risks. Representatives of 8 of the  12 participating organizations we spoke with about IP issues said the  Technology Management Plan was helpful, while others said it had no  effect on CERC projects or that they had not had an opportunity to test it.  Notably, one participant found the Technology Management Plan helpful  in resolving a joint venture negotiation issue. Specifically, the U.S. CERC  participant wanted to license technology related to a CERC project to a  Chinese company with a nonexclusive license so that it could also license  the technology to other companies in China, while the Chinese company  wanted an exclusive license to the technology. According to the  participant, the Technology Management Plan was helpful in resolving the  issue diplomatically and arriving at the desired agreement.  Representatives of 9 of the 12 participating organizations said that there  was nothing more that DOE could or should do to address IP risks. One  participating software company suggested that CERC could further  mitigate IP risks by providing software protection technology to  participants. Another participating organization suggested that DOE could  request that IP terms be summarized in project proposals, so there could  be easy access to understanding how each project is managing IP risks.
Although CERC participants reported no significant issues with DOE’s  approach to managing IP risks, U.S. companies participating in CERC  have been reluctant to share background IP as a part of CERC. U.S.  CERC participants shared background IP with Chinese organizations for  3 of the more than 80 projects that took place during the first 5-year  phase of CERC, according to a DOE survey of CERC tracks about IP  completed in December 2015. The seven companies we spoke with  regarding IP issues said that they have their own IP protection strategies  in place, and several said they generally considered it a risk to share IP  with any other companies that are potential competitors. Representatives  of three of the companies mentioned that their companies had additional  concerns about IP protection related to working in China for reasons such  as a perception that the Chinese legal system will not reliably protect their  IP rights.
For its second 5-year phase, at the direction of higher-level management  in DOE and DOE’s counterpart ministry in China, CERC will make an  effort to bring more results to market, according to CERC officials. To that  end, CERC is planning to focus more on demonstration projects and  other projects that are closer to commercialization. A member of the  CERC IP experts group said that IP risk is greater once technology is  closer to commercialization because companies have invested more in  the technology.
This greater focus on projects closer to commercialization will continue,  and may increase, the importance of sharing background IP during  CERC’s second 5-year phase. DOE officials said they would like to  encourage more sharing of background IP during CERC’s second 5-year  phase and that through demonstration projects there is more likely to be  sharing of background IP; however, according to two CERC participants  we spoke with, sharing background IP may not be necessary for some  demonstration projects. In addition, we found that participants’ willingness  to share IP for demonstration projects varies. Specifically, participants in  the clean coal track and one participant from the energy efficiency in  buildings track said they were interested in demonstration projects and  were potentially willing to share, or had shared, IP under CERC. Two  participants in the energy efficiency in buildings track said they may be  able to demonstrate their products without sharing IP, such as by using  technology designed to protect software. However, the representatives of  the two companies we spoke with from the clean vehicles track about IP  issues said that they were not interested in participating in demonstration  projects and that they would not share IP as part of any joint research  effort such as CERC.
DOE officials acknowledged that companies participating in CERC face a  tradeoff between the risks of sharing background IP and the potential  benefits, such as valuable research and development outcomes and  gaining a market advantage through demonstrating projects in China.  These officials also stated that it is appropriate for companies to assess  risks for themselves and not share their most valuable IP if the related  risk is determined to be too great. Willingness to share background IP is  important for valuable research and development collaboration, but  researchers would still be able to engage in work that could prove  worthwhile if companies or researchers are unwilling to share their  background IP under CERC, according to DOE officials. While not much  background IP was shared by U.S. CERC participants during CERC’s first  phase, U.S. and Chinese CERC researchers exchanged other types of  information as inputs to their projects in ways that helped to further their  research, according to CERC lead organizations. For example, some of  the U.S. and Chinese organizations participating in the clean vehicles  track agreed to share battery testing data. Because many batteries must  be discharged repeatedly to understand their full life cycle under differing  conditions, battery testing can take from months to years; this agreement  to share data eliminated months of testing time, according to  representatives from the clean vehicles CERC track.
section_title: Conclusions
paragraphs: Both the United States and China have committed to efforts to address  climate change, including doubling their research and development  investments on clean energy. The three U.S. government programs we  examined—DOE’s CERC, USTDA’s East Asia Program, and State’s  CCWG—are among the mechanisms for cooperating with China to make  progress in advancing clean energy technologies. CERC and CCWG  officials are in the process of planning the next phases of those programs,  and USTDA describes itself as an agency that values the role of data in  making program decisions. All three programs realized some results as of  the end of 2015 and monitor progress toward their goals by employing a  variety of tools, such as performance measures and reporting and  evaluation systems. However, we found that all three programs and  USTDA for its agency-wide performance measures generally lacked  targets, which would enable them to compare the results that they have  achieved with the results they had planned to achieve. Not having targets  linked to program performance measures limits opportunities to identify  potential program improvements and managers’ ability to generate and  communicate more meaningful performance information. Furthermore,  without published agency-wide targets, Congress and the public are  unable to compare USTDA’s planned and actual performance, which  would help them in providing oversight and ensuring the agency’s  accountability.
section_title: Recommendations for Executive Action
paragraphs: 1.  To improve CERC’s performance monitoring, the Secretary of Energy  should ensure that for CERC’s second phase the program creates  targets and tracks progress against those targets in order to measure  program performance.  2.  To improve the agency’s performance monitoring, the Director of the  U.S. Trade and Development Agency should develop and make  public annual targets for the agency’s performance measures.  3.  To improve the East Asia Program’s performance monitoring, the  Director of the U.S. Trade and Development Agency should ensure  that the East Asia Program sets targets for its performance measures  and tracks progress against those measures.  4.  To improve CCWG’s performance monitoring, the Secretary of State  should ensure that CCWG develops measures and targets at the  program level and tracks its performance against those measures and  targets.
section_title: Agency Comments and Our Evaluation
paragraphs: We provided a draft of this report for review and comment to DOE, State,  and USTDA; the Departments of Agriculture, Commerce, the Interior, and  Transportation; the Environmental Protection Agency; the Federal Energy  Regulatory Commission; and the U.S. Agency for International  Development. In their written comments reproduced in appendices IV, V,  and VI, DOE, State, and USTDA, respectively, agreed with our  recommendations and noted plans to take action to address them. In  addition, USTDA reiterated information about its performance monitoring  and evaluation processes that we included in our report, such as the  target it set for one of its performance measures on federal prime  contracts awarded to small businesses, the value of its U.S. export  multiplier, and a description of its monitoring and evaluation processes.  Furthermore, USTDA indicated that the agency has a target for the  amount of U.S. exports generated in fiscal year 2017. We did not include  this information in our report because the annual amount of U.S. exports  generated agency-wide is not one of USTDA’s performance measures  specified in its strategic plan.
Commerce, DOE, State, and USTDA also provided technical comments  that were incorporated, as appropriate. The other agencies provided no  comments.
We are sending copies of this report to the appropriate congressional  committees and to the Secretaries of Energy and State; the Director of  USTDA; the Secretaries of Agriculture, Commerce, the Interior, and  Transportation; the Administrator of the Environmental Protection Agency;  the Chairman of the Federal Energy Regulatory Commission; and the  Administrator of the U.S. Agency for International Development; and other  interested parties. In addition, the report will be available at no charge on  the GAO Web site at http://www.gao.gov.
If you or your staff have any questions about this report, please contact  Kimberly Gianopoulos at (202) 512-8612 or gianopoulosk@gao.gov, or  John Neumann at (202) 512-3841 or neumannj@gao.gov. Contact points  for our Offices of Congressional Relations and Public Affairs may be  found on the last page of this report. GAO staff who made major  contributions to this report are listed in appendix VII.
section_title: Appendix I: Objectives, Scope, and Methodology
paragraphs: You asked us to review U.S.-China cooperation on clean energy. This  report examines (1) how much funding U.S. agencies have obligated to  clean energy cooperation with China; (2) what is known about the results  of key U.S.-China cooperation programs and the extent to which these  programs follow leading practices in performance monitoring; and (3) the  extent to which the U.S. Department of Energy (DOE) manages risks that  may face U.S. participants in the U.S.-China Clean Energy Research  Center (CERC).
To determine which types of technologies would be related to clean  energy, we looked for a U.S. government definition of the term, but found  that the U.S. government has no uniform definition of clean energy that is  applied government-wide. Instead, based on consultation with  participating agencies and review of the White House’s June 2013  Climate Action Plan, we have determined that the following types of  energy technologies are relevant for this review: renewable energy  (including  solar, wind, hydro, geothermal, and biofuels); energy efficiency  technologies (i.e., technologies that decrease the intensity of energy  usage); nuclear power; natural gas; clean coal (e.g., coal with carbon  capture/sequestration); clean vehicle technologies; and improved energy  infrastructure (e.g., smart grids).
To describe the funding amounts that U.S. agencies have obligated to  clean energy cooperation with China, we first took steps to determine the  agencies involved in and providing funding to these efforts. To identify  these agencies, we analyzed publicly available information on agency  websites and outcome statements from two key annual meetings  between the United States and China: the U.S.-China Strategic and  Economic Dialogue and the Joint Commission on Commerce and Trade.  In addition, when meeting with agencies that we had identified as  involved in U.S.-China clean energy cooperation, we asked them which  other agencies they worked with in this cooperation. We identified 10  agencies involved: (1) the Department of Commerce, (2) DOE, (3) the  Department of the Interior, (4) the Department of State (State), (5) the  Department of Transportation, (6) the Environmental Protection Agency,  (7) the Federal Energy Regulatory Commission, (8) the U.S. Agency for  International Development, (9) the Department of Agriculture, and (10)  the U.S. Trade and Development Agency (USTDA).
To determine which of these agencies obligated funds to U.S.-China  clean energy cooperation and the amounts obligated in fiscal years 2008  through 2015, we sent a data collection instrument to all involved  agencies that asked them to identify, among other items, the agencies’  U.S.-China clean energy cooperative activities; a description of each  activity, including its purpose and the type(s) of clean energy it focused  on; identification of the type of activity (e.g. joint research and  development, trade mission, forum for technical discussion, feasibility  study, regulatory cooperation, technical assistance, other); other agencies  participating in the activity; the amount of funding obligated to each  activity by fiscal year; the appropriations funding account used for such  obligations; and the source of that information. Upon receipt of the  agencies’ responses, we took multiple steps to ensure that responses  were complete, including comparing all responses against each other to  determine if activities were reported by one agency but not others that  also participated in the activity, comparing responses against other  agency documentation describing U.S.-China clean energy cooperative  efforts, and following up with the agencies to ask for clarifications  regarding any activities that seemed that they may have been missing.  We also sent each agency a set of questions to help determine the  reliability of the sources of the data and to ensure that the agencies  considered the information provided as a complete and accurate  characterization of their agency’s participation in and funding of U.S.- China clean energy cooperation. Our analysis of these responses showed  there were some activities that had been reported but for which the  funding was not solely for the purpose of clean energy cooperation with  China. For example, funding may have gone to multiple countries or have  been also used for other purposes. In cases where agencies were unable  to separately identify the funding for clean energy cooperation with China,  we excluded those activities and their funding from our analysis. After  taking these steps, we determined that the data provided are sufficiently  reliable for our purpose of identifying U.S.-China clean energy  cooperation efforts and their obligated funding in fiscal years 2008  through 2015. We then analyzed these data by agency, by key programs,  by fiscal year, by type of activity, and by type of clean energy to be able to  describe the uses of funding provided to U.S.-China clean energy  cooperation.
To describe what is known about the results of U.S.-China clean energy  cooperation, we focused on the programs that received the largest  amount of funding from each of the three agencies that provided the most  funding to U.S.-China clean energy cooperation. The key programs we  identified were DOE’s CERC program, USTDA’s East Asia Program, and  State’s U.S.-China Climate Change Working Group (CCWG). Both  USTDA’s East Asia Program and State’s CCWG program have some  aspects not related to China or clean energy. For our report, we limited  our analysis of program results to those aspects of the programs related  to clean energy cooperation with China. However, for our analysis of  these programs’ performance monitoring, we looked at the whole  programs because the same monitoring processes were followed for all  aspects of the programs.
We analyzed the results of the three key clean energy programs.
To describe the results that CERC yielded as of December 31, 2015,  for its 19 key performance measures, we discussed with agency  officials which performance measures could be aggregated across the  years of CERC’s first phase—2011 to 2015. CERC has 12 key  performance measures that can be aggregated to show total results  for that time period. Of those 12 performance measures, we excluded  3 measures related to funding and cost-share because those are  related to inputs and not program results. We determined the results  for the remaining 9 performance measures based on agency  documentation. To illustrate different types of results, we also  selected examples of nonquantifiable key outcomes as reported by  CERC for each of the three program tracks from lists of outcomes that  DOE considers most important.
To describe the export results generated by the 24 USTDA East Asia  Program projects in our review as of the end of fiscal year 2015, we  analyzed project documents for reported export outcomes and  compared those data to information provided by USTDA from its  internal database. We calculated the export multiplier for these  projects using USTDA’s formula of dividing the total amount of exports  by the amount obligated for these projects. We also judgmentally  selected examples of USTDA’s development impacts from our review  of project documents to illustrate different types of development  impacts.
To describe the results of the six CCWG initiatives in our review as of  the end of fiscal year 2015, we examined the interagency agreements  between State and the agencies implementing the initiatives. These  agreements establish the performance measures and targets for each  initiative. We obtained results data for each performance measure  from the agencies’ performance reporting to State. We then  aggregated the targets and results across the six initiatives for each  performance measure.
To assess the reliability of the results data, we reviewed agency  documents, including an external audit of one agency’s data system; to  the extent possible, cross-checked results information that was reported  in multiple documents; and interviewed agency officials regarding how  they validate their data. We determined that the data are sufficiently  reliable for describing the results of USTDA’s East Asia Program and  CCWG through fiscal year 2015 and CERC’s results through December  2015.
To describe how DOE, USTDA, and State monitor the performance of  CERC, the East Asia Program, and CCWG, respectively, we analyzed  documents from each agency such as evaluation manuals, contracts, and  performance reports. We also interviewed knowledgeable officials from  each agency to discuss their processes for monitoring program  performance. The GPRA Modernization Act of 2010 (GPRAMA) requires  agencies to have a performance plan that, among other things, contains  performance measures with established targets that can be used to  assess progress toward achieving those targets. GPRAMA also requires  that agencies make public their performance plans containing information  on their goals and performance measures. Furthermore, linking goals to  performance measures that are tracked against established targets is a  leading practice for federal programs. To examine whether the three  agencies followed these leading practices to measure their programs’  performance, we reviewed agency planning and performance reporting  documents to determine whether they contained goals, performance  measures, and targets. We then confirmed our analysis by meeting with  officials from each program to discuss whether the programs had goals  linked to performance measures with established targets, as well as  whether USTDA had these elements agency-wide.
We conducted fieldwork at the locations of the CERC lead organizations  in West Virginia, Michigan, and California in November and December  2015 to interview representatives of the CERC lead organizations, other  CERC participants, and CCWG participants to collect information on  results and reporting. Also, during these interviews with CERC  participants and others conducted by phone, we discussed the potential  benefits and risks of participating in CERC and DOE’s management of  risks. In addition to management and researchers at the three CERC lead  organizations, we interviewed representatives from a nonprobability  sample of 10 other organizations that have participated in CERC.  Specifically, we interviewed representatives of three participants in the  clean vehicles track, four participants in the clean coal track, and three  participants in the energy efficiency in buildings track. These participants  included eight private companies, one university, and one national lab  that participate, or have participated, in CERC. One private company did  not comment on IP because it did not participate in a research project  with China, so that company is not included in our analysis of IP issues.  We selected these CERC participants using criteria such as type of  organization, current or former participant, amount of involvement in  CERC, and proximity to a CERC lead organization we were visiting.  Because we selected a nonprobability sample, the information obtained  from these interviews is not generalizable to other CERC participants, but  it provides illustrative information. During the site visits, we also  interviewed four CCWG participants to learn more background on this  program. These participants were selected based on proximity to a CERC  lead organization we were visiting. For CCWG, we also interviewed  officials from the agencies implementing all six of CCWG’s clean energy  initiatives through 2015, including DOE, the Department of  Transportation, and the Environmental Protection Agency.
To determine what risks, if any, U.S. companies and researchers  participating in CERC may face, we analyzed relevant documents and  conducted multiple sets of interviews. We reviewed documents from the  Office of the United States Trade Representative and the U.S. Patent and  Trademark Office that describe IP issues and other risks related to doing  business in China. In addition to interviewing CERC participants and DOE  officials, we interviewed eight individuals we learned to be knowledgeable  about U.S.-China cooperation on clean energy to get their perspective on  the potential IP risks for CERC participants and, in some instances, the  steps DOE has taken to address those risks. To determine the extent to  which DOE has taken steps to manage these risks, we identified the risk  management steps DOE has taken and then compared these steps to  federal internal control standards for risk assessment, which state that  agencies should assess the risks the agency faces from both internal and  external sources and decide how to manage those risks and what actions  should be taken. Specifically, we analyzed relevant documents, including  the CERC Protocol and IP Annex, the Technology Management Plan for  each CERC track, the CERC “Researchers’ Guide to IP and Technology  Transfer,” and the results of a CERC IP survey. We directed clarifying  questions about the IP survey to DOE officials and a CERC participant  and determined that the survey information is accurate and reliable for  our purposes. We also interviewed knowledgeable DOE officials to  understand what steps DOE took to identify and respond to any risks that  U.S. participants in CERC may face, and we interviewed CERC  participants to get their feedback on the effectiveness of these steps.
We conducted this performance audit from June 2015 to July 2016 in  accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.
section_title: Appendix II: Appropriations Accounts Used for Bilateral Cooperation with China on Clean Energy
paragraphs: The seven U.S. agencies that obligated funding to U.S. participation in  bilateral cooperation with China on clean energy in fiscal years 2008  through 2015 did so using funding from a variety of appropriations  accounts. Table 3 provides a list of those accounts.
section_title: Appendix III: Key U.S.-China Clean Energy Cooperation Programs’ Organization and Reporting Relationships
paragraphs: The following figures show the organizational and reporting relationships  for the three key U.S.-China clean energy programs: the Department of  Energy’s U.S.-China Clean Energy Research Center (CERC), the U.S.  Trade and Development Agency’s (USTDA) East Asia Program, and the  Department of State’s U.S.-China Climate Change Working Group  (CCWG).
Figure 5 shows how CERC fits within the Department of Energy, including  where CERC tracks get their funding, responsibilities for program  oversight, and performance reporting channels.
Figure 6 shows how the USTDA East Asia Program fits within USTDA,  including where the East Asia Program’s projects get their funding,  responsibilities for program oversight, and data analysis and performance  reporting channels.
Figure 7 shows how CCWG fits within the Department of State, including  where the CCWG initiatives get their funding, responsibilities for program  oversight and strategic direction, and performance reporting channels.
section_title: Appendix IV: Comments from the Department of Energy
section_title: Appendix V: Comments from the Department of State
section_title: Appendix VI: Comments from the U.S. Trade and Development Agency
section_title: Appendix VII: GAO Contacts and Staff Acknowledgments
subsections: section_title: GAO Contacts
section_title: Staff Acknowledgments
paragraphs: In addition to the contacts named above, Kim Frankena (Assistant  Director), Karla Springer (Assistant Director), Heather Latta (analyst in  charge), David Dayton, Rachel Girshick, Marya Link, and Dina Shorafa  made key contributions to this report. Jill Lacey, Lisa Pearson, Sara  Sullivan, and Alex Welsh provided technical assistance.