text: Background Establishment, Mission, and Objectives of the Lab In 2014, USAID established the Lab as a USAID bureau by merging and restructuring two offices—the Office of Science and Technology and the Office of Innovation and Development Alliances. According to USAID officials, the agency moved a number of the two offices’ core programs and activities, along with staffing functions, to the Lab. In a January 2014 notification, USAID informed Congress of its intent to establish the Lab and noted initial staffing levels, funding, and short-term plans. The Lab is generally subject to guidance pertaining to operating units and bureaus, including policies and procedures set out in USAID’s ADS. It also publishes and contributes to various performance and financial reporting of information, such as USAID’s Annual Performance Plan and Report, which are provided to Congress and available to the public, according to Lab officials. The Lab was created to work collaboratively within USAID and with other government and nongovernment partners to produce development innovations, among other things. According to Lab officials, the Lab seeks to improve USAID’s ability to harness the power of science, technology, innovation, and partnerships (STIP) with private and public sectors by funding and scaling breakthroughs that would accelerate the completion of foreign policy and development goals. The Lab has a two-part mission: 1. Produce development breakthroughs and innovations by funding, testing, and scaling proven solutions that will affect millions of people. 2. Accelerate the transformation of development enterprise (i.e., to build capacity of the public and private sectors to work in the development arena) by opening it to people everywhere with good ideas, promoting new and deepening existing partnerships, applying data and evidence, and harnessing scientific and technological advances. The Lab’s mission, objectives, and goals are laid out in its strategic plan, which has evolved since the Lab’s creation. In fiscal years 2014 and 2015, the Lab operated under an initial strategy that focused on examining the delivery capabilities and constraints of current and ongoing Lab programs; prioritizing investments of time and resources; and confirming new activities and programs. The strategy for fiscal years 2016 through 2020 presents a results framework that includes the Lab’s two- part mission statement as well as five objective statements and corresponding intermediate result statements explaining how the Lab intends to achieve its goals (see fig. 1). Structure of the Lab The Lab, which is headed by an Executive Director, includes five centers—the Center for Development Research, the Center for Digital Development, the Center for Development Innovation, the Center for Transformational Partnerships, and the Center for Agency Integration— each focused on one of the Lab’s five strategic objectives. The Lab also includes two offices, the Office of Engagement and Communication and the Office of Evaluation and Impact Assessment, which provide support services. Figure 2 shows the Lab’s organizational structure. Table 1 describes each of the Lab’s centers and offices. In April 2018, the USAID Administrator announced agency reorganization plans that will affect the Lab. USAID leadership plans to create a new Bureau for Development, Democracy, and Innovation and a Bureau for Policy, Resources, and Performance. According to USAID, the new bureaus will combine existing operating units that provide technical and program design support and expertise into a “one-stop shop” of consultancies that USAID missions can utilize. The new bureaus will absorb the Lab, along with other units, and track its contributions using new metrics that measure customer service to determine whether missions and bureaus have access to the right expertise at the right time, according to the USAID Administrator. As of October 2018, USAID had not indicated time frames for implementing the reorganization plans. Funding Mechanisms for Lab Activities To achieve its objectives and goals, the Lab funds and manages awards (which result in activities) that cover STIP programming as well as the Lab’s operations. The Lab uses a number of different mechanisms—for example, broad agency announcement procedures, annual program statements, and requests for applications—when making awards, which include grants, cooperative agreements, and contracts. Global Development Alliance A Global Development Alliance (GDA) is a partnership involving the U.S. Agency for International Development (USAID) and the private sector. GDA partners work together to develop and implement activities that leverage and apply assets and expertise to advance core business interests, achieve USAID’s development objectives, and increase the sustainable impact of USAID’s development investments. Generally, according to USAID, the value of private sector expertise, capabilities, and resources contributed to an alliance must equal and, in general, should significantly exceed the value of resources provided by USAID. The Lab also holds competitions focused on new ideas, approaches, and technologies to address development problems, and awards prizes to individuals or groups that meet the competition’s requirements. Some awards include funding from USAID as well as cash or in-kind contributions from non-USAID sources in the private or public sector. The Lab refers to the use of all non-USAID contributions as leverage and reports leverage as a programmatic performance indicator. According to USAID documents, the agency seeks to build partnerships that leverage the assets, skills, and resources of the public, private, and nonprofit sectors to deliver sustainable development impact. Examples of such leverage contributions include donated cash, services, or supplies from implementing partners or third parties to specific awards managed by the Lab. Third parties contributing to Lab managed programs have included foreign governments, international organizations, businesses and corporations, philanthropic foundations, non-governmental organizations, and higher education institutions, among others. One method USAID has approached this goal is through Global Development Alliances (see sidebar). The Lab Aligns Programs to Support Its Five Strategic Objectives; Funding and Staffing Have Decreased since Fiscal Year 2015 The Lab Aligns Programs with Its Strategic Objectives Staff in the Lab’s five centers, offices, and Lab-Wide Priorities manage more than 25 programs and portfolios, which encompass projects and activities under a specific issue, aligned with the Lab’s five strategic objectives. The programs focus on development research (science objective), digital development (technology objective), innovation ventures (innovation objective), and private-sector engagement (partnerships objective). Table 2 shows examples of programs and portfolios aligned with each strategic objective. Examples of the Lab’s programs and activities include the following (see app. II for more information about these and other Lab programs): Staff in the Lab’s Center for Development Innovation manage the Grand Challenges for Development initiative, intended to foster innovations to address key global health and development problems. Since 2011, USAID and its partners have launched 10 Grand Challenges that are implemented by USAID bureaus, including the Lab. The Lab is responsible for managing the Securing Water for Food Grand Challenge and also the Scaling Off-Grid Energy Grand Challenge. Other USAID bureaus implement the other eight Grand Challenges (see app. III for a description of the Grand Challenges). Staff in the Lab’s Center for Development Research manage the Higher Education Solutions Network. The program is a partnership with seven universities that also work with partners in academia, the private sector, civil society, and governments worldwide. The universities established eight development labs that focus on efforts to solve a range of development problems. The Lab’s two offices support various aspects of the centers’ programs and portfolios, such as internally promoting center programs throughout USAID and conducting monitoring and evaluation activities. Types of STIP Services Provided by the Global Development Lab Digital development: Technologies and data-driven approaches to extend the reach of development programs Catalyzing innovation: Integration of design methodologies, development innovations, and programming solutions to solve development challenges differently. Partnerships/private sector engagement: Relationships between USAID and one or many organizations, including private sector entities, in an effort to create development impact. Scientific research and capacity building: Application of science and research to solve development problems. In addition to managing programs, the centers provide a variety of STIP- focused services and support, including assistance with programming, to USAID field missions and headquarters bureaus as part of the Lab’s mission to accelerate development impact. According to Lab documentation, the Lab can provide services related to country and regional strategic planning; project design and implementation; activity design and implementation; and monitoring and evaluation. The Lab’s STIP services fall into several categories—digital development, catalyzing innovation, partnerships and private sector engagement, and scientific research and capacity building, according to Lab documents (see sidebar). The centers, led by the Center for Agency Integration, deliver internal STIP services and mechanisms through toolkits, training, advisory services, and assessment and analysis of STIP activities or programming, according to Lab documentation. For example, at the request of missions or bureaus, the Digital Finance team in the Center for Digital Development can, among other things, review and provide technical input on awards related to digital finance. In addition, the Lab has provided advisory services to USAID operating units regarding innovative design or methods, such as co-creation, which can be used throughout the program cycle including in procurement (i.e., the broad agency announcement, annual program statements, etc.). According to Lab officials, some services are funded by the Lab at no cost to USAID operating units, while other services must be funded by the USAID operating units through funding mechanisms such as “buy-ins” or cooperative agreements. Lab data for fiscal years 2014 through 2017 show that the Lab provided services or support frequently in digital development activities, such as geospatial support to USAID field operations, and partnership services. For example, the Lab has provided technical services to missions around the world related to the GeoCenter (housed in the Center for Digital Development), which supports the application of advanced data and geographic analysis to international development challenges to improve the strategic planning, design, monitoring, and evaluation of USAID’s programs. In addition, the Lab provided partnership services related to private-sector engagement, including technical assistance and consultative services to USAID missions for more efficiently engaging, building, and maintaining relationships with the private sector at local or regional levels. Officials we interviewed at USAID missions and headquarters bureaus described services or tools they had received from the Lab, such as technical advice and training related to establishing private-sector partnerships and leveraging funding. For example, some USAID headquarters officials told us they had taken Lab-led private-sector engagement training that addressed developing collaborations with external stakeholders, establishing risk-sharing agreements, and engaging investors and other financial sector actors. In addition, some mission officials stated that they were involved in Lab-supported programs such as the Partnerships for Enhanced Engagement in Research and the Partnering to Accelerate Entrepreneurship Initiative and had received Lab support related to geographic information system mapping. One mission had a Lab-funded embedded advisor who provided technical assistance to a country’s Ministry of Health. According to Lab officials, demand for the Lab’s services and support exceeds the Lab’s capacity and its resources. Program Funding for the Lab Has Decreased since Fiscal Year 2015 Allocations of program funds from USAID to the Lab have decreased over the past few fiscal years, from $170.7 million in fiscal year 2015 to $77 million in fiscal year 2017. Similarly, the Lab’s obligations of program funds have also decreased since fiscal year 2015, according to Lab data. Obligations reached around $170 million in fiscal year 2015, the Lab’s first full year of operations. By fiscal year 2016, the Lab’s obligations had decreased to about $109 million—a reduction of over 35 percent. Although the Lab is still obligating fiscal year 2017 funding, its obligations would not exceed $77 million if it obligated the full amount of program funding provided to the Lab. As table 3 shows, from fiscal year 2014 through fiscal year 2017, the Lab obligated over $435 million of its program funds for its centers and support services (see app. IV for an overview of funding from various appropriations accounts in fiscal years 2014-2017). According to Lab officials, the program funds cover Lab- managed programs and programming (including funding for awards comprised of many activities) and the centers’ services, STIP activities, and staffing (including contractors), among other things (see app. V for a discussion of Lab-managed activities and corresponding obligations for fiscal years 2014-2017). As table 3 shows, in fiscal years 2014 through 2017, the Lab’s Center for Development Innovation obligated the most funds overall. The center houses the Development Innovation Ventures, a portfolio of innovations with the goal of reducing global poverty. Borrowing from the private sector’s venture capital model, the portfolio seeks to identify and test innovative development solutions based on three principles: rigorous evidence, cost-effectiveness, and potential to scale up. Lab officials indicated that the Lab has reassessed and realigned programming priorities because of decreased funding. For example, the Lab temporarily suspended new applications for awards through the Development Innovation Ventures program from the end of July 2017 due to budget uncertainties in fiscal year 2018. However, Lab officials indicated that the Lab has recently secured funding for new applications for the program. Funding decreases have also caused the Lab to scale back or put some programs on hold, according to Lab officials. For example, the Lab scaled back its Partnering to Accelerate Entrepreneurship Initiative; its Lab-Wide Priorities; and its Monitoring, Evaluation, Research, and Learning Innovation programs. The Lab also put its partnerships with NextGen missions on hold indefinitely, according to Lab officials. In addition, the Lab reported that it has been able to provide only minimal support for multi-stakeholder partnerships, such as the Digital Impact Alliance and the Global Innovation Fund. Lab Staffing Has Decreased since Fiscal Year 2015 The number of staff in the Lab has decreased since fiscal year 2015, the first year for which staffing numbers are available. Lab staff include both direct-hire staff, comprising civil service and foreign-service employees, and contractors with specialized skills who supplement the efforts of direct-hire staff. Contractors have made up at least 35 percent or more of staff each fiscal year since 2015. The total number of staff, including direct-hire staff and contractors, decreased by over 30 percent from fiscal years 2015 through 2018, dropping from 224 in fiscal year 2015 to 155 in fiscal year 2018 (see table 4). Lab information shows that the staff primarily comprise senior technical and professional experts and that about 80 percent of staff are on time- limited appointments, which can last 1 to 5 years, according to Lab officials. Further, according to Lab officials, due to the ever-changing nature of work in the Lab, staff may work on multiple projects and activities across several teams or may be assigned to work with one team or on a single project until it is completed. For example, Lab officials stated that when Lab-Wide Priorities are established, staff members are brought in to contribute to these efforts while also working on activities in the centers they support. In addition to declining staff numbers overall, since fiscal year 2015, the number of direct-hire staff employed by the Lab has decreased. According to Lab officials, because of the technical focus of its programming, the Lab has not been able to staff all authorized positions with direct-hire employees who have the necessary expertise. Instead, the Lab has filled some of these positions with contractors or science fellows. The Lab also uses a variety of other hiring mechanisms, such as the Participating Agency Service Agreement with the Department of Agriculture and the American Association for Advancement of Science fellows, to allow for flexibility and obtain the needed expertise to implement STIP and technical services throughout USAID. By fiscal year 2017, the Center for Digital Development had 40 staff members—the highest overall number, including the highest number of contractor staff members—among all the Lab’s centers. This center’s contractor staff primarily consisted of technical specialists assisting the GeoCenter (see app. VI for numbers of direct hires and contractors at each center in fiscal years 2015-2018). Lab officials stated that the decline in staff numbers—primarily direct-hire staff—over the years was due to a number of factors, including a government-wide hiring freeze, budget constraints, and a high attrition rate among the Lab’s staff beginning in 2017. According to several Lab officials, the high attrition rate was due to uncertainty about the USAID reorganization and its impact on the Lab, since a large percentage of the Lab’s staff is employed on a term-limited basis. The Lab Documented Its Oversight of Awards with Non- USAID Contributions, but Some Data Are Outdated and Public Reporting Lacks Transparency The Lab’s Documented Oversight of Awards with Non-USAID Contributions Followed USAID and Lab Guidance Our review of Lab documents showed that, for all 24 Lab-managed awards we reviewed, the Lab consistently documented certain oversight requirements for non-USAID contributions (i.e., committed, rather than actual, contributions from the private sector, the public sector, and other U.S. government agencies). We reviewed 24 Lab-managed awards that included non-USAID contributions to determine whether the Lab documented its compliance with key award oversight requirements we identified in USAID and Lab guidance. For all 24 awards, the Lab documented its compliance with the following key requirements: report funding amounts committed from non-USAID sources; conduct valuations of in-kind contributions, as applicable; document partners met cost-share or matching funds, if required; maintain copies of the award agreement and any modifications. Additionally, for awards receiving in-kind contributions, the Lab maintained documentation in award files demonstrating that officials reviewed the valuation of in-kind services and supplies. Further, in the 10 awards we reviewed containing cost-share requirements, the Lab maintained documentation to show partners’ progress in meeting those requirements. The Lab’s Data for Some Non-USAID Contributions Are Outdated We found that the Lab’s management information system contained outdated data on non-USAID contributions, which the Lab reports as leverage. According to ADS 596, information should be communicated to relevant personnel at all levels within an organization and the information should be relevant, reliable, and timely. Further, Standards for Internal Control in the Federal Government states that management should use quality information to achieve the entity’s objectives, including obtaining relevant data from reliable internal sources in a timely manner. Further, the Lab’s “Internal Guide to Accounting for Leverage” (internal guide) states that data on non-USAID contributions will be collected from Lab teams semi-annually. Our analysis of data in the Lab’s management information system found that one of two tables used to develop a number of internal and external reports contained outdated data for 10 of the 24 awards we reviewed and, in some cases, had not been updated for more than 2 years. Although this table showed a total of about $24.5 million in non-USAID contributions for these 10 awards, award documentation provided by the Lab showed the updated amount of non-USAID contributions to be about $12.1 million. For example, for an award aimed at providing hydro- powered irrigation pumps in Nepal, the table showed committed non- USAID contributions of about $262,000, while our review of award documentation found that the updated amount was about $410,000. For another award aimed at providing drip irrigation systems for small-plot farmers in India, the table showed partners had committed $362,000 in non-USAID contributions. However, in reviewing award documentation, we found that partners had ultimately committed about $61,600 to this award. The Lab’s internal guide does not provide instructions for ensuring that the non-USAID contributions data in USAID’s management information system are timely. According to Lab officials, the outdated data we identified resulted from staff’s failure to manually enter updated data in both of the two tables used for external reporting. Lab officials stated that leverage data are entered manually because the Lab’s management information system does not have the capacity to automatically update the tables. However, we found that the Lab’s internal guide does not describe the Lab’s current process for entering leverage data in the system or include instructions for ensuring that these data are regularly updated. Instead, the internal guide refers to a data collection practice that predates the Lab’s management information system and that, according to Lab officials, is no longer in use. To the extent that the Lab used outdated data when generating external reports and budget exercises, it risks reporting incorrect information about non-USAID contributions to Lab awards. According to Lab officials, the table with outdated data on non-USAID contributions that we identified in the Lab’s management information system is one of the data sources that the Lab uses for reports to the USAID Administrator’s Leadership Council and the Department of State and in USAID’s Annual Performance Plan and Report. According to Lab documentation, the Lab also uses these data to develop a number of annual budget formulation and justification exercises, including congressional communications. Providing instructions for updating all non-USAID contributions data in its management information system could help the Lab strengthen the timeliness and reliability of these data and of the external reports that include them. The Lab’s Internal Guide Does Not Require Its Public Reporting of Leverage Data to Disclose Types of Non-USAID Contributions Represented The Lab’s internal guide does not require its public reporting of data on non-USAID contributions, or leverage, to disclose the types of contributions represented. According to ADS 596, information should be communicated to relevant personnel at all levels within an organization and the information should be relevant, reliable, and timely. In addition, according to Standards for Internal Control in the Federal Government, management should externally communicate complete and accurate information to achieve an entity’s objectives. The Lab defines leverage more broadly than the Agency’s definition found in USAID’s ADS 303. Specifically, these definitions differ in two ways. First, the Lab definition includes cost-share contributions, which the ADS definition excludes. Second, the ADS definition limits leverage to public-private partnership awards, while the Lab’s definition does not contain a similar limitation. Because the Lab’s definition of leverage differs from the definition in ADS, the Lab uses two separate indicators to track non-USAID contributions, according to Lab officials. For the leverage data it collects for USAID reporting on public-private partnerships, the Lab adheres to the ADS definition, accounting as leverage all non-USAID resources, excluding cost sharing, that are expected to be applied to a program in USAID public-private partnership awards. For the leverage data it collects for its internal performance management and external reports, the Lab accounts in its leverage calculations all cost-share contributions (from both private and public-sector partners); all other contributions (from the private sector, the public sector, and other U.S. government agencies); and gifts (from bilateral donors). According to Lab officials, the Lab’s definition of leverage differs from the ADS definition because the Lab partners with both the private and public sectors in its contracts and awards, and the Lab’s more expansive definition allows it to fully account for all non-USAID contributions. However, despite the difference in the Lab’s and USAID’s definitions, the Lab’s internal guide does not require that its public reporting of leverage data identify the types of non-USAID contributions represented in the data. As a result, the Lab’s public reporting—for example, on its webpage—provides the total amount leveraged but does not specify the types of contributions committed by non-USAID partners. Given the difference between the Lab’s definition used in its public reporting and the ADS definition of leverage, USAID lacks assurance that it is reporting transparent data on leveraged non-USAID contributions. Moreover, because the Lab’s internal guide does not require the Lab’s public reporting of leverage to disclose the types of contributions, Congress and the public lack access to complete information about the extent and nature of the Lab’s partnerships. By specifying the types of non-USAID contributions included in its data on leveraging, the Lab could increase the transparency of its public reporting for this key metric. The Lab Uses Various Tools to Assess Its Performance; Assessments Have Identified Both Positive Results and Some Weaknesses The Lab uses various tools, such as its results framework, portfolio reviews, strategic learning reviews, and evaluations, established by USAID policy or Lab-specific practices to assess its performance. Because the Lab has existed only since 2014 and has had a strategy only since 2016, it has been able to collect a limited amount of data with which to assess its performance to show any trends in achieving results. However, the performance assessment tools that the Lab uses have identified both positive results and some weaknesses or challenges. Results Framework Tool and Identified Results The Lab’s strategy for fiscal years 2016 through 2020 includes a results framework comprising the Lab’s five strategic objectives, as shown previously in figure 1. For each strategic objective, the framework presents a corresponding development objective—that is, the most ambitious result that a Lab center aims to achieve through its projects and activities—as well as targets the Lab is focused on achieving by 2020. Progress toward the targets is tracked with annual and, in some cases, semi-annual performance indicators, according to Lab officials (see app. VIII for a list and descriptions of the Lab’s indicators). According to Lab officials, the Lab considers the results framework a living document and adjusts indicators and targets as necessary based on changing circumstances. The Lab’s indicator data indicate that, overall, the Lab met or exceeded its targets slightly more often than it did not meet them (see table 5). As table 5 indicates, the Lab met or exceeded its targets for 20 of its 39 indicators in fiscal years 2016 and 2017. For example, for one indicator— total number of program or policy changes made by public sector, private sector, or other development actors that are influenced by Lab-funded research results or related scientific activities—the Lab reported that it exceeded its target for both fiscal years. The Lab’s targets for this indicator for fiscal years 2016 and 2017 were set at 42 and 48, respectively, with reported results of 83 and 84. For another indicator— total dollar value of private and public capital catalyzed for early-stage entrepreneurs as a result of USAID support—the Lab reported it had exceeded its fiscal year 2017 target of $575 million, with an actual result of around $686 million. In addition, the Lab improved its performance for seven indicators, according to its data. For instance, for agency integration indicators—such as the number of operating units that have integrated STIP at the strategic, programmatic, and organizational levels—the Lab went from not meeting its targets in fiscal year 2016 to exceeding its targets in fiscal year 2017. The Lab’s indicator data also show some areas in which the Lab has faced challenges or has not met its targets. As table 5 shows, the Lab did not meet its targets for 19 of the 39 indicators in fiscal years 2016 and 2017. For example, for one indicator—number of operating units that have integrated STIP at the strategic, programmatic, and organizational levels—the Lab did not meet its targets of 15 and 20, respectively, for fiscal years 2016 and 2017, with reported results of 12 and 19. For another indicator—number of smart innovation methods adopted by USAID operating units—the Lab set a target of eight but reported an actual result of six. Moreover, from fiscal year 2016 to fiscal year 2017, the Lab’s performance declined for seven indicators. For instance, for innovation indicators—number of system actors engaged in innovation methods and number of smart innovation methods adopted by agency operating units—the Lab went from exceeding its targets in fiscal year 2016 to not meeting them in fiscal year 2017. Lab officials stated that the Lab’s performance goals were meant to be ambitious and that the Lab would adjust goals on the basis of resource and budget constraints. Portfolio Review Tool and Identified Results The Lab has implemented biannual portfolio reviews of projects and activities. According to Lab officials, the portfolio reviews assess progress toward strategic objectives, provide Lab staff an opportunity to share lessons learned, and foster collaboration across the centers. In fiscal years 2016 and 2017, the Lab conducted four portfolio reviews— two at midyear and two at the end of both years. Each portfolio review discussed the performance of each center, examined how well the center was meeting the targets for its performance indicators, and addressed topics such as key achievements and challenges and priority evaluation and research questions for the upcoming fiscal year. Lab officials stated that portfolio reviews have helped the Lab become more rigorous and better understand the reasons for implementing the various projects and activities. The Lab’s portfolio reviews for fiscal years 2016 and 2017 highlight, among other things, lessons learned and achievements made for particular projects and toward the Lab’s overall strategic objectives and targets. The reviews also note challenges faced Lab-wide as well as planned adjustments to address these challenges. Examples of the portfolio reviews’ findings, by strategic objective, include the following for each of the five Lab centers: Science. The review noted that lessons learned by the Center for Development Research included emphasis on managing relationships and the need to communicate with missions about the ways in which research can help them contribute to their objectives. The review also noted that the center’s challenges included striking the right balance between different elements of the science objective in the Lab strategy and developing mission-focused tools for integrating research. Technology. The review noted that the Center for Digital Development achieved largely positive ratings for digital development training and for a substantial amount of technical assistance, trainings, and knowledge products. The review also noted that the center had faced some challenges, such as staffing constraints that limited staff’s ability to prioritize both internal and external engagements. Innovation. The review noted that the Center for Development Innovation had several achievements, including positive feedback from innovators who received technical assistance from the center as well as agency partners who received program design services. The challenges noted included the center’s need for more engagement with key missions and for finding balance between advisory services and direct project implementation. Partnerships. The review noted that the Center for Transformational Partnerships had identified lessons learned in areas such as the center’s ability to support missions by helping them to identify opportunities and determine when and where partnership makes sense. One challenge that the review identified was the possibility that the center’s limited resources might inhibit technical assistance to missions and bureaus. Planned adjustments included prioritizing advisory and liaison support to the regions that have lower capacity for private sector engagement. Agency integration. The review noted that the Center for Agency Integration achieved several successes, including introducing the Lab and STIP to over 30 Foreign Service nationals (i.e., local, non-U.S. citizens employed by USAID), several of whom continued to champion STIP at their missions. The review also noted challenges, such as staffing and capacity gaps, that hampered training efforts as well as USAID staff being overwhelmed by the amount of information flowing from the Lab. Strategic Learning Review Tool and Identified Results In October 2017, the Lab implemented an evaluation, research, and learning plan that includes practices recommended for bureaus. According to Lab officials, the Lab’s plan is intended to help build evidence within and across the centers and ensure that resources are prioritized to support evaluation and research. As part of this plan, the Lab identified five key questions for all of the centers that evaluations, research, and learning efforts should attempt to help answer. According to Lab officials, the Lab began holding strategic learning reviews, beginning in spring 2018, to help it address theories of change—that is, descriptions of how and why a result is expected to be achieved through a particular project or activity. The Lab developed the reviews to complement its portfolio reviews, according to Lab officials. The Lab, led by the Office of Evaluation and Impact Assessment, completed its first cross-Lab strategic learning reviews in the spring of 2018, according to Lab officials. The reviews focused on three of five key questions in the Lab’s evaluation, research, and learning plan: addressing adaptive management; supporting innovators, entrepreneurs, and researchers; and sustaining results. According to the Lab, the 2-hour sessions, in which Lab officials and other selected agency subject-matter experts participated, resulted in discussions about issues that the participants considered most important for the Lab to address or improve in the future. For example, participants identified actions that could be currently achieved, such as designating time for “pause and reflect” exercises, particularly for reducing USAID’s administrative burden for first-time Lab partners that lack the capacity to manage USAID requirements; and focusing on larger market-enabling environments rather than on a single value chain. According to Lab documents, the Lab plans to use data from the reviews to develop recommendations that will be reflected in an action memo and to track any actions the Lab takes to implement the recommendations. Lab officials stated that the Lab plans to hold three additional 2-hour strategic learning reviews in fall 2018. Evaluation Tool and Identified Results Evaluation Evaluation is the systematic collection and analysis of information about the characteristics and outcomes of programs and projects that provides a basis for judgments to improve effectiveness and/or inform decisions about current and future programming. The Lab assesses its performance through evaluations (see sidebar). According to Lab officials, the Lab has conducted both external evaluations and internal evaluations, and the majority of its performance evaluations are external. As of October 2018, the Lab had primarily completed performance evaluations, although Lab officials reported that the Lab was also conducting three impact evaluations and one developmental evaluation. In addition to conducting evaluations, the Lab conducts assessments—management tools used to gather information about context or operating environment or to review an activity or project. As of October 2018, the Lab reported that it had completed 7 external performance evaluations of its programs or projects and had an additional 12 ongoing evaluations, both internal and external. The Lab’s completed performance evaluations cover a variety of programs, activities, and USAID services, such as the Securing Water for Food Grand Challenge project and the Lab’s technical assistance services. We reviewed the seven completed external performance evaluations and found that they identified a range of program strengths as well as challenges or weaknesses. For example: Mid-Term Review of Securing Water for Food: A Grand Challenge for Development. The evaluation identified program strengths, such as a diversity of innovations in the portfolio. The evaluation also found that the program had potential weaknesses, including a lack of focus on innovations for locations with greater water scarcity. Mid-Term Evaluation of the Partnerships for Enhanced Engagement in Research Program. The evaluation found, among other things, that partnerships between scientists in developing countries and in the United States have been of value for scientific output and strengthening professional relationships. In addition, the evaluation identified potential weaknesses in the program, including the need to facilitate broader dissemination of research findings by convening program grantees, the private sector, government officials, and civil society partners to network and share findings as well as policy and program challenges. Mid-Term Evaluation of the Higher Education Solutions Network. The evaluation found, among other things, that development labs housed in seven higher education institutions have begun providing data to inform USAID operating units’ decision making, collaborating to develop and test new technologies and innovative approaches, and engaging in knowledge sharing and learning. Additional findings included the need for Higher Education Solutions Network labs to streamline activities, adjust resource allocations, and increase synergies based on the insights gained through the first 5 years. Global Broadband and Innovations Alliance Performance Evaluation. The evaluation found, among other things, successful outcomes of specific projects focused on sustainably increasing broadband internet connectivity in the developing world. The evaluation also found that USAID had been challenged by changing leadership in the agency, which resulted in shifting priorities. In addition, the evaluation found that limited marketing of the mechanism to missions and other bureaus and offices resulted in lower-than- expected initial buy-in from the missions. STIP Integration Performance Evaluation: West Africa Regional and Uganda. The evaluation found, among other things, that mission staff want to build their capacity to use STIP but would prefer more demand-driven services from the Lab, rather than services that do not align with mission strategies. In addition to completing formal evaluations, the Lab has completed over 15 assessments of its activities or projects since 2014 and also is conducting a number of ongoing assessments. The completed assessments reflect work in all five centers and cover areas such as digital finance services, co-creation, and STIP integration. Conclusions Since its establishment as a USAID bureau more than 4 years ago, the Lab has supported the agency’s efforts to address science, technology, innovation, and partnerships. Further, the Lab has funded and managed opportunities for innovators to propose new ideas, approaches, and technologies that tie into USAID’s overall development goals and programming. The Lab’s centers have pursued global partnerships with a wide range of non-USAID public and private sector stakeholders in an effort to augment their programming and further their efforts. However, because non-USAID contributions data that the Lab collects are not always current, some of the leverage data the Lab reports internally and externally to help demonstrate its accomplishments risks being outdated. Moreover, because the Lab’s Internal Guide to Accounting for Leverage does not require its public reporting of leverage data to identify the types of contributions represented, the Lab’s public reporting lacks transparency. Ensuring that the Lab’s internal data on non-USAID contributions are updated and that its publicly reported information about leveraged resources from the public and private sector is transparent will enable the Lab and USAID to better demonstrate to Congress and American taxpayers that the agency is maximizing its use of development resources to pursue new and innovative approaches to development challenges. Recommendations for Executive Action We are making the following two recommendations to USAID: The USAID Administrator should ensure that the Executive Director of the Lab assures that the Lab’s Internal Guide to Accounting for Leverage includes instructions to update all non-USAID contributions data in the Lab’s management information system at least annually. (Recommendation 1) The USAID Administrator should ensure that the Executive Director of the Lab assures that the Lab’s Internal Guide to Accounting for Leverage requires that the Lab’s public reporting of leverage data discloses the types of non-USAID contributions represented. (Recommendation 2) Agency Comments We provided a draft of this report to USAID for review and comment. USAID provided written comments that are reprinted in appendix IX. In its letter, USAID concurred with, and indicated that it is already addressing, both recommendations. In addition, USAID provided technical comments on the draft, which we incorporated as appropriate. We are sending copies of this report to the appropriate congressional committees, the USAID Administrator, and other interested parties. In addition, the report is available at no charge on the GAO website at http://www.gao.gov. If you or your staff have any questions about this report, please contact me at (202) 512-3149 or gootnickd@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made key contributions to this report are listed in appendix X. Appendix I: Objectives, Scope, and Methodology In this report, we examine (1) the Global Development Lab’s (the Lab) programs, funding, and staffing resources, (2) the extent to which the Lab has documented its oversight of awards with non-U.S. Agency for International Development (USAID) contributions and clearly reported these contributions, and (3) the tools that the Lab uses to assess its performance as well as results that such assessments have shown. Programs, Funding, and Staffing To examine the Lab’s programs, funding, and staffing resources, we reviewed and analyzed Lab program, funding, and staffing documents and data covering fiscal years 2014 to 2017. We reviewed the congressional notification in which USAID advised Congress of its intent to establish the Lab, program description documents, as well as the Lab’s current strategy document which contains the Lab’s results framework and strategic objectives covering science, technology, innovation, partnerships (STIP), and agency integration. In addition, we reviewed documents that provided information on services and tools the Lab provides to operating units within USAID. We reviewed and analyzed Lab funding data, by appropriations accounts, which included allocations and obligations for Lab programs by centers and offices covering fiscal years 2014 to 2017. The Lab did not yet have fiscal year 2018 funding information available. In addition, we reviewed and analyzed obligation data on Lab-managed activities for fiscal years 2014 to 2017. To report on staffing, we reviewed and analyzed Lab staffing data for fiscal years 2015 to 2018 which included data on the number of direct hire staff and contractors, hiring mechanisms used to bring staff on board, as well information on the centers and offices the staff worked in. To assess the reliability of the staffing data for fiscal years 2015 to 2018 and the funding data for fiscal years 2014 to 2017, we compared and corroborated information provided by the Lab with staffing and funding information in the Congressional Budget Justifications for the fiscal years. On the basis of the checks we performed, we determined these data to be sufficiently reliable for the purposes of this report. We interviewed Lab officials representing every center—Center for Development Research, Center for Digital Development, Center for Development Innovation, Center for Transformational Partnerships, and Center for Agency Integration; each support office—Office of Engagement and Communication, and Office of Evaluation and Impact Assessment; and all Lab-Wide Priorities—Ebola, Digital Development for Feed the Future, and Beyond the Grid—to understand the Lab’s organizational structure, roles and responsibilities, programs, and services, among other things. We also spoke with officials in the Administrative Management Services and Program and Strategic Planning offices, which cover the Lab’s financial and human resources, as well as strategic planning and reporting. To obtain insight into the Lab’s interaction and STIP integration within USAID, we also interviewed agency officials from five USAID bureaus in Washington, D.C.— Democracy, Conflict, and Humanitarian Assistance; Economic Growth, Education, and Environment; Food Security; Global Health; and Policy, Planning, and Learning; and from six USAID missions overseas— Albania, Cambodia, Guinea, Haiti, Uganda, and the Regional Development Mission for Asia. To determine the number of activities the Lab managed from fiscal years 2014 through 2017, and the amount it had obligated for these activities in this timeframe, we reviewed and analyzed data from USAID’s financial management system—Phoenix. Additionally, we met with Lab officials responsible for managing and reviewing the data in this system. To ensure that we accounted for only programmatic activities in our timeframe, we removed activities, in consultation with Lab Officials, from the dataset that pertained to institutional support contracts and fellowships. We also met with officials from each of the Lab’s centers to discuss the activities that they manage. We determined that the data were sufficiently reliable to account for Lab managed activities. Oversight, Documentation, and Reporting of Non- USAID Contributions To address oversight and documentation of awards with non-USAID contributions, we reviewed Lab and USAID policies and guidance for oversight of non-USAID contributions as of fiscal year 2017, including Lab guidance, and relevant chapters of USAID’s Automated Directives System (ADS), which contain the agency’s policy. We analyzed Lab- managed awards with committed funding from non-USAID partners from fiscal years 2014 through 2017 (a total of 154) from the Lab’s information management system DevResults, which we determined was sufficient to allow us to select a sample of these awards for further review. Our sample included 24 awards, which represented all Lab-managed awards containing non-USAID contributions issued on or after fiscal year 2014, and ending in or before fiscal year 2017. We selected these timeframes to ensure that the awards we reviewed did not predate the creation of the Lab (fiscal year 2014) and to ensure that activities and all award documentation on activities had been completed. To assess the reliability of these committed funding data, we reviewed documentation and interviewed USAID officials to identify and rectify any missing or erroneous data. Since we selected only awards in our given timeframe, the results cannot be generalized to all Lab managed awards receiving non-USAID committed contributions. We determined that the data and information were sufficiently reliable to compare against award documentation. The awards we reviewed covered four of the Lab’s five objectives: science (1 award), technology (3 awards), innovation (19 awards), and partnerships (1 award). To determine the extent to which the Lab had documented certain oversight requirements for these awards, we reviewed award documentation contained in the 24 award files against key oversight requirements and best practices established by USAID and the Lab. These oversight requirements include: report committed funding amounts received from non-USAID sources; conduct valuations of in-kind contributions, as applicable; document partners met cost-share or matching funds, if required; and maintain copies of the award agreement and any modifications. To determine the extent to which the Lab’s information management system contained current data on non-USAID contributions, we reviewed committed funding data for the 24 selected awards in this system against documentation in the award files. We also reviewed the Lab’s guidance on accounting for non-USAID contributions in addition to meeting with Lab officials responsible for data input and oversight of such contributions. However, we did not independently assess the accuracy of the committed contributions against actual contribution amounts because the Lab does not collect data on actual contributions received in all of its awards. To determine the extent to which the Lab’s guidance on accounting for non-USAID contributions differs from USAID agency guidance, we compared guidance documents provided by the Lab with agency guidance from USAID’s ADS 303. Among other guidance documents, we reviewed the Lab’s Global Development Lab Internal Guide to Accounting for Leverage, and the Lab’s “Indicator Reference Sheet.” We also interviewed Lab officials responsible for implementing the Lab’s guidance for accounting for non-USAID contributions, as well as officials from USAID’s office of Policy, Planning, and Learning who are responsible for developing and updating ADS guidance on non-USAID contributions. We also reviewed the Lab’s public reporting of non-USAID contributions on USAID’s website. Performance Assessment and Results To report on the tools that the Lab uses to assess its performance, we reviewed and analyzed numerous Lab program and performance documents. These included the Lab’s strategic plan that covers fiscal years 2016 to 2020 and the Lab’s results framework that outlines the strategic objectives; Performance Management Plan; evaluation, research, and learning plan; Lab portfolio reviews; and Lab strategic learning reviews. To learn about the Lab’s performance management, program evaluation, and assessment process, we interviewed Lab officials from the Office of Evaluation and Impact Assessment and the Program and Strategic Planning office. We reviewed sections of USAID’s ADS 201 that pertain to strategic planning and implementation; project design and implementation; activity design and implementation; and monitoring, evaluation, and learning. We also spoke with officials in the Bureau for Policy, Planning, and Learning regarding the performance management requirements for bureaus outlined in ADS 201. To report on the results of the Lab’s performance indicators, we reviewed indicator data from the Lab for fiscal years 2014 to 2017. Since the Lab’s strategy was created in 2016, we focused our analysis on indicator data for fiscal years 2016 and 2017 that represent the Lab’s objectives as laid out in the Lab’s Results Framework. The Lab provided this information from DevResults, to include targets and measurements for each indicator by fiscal year. The data that we received from the Lab contained over 250 total indicators, which included those at the objective level, intermediate level, and sub-intermediate results level. We identified and analyzed 39 indicators representing the objective and intermediate results levels (for the science, technology, innovation, partnerships, and agency integration objectives) and looked at the targets and actuals for these for fiscal years 2016 and 2017. We compared each target value with the actual value to determine whether the Lab met, exceeded, or did not meet its targets for each indicator. If the target and the actual were the same value, we designated this as “meets.” If the target value was less than the actual value, we designated this as “exceeds.” Finally, if the target value was more than the actual value, we designated this as “does not meet.” We also identified indicators (both at the objective and intermediate results levels) where the Lab improved its performance from fiscal year 2016 to fiscal year 2017 as well as indicators where the Lab had declined in its performance from fiscal year 2016 to fiscal year 2017. To assess the reliability of the Lab’s performance data base, we interviewed Lab officials and reviewed documentation, and we determined that the data was sufficiently reliable for the purposes of comparing the Lab’s targets to reported results. However, it was beyond the scope of this engagement to assess the reliability of each of the 39 indicators. To report the results of the Lab’s seven external evaluations, we reviewed the completed external evaluations that were conducted in 2016 and 2017. As applicable, we looked at the purpose of those evaluations, findings, lessons learned, and any challenges to the program or project that the evaluation covered. We did not assess whether the Lab met its evaluation requirements under ADS 201, as this issue was outside of the scope of our review. We did not independently assess the methodology that was used in the evaluations. To report the results of the Lab’s portfolio reviews, we reviewed four portfolio reviews—two at midyear and two at the end of the year—that the Lab conducted in fiscal years 2016 and 2017. The portfolio reviews included sections on the Lab’s five objectives. As the portfolio reviews used different approaches to collect information, we analyzed them and identified headings in the documents that pointed towards results, including findings, challenges, achievements, and lessons learned and summarized this information. To report on the results of the strategic learning reviews, we reviewed the three strategic learning reviews—each a 2-page document—that the Lab had conducted in spring of 2018. We summarized each review and reported on each of the reviews’ questions and one of the “now what” actions from each review to provide an illustrative example. We conducted this performance audit from July 2017 to November 2018 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. Appendix II: Description of Key Global Development Lab Programs and Portfolios by Each Center The Global Development Lab’s (the Lab) five centers, its offices, and Lab- Wide Priorities manage more than 20 key programs and portfolios. The following are descriptions of key programs or portfolios implemented or managed by the Lab’s five centers—Development Research, Digital Development, Development Innovation, Transformational Partnerships, and Agency Integration. Center for Development Research Higher Education Solutions Network (HESN): According to Lab documentation, HESN is a partnership with seven universities working with partners worldwide. Leveraging nearly equal investments from each higher education institution, the universities established eight development labs that collaborate with a network of 685 partner institutions in academia, the private sector, civil society, and government across 69 countries. HESN’s development labs work with the U.S. Agency for International Development (USAID) to address problems faced by developing countries. Partnership for Enhanced Engagement in Research (PEER): According to Lab documentation, PEER supports competitively awarded grants for collaborative research projects led by developing country scientists and engineers who partner with American researchers. PEER-funded scientists conduct applied research that can inform public policy or new practices in development with a goal of creating and leading new innovations or generating evidence for how to scale innovations. PEER also builds research capacity by providing funds, tools, technical assistance, and research opportunities for local scientists and students. The program is implemented in partnership with the U.S. National Academy of Sciences. Science and Research Fellowship Programs: According to Lab documentation, the Lab supports three fellowship programs that are characterized by a commitment to the use of science, technology, innovation, and partnerships. The American Association for the Advancement of Science (AAAS) Science and Technology Policy Fellowship and the Jefferson Science Fellowship both bring scientists and technical experts to serve 1- to 2-year fellowships at the U.S. Agency for International Development, contributing their knowledge and analytical skills to development policy, research, and programming. Further, the Research and Innovation (RI) Fellowship program connects U.S. graduate student researchers research, or technical expertise, to address pressing development challenges. Research Policy Support: According to Lab documentation, the Lab provides advice to the agency on implementing the USAID Scientific Research Policy. This may include areas like peer review and open access to research products including data and USAID staff publications. Center for Digital Development Digital Inclusion: According to Lab documentation, the Lab helps improve connectivity by expanding access to the internet in countries where USAID works to help ensure that the most marginalized citizens have the skills and resources to be active participants in the digital economy. The team supports missions to integrate internet solutions into existing programs to ensure health clinics, schools, and other critical facilities are connected and offer access to modern internet services. Development Informatics (portfolio): According to Lab documentation, the Lab seeks to make development more adaptive, efficient, and responsive to citizens and decision makers by helping transform the use of data and technology throughout development. The Lab supports mission investments in technology platforms that can collect and analyze data more efficiently to improve strategic planning and program implementation. The Lab also leads the public advocacy campaign for the Principles for Digital Development, a set of agency best practices for applying digital technology and data in development. GeoCenter: According to Lab documentation, the Lab applies geographic and other data analysis to improve the strategic planning, design, implementation, monitoring, and evaluation of USAID’s programs. The GeoCenter works directly with USAID bureaus and missions to integrate geographic analysis, futures analysis (including scenario planning), and data analytics to inform development decisions. The team also leads a geospatial community of 50 geographic information systems specialists in field-based missions and in Washington, D.C. Digital Finance (portfolio): According to Lab documentation, the Lab’s Digital Financial Services team is working with USAID missions and bureaus through multi-stakeholder alliances and direct technical assistance to help the world’s financially excluded and underserved populations obtain access to and use financial services that meet their needs. The Digital Finance team has worked with over 30 missions and agency operating units to improve operational and programmatic efficiency as a means to accelerating development objectives within USAID projects and programs. Center for Development Innovation Development Innovation Ventures (DIV): According to Lab documentation, DIV is the agency’s venture capital-inspired, tiered, evidence-based funding model that invests comparatively small amounts in relatively unproven concepts, and continues to support only those that prove to work. It applies three core criteria to its application review process—evidence of impact, cost-effectiveness, and potential to scale. DIV accepts applications at three different funding stages from Proof of Concept ($25,000-150,000); Testing ($150,000–$1.5 million); and Transitioning to Scale ($1.5 million–$15 million). Grand Challenges for Development: According to Lab documentation, grand challenges call on the global community to discover, test, and accelerate innovative solutions around specific global challenges. The Lab is also leading efforts to apply innovation methods such as funding for challenges and prizes to accelerate innovation or incentivize action toward specific outcomes, such as the development of more efficient, lower-cost refrigeration solutions in the recently launched Off-Grid Refrigeration Competition. The Global Innovation Exchange: According to Lab documentation, this effort is an online platform to convene and connect innovators, funders, and experts working on development innovations around the world. The exchange is co-funded by USAID, the Australian Department of Foreign Affairs and Trade, the Korea International Cooperation Agency, and the Bill and Melinda Gates Foundation. Innovative Design (portfolio): According to Lab documentation, innovative design tools and approaches can help make a process more open and collaborative, incorporate human-centered design, or find a more innovative approach to solving a development problem. The Lab works to reframe development challenges, reach new audiences, and spur new ways of solving problems. It seeks to equip USAID teams with skills to design innovative programs using tools like design thinking and co- creation. It also builds diverse networks around critical systems challenges and facilitates a dialogue on the practice of innovation and design across USAID and the industry. Center for Transformational Partnerships Global Development Alliances (GDAs): According to Lab documentation, GDAs are partnerships between USAID and the private sector that use market-based solutions to advance broader development objectives. These partnerships combine the assets and experiences of the private sector to leverage capital, investments, creativity, and access to markets to work to solve the complex problems facing governments, businesses, and communities. GDAs leverage market-based solutions to advance broader development objectives. GDAs are co-designed, co-funded, and co-managed by all partners involved so that the risks, responsibilities, and rewards of partnership are shared. Partnering to Accelerate Entrepreneurship (PACE): According to Lab documentation, the Lab’s PACE initiative catalyzes private-sector investment into early-stage enterprises and helps entrepreneurs grow their businesses. Diaspora Engagement (portfolio): According to Lab documentation, the diaspora engagement is a core focus area for the Lab, which works with non-traditional partners in diaspora communities and organizations in under-addressed technical areas to test and incubate innovative partnership models. Center for Agency Integration Science, Technology, Innovation, and Partnerships (STIP) Agency Integration (portfolio): According to Lab documentation, the Lab supports the application of STIP across the agency by providing technical assistance, training, and catalytic investments in mission-driven STIP programs. In fiscal year 2016, the Lab worked closely with eight missions to integrate STIP tools and approaches to accelerate their development objectives. For example, the Lab is supporting ongoing efforts with the Uganda mission and a range of local partners, including the government of Uganda, to promote and source local, sustainable off-grid power solutions to impact a majority of underserved citizens. Digital Development for Feed the Future: According to Lab documentation, the Lab is collaborating with USAID’s Bureau for Food Security on integrating digital technologies into Feed the Future activities to accelerate reductions in global hunger, malnutrition, and poverty. An example includes facilitating greater precision agriculture through richer data collection, analysis, and packaging. Operational Innovation: According to Lab documentation, the Operations Innovations Team collaborates with partners to test and demonstrate viable disruptions which improve efficiency and effectiveness of Agency’s internal business processes, practices, and procedures. Appendix III: Description of 10 Grand Challenges for Development Since 2011, the U.S. Agency for International Development (USAID) and its partners have launched 10 Grand Challenges for Development. Grand Challenges for Development mobilize governments, companies, and foundations around important issues. According to USAID, through these programs, USAID and public and private partners bring in new voices to solve development problems through sourcing new solutions, testing new ideas, and scaling (expanding) what works. Table 6 includes a description of each of the Grand Challenges, identifies the founding partners, and lists the primary bureau within USAID responsible for the programs. According to Global Development Lab (the Lab) officials, the Lab manages Securing Water for Food and Scaling Off-Grid Energy Grand Challenges. Appendix IV: Global Development Lab Program Fund Allocation and Obligation Totals by Account, Fiscal Years 2014-2017 The Global Development Lab’s (the Lab) funding comes from different appropriations accounts. While the majority of the funding for fiscal years 2014 to 2017 is from the Development Assistance account, the Lab has also received lesser amounts of funding from four other accounts (see table 7). Appendix V: Global Development Lab Managed Activities in Fiscal Years 2014 through 2017 In fiscal years 2014 through 2017, the Global Development Lab (the Lab) managed a total of 339 activities addressing science, technology, innovation, and partnerships implemented by partners and obligated about $371 million for these awards. As figure 3 shows, the number of activities the Lab managed increased each year during this period, from 149 in fiscal year 2014 to 226 in fiscal year 2017. Obligated funding for all activities also increased annually until fiscal year 2017, when it declined by 27 percent. The Global Development Lab obligated funds to other activities it managed during this period that are not reflected in the data presented. These include obligations for institutional support contracts and staff fellowships. In fiscal years 2014 through 2017, four of the Lab’s centers managed a variety of activities addressing the Lab’s science, technology, innovation, and partnerships objectives. The Center for Development Research managed 28 activities addressing the Lab’s science objective. Obligations for these activities totaled about $120.4 million. The majority of this funding went to two programs, the Higher Education Solutions Network (about $81.2 million) and the Partnership for Enhanced Engagement in Research (about $27.7 million). The Center for Digital Development managed 17 activities addressing the Lab’s technology objective, ranging from providing geospatial satellite imagery to increasing the use of mobile money and e- payments in developing countries. Obligations for these activities totaled $64.5 million, with the majority of this funding going to Digital Finance activities. The Center for Development Innovation managed 205 activities addressing the Lab’s innovation objective. Obligations for these activities totaled about $115.4 million. This funding went to three programs: the Development Innovation Ventures program (about $57 million), the Innovation Acceleration program (about $19.3 million) and the Innovation Design program (about $39.2). The Lab’s Innovation Acceleration and Design program houses the Securing Water for Food Grand Challenge. The Center for Transformational Partnerships managed 37 activities addressing the Lab’s partnerships objective. Obligations for these activities totaled $39.8 million. For example, the Lab obligated about $13.9 million for the Partnering to Accelerate Entrepreneurship program, which aims to bring private-sector investment into businesses at early stages of development, among other things. In addition, other U.S. Agency for International Development (USAID) missions and bureaus have provided funding to Lab-managed projects through buy-ins. From fiscal years 2014 to 2017, USAID missions and bureaus provided funding to 55 Lab-managed projects, totaling $53 million. According to Lab officials, missions and bureaus can buy into projects in the development stage and can also buy into existing projects. For example, according to officials at USAID’s mission in Haiti, the Lab developed and funded a Higher Education Solutions Network project in Haiti, which provided the Haitian Ministry of Planning with capacity- building training to improve the collection of development and funding data for all donors in the country. Because the USAID mission saw the value of this project, it bought into the project, using its own funding, to allow the project to continue for an additional 2 years. Appendix VI: Global Development Lab Centers’ Direct Hires and Contractors, Fiscal Years 2015-2018 The Global Development Lab (the Lab) has numerous contractors who provide technical expertise in the centers and fill gaps when direct-hire staff are not available, according to Lab officials. In fiscal years 2016 to 2018, the Center for Digital Development had the most contractors of all the centers (see table 8). The contractors in this center are technical specialists mainly in the Lab’s GeoCenter, which uses geographic information systems to collect data to help aid development decisions in countries around the world. In fiscal year 2018, there were more contractors than direct-hire staff in the Center for Digital Development. Appendix VII: U.S. Agency for International Development Missions’ and Bureaus’ Views on the Global Development Lab Officials in the five U.S. Agency for International Development (USAID) bureaus and six missions we spoke with provided positive feedback on their interactions with the Global Development Lab (the Lab) but also identified some challenges. USAID officials identified numerous positive aspects or benefits of working with the Lab, such as the following: Lab staff brings diverse expertise and outside perspectives to the agency and provides technical assistance to projects that would not have been implemented otherwise. For example, some USAID officials mentioned that the Lab staff has insight into innovative approaches—whether procurement-related or project design and monitoring—and that the Lab has the ability to bring in contractors with specific technical expertise that the traditional development arena lacks. Lab staff is responsive and often willing to help with technical issues. Some USAID staff mentioned that Lab staff provide expertise and answer questions on an informal basis, sometimes covering areas where they are not the assigned point of contact with a particular bureau or mission. The Lab coordinates cross-cutting projects across the agency, such as the Grand Challenges for Development. Some bureau officials stated that Lab officials have been able to share their perspectives at training and other activities which has allowed them to be aware of what others across USAID are doing relevant to activities related to science, technology, innovation, and partnerships (STIP). The Lab funds projects and activities that missions and USAID headquarters operating units cannot afford. Some USAID officials mentioned that the Lab has sent staff out to provide STIP training, with the Lab covering the costs. However, some officials also mentioned that they have seen that recent budget cuts have had an impact on the Lab’s funding for more recent activities. The Lab holds trainings on topics such as procurement processes and private sector engagement that have helped missions and bureaus adopt new approaches to work and development partnerships. USAID officials also noted problematic aspects or challenges in working with the Lab, such as: Some Lab services can be cost prohibitive. For example, some mission officials mentioned that Lab resources are centralized in headquarters and therefore the cost to missions might be high and not affordable. Staff turnover at the Lab is frequent, making it difficult for bureau or mission officials to maintain relationships with the Lab. For example, some officials stated there has not been consistent contact with the Lab due to Lab staff frequently moving around or leaving. This has included changes in contacts for agreement officer representatives responsible for awards impacting the mission. The centers’ services and the ways in which bureaus or missions could work most effectively with the Labs are not always clear. For example, some mission and bureau officials mentioned that Lab staff does not always understand a country’s context when suggesting or deploying potential programs or activities related to STIP. This includes working to integrate STIP activities or innovations into the Country Development Cooperation Strategy when these might not be feasible for a country context or responsive to the needs of the mission. USAID officials noted that when they have provided feedback to the Lab, the Lab has generally been responsive. In addition, bureau officials mentioned that the Lab’s communications have improved. Appendix VIII: List and Description of Global Development Lab’s Performance Indicators, Fiscal Years 2016-2017 The Global Development Lab (the Lab) established its performance indicators when it created its strategy in fiscal year 2016 to cover fiscal years 2016-2020. The Lab’s results framework, which is reflected in the strategy, includes the Lab’s objective statements and intermediate results statement from which the Lab’s performance indicators flow. See table 9 for a description of indicators for the Lab’s five strategic objectives for fiscal years 2016 to 2017. Appendix IX: Comments from the U.S. Agency for International Development Appendix X: GAO Contacts and Staff Acknowledgments GAO Contact Staff Acknowledgments In addition to the contact named above, Leslie Holen (Assistant Director), Andrea Riba Miller (Analyst in Charge), Nick Jepson, and Kelly Friedman made key contributions to this report. Also contributing were Martin De Alteriis, Jeff Isaacs, Chris Keblitis, Reid Lowe, Aldo Salerno, and Nicole Willems.
target: The Lab was created as a USAID bureau in April 2014. The Lab was intended to institutionalize and improve USAID's ability to harness and leverage science, technology, innovation, and partnerships in addressing development issues and goals worldwide. The Lab supports projects and activities and announces, issues, and manages awards—or funding opportunities—for innovators to propose new ideas, approaches, and technologies. The Lab also incorporates external (i.e., non-USAID) contributions into its programming. Senate Report 114-290 included a provision for GAO to review the Lab. GAO's report examines, among other things, (1) the Lab's programs, funding, and staffing resources and (2) the extent to which the Lab has documented its oversight of awards with non-USAID contributions and clearly reported these contributions. GAO reviewed and analyzed agency documents and interviewed agency officials in Washington, D.C., and from six missions. GAO also analyzed selected Lab documentation for fiscal years 2014 through 2017. The U.S. Agency for International Development's (USAID) Global Development Lab (the Lab) has programs and activities for each of its five strategic objectives: science, technology, innovation, and partnerships (STIP) and agency integration of STIP. The Lab comprises five centers and two support offices (see figure). The centers house more than 25 Lab programs focused on issues such as development research, digital development, innovation ventures, and private sector engagement. The Lab's funding for its programs has generally been decreasing, as have its staffing numbers, since fiscal year 2015. USAID allocations of program funds to the Lab decreased from $170.7 million in fiscal year 2015 to $77 million in fiscal year 2017. Although the Lab has documented its oversight of awards that include non-USAID contributions, some data it collects for these contributions are outdated and its public reporting of such data lacks transparency. For awards GAO reviewed, the Lab consistently documented its compliance with key award oversight requirements. However, its Internal Guide to Accounting for Leverage (internal guide) does not include instructions for ensuring the data for these contributions are current. As a result, GAO found the Lab's management information system contained outdated data for non-USAID contributions in 10 of 24 awards GAO reviewed. The Lab publicly reports a broader range of non-USAID contributions than the types described in USAID policy. However, the Lab's internal guide does not require the Lab to disclose the types of contributions represented in its public reporting. As a result, the Lab's public reporting of such contributions lacks transparency. USAID policy and standards for internal control in the federal government require the use and communication of timely and reliable information. Revising the Lab's internal guide to include instructions for updating data on non-USAID contributions and requiring the Lab's public reporting to disclose the types of contributions represented would help the Lab ensure accuracy and transparency in the information it reports to Congress and the public.