text: Background OMB Guidelines In January 2018, OMB released (M-18-04) Monitoring and Evaluation Guidelines for Federal Departments and Agencies that Administer United States Foreign Assistance (the “Guidelines”) in response to the 2016 FATAA legislation. (See appendix III for additional information on the requirements in the legislation). The Guidelines provide direction to federal departments and agencies that administer foreign assistance on monitoring the use of resources, evaluating the outcomes and impacts of the foreign assistance projects and programs, and applying the findings and conclusions of such evaluations to proposed project and program design. The goals of the Guidelines are to set forth key principles to guide each agency and to specify requirements, where appropriate, that agencies must cover in their own policies on M&E of foreign assistance. The Guidelines define monitoring and evaluation as follows: Monitoring is the ongoing and systematic tracking of data and information relevant to policies, strategies, programs, projects, and/or activities and is used to determine whether desired results are occurring as expected during program, project, or activity implementation. Monitoring often relies on indicators, quantifiable measures of a characteristic or condition of people, institutions, systems, or processes that may change over time. Evaluation is the systematic collection and analysis of information about the characteristics and outcomes of the program, including projects conducted under such program, as a basis for making judgments and evaluations regarding the program; improving program effectiveness; and informing decisions about current and future programming. Table 1 lists OMB’s M&E requirements and key excerpts of the descriptions as noted in the OMB M-18-04. GAO Leading Practices In 2016, we reported on leading practices for foreign assistance program M&E. We identified 28 leading practices—14 for monitoring and 14 for evaluation. Table 2 lists and defines these monitoring practices. Table 3 lists the evaluation practice and corresponding definition. OMB’s Foreign Assistance Monitoring and Evaluation Guidelines Incorporate Most but Not All of GAO’s Leading Practices Based on our review, the Guidelines incorporate most of GAO’s leading practices for monitoring and evaluation. However, they do not incorporate practices on developing monitoring plans that are based on risks, ensuring that staff are appropriately qualified to conduct monitoring, establish procedures to close out programs, developing staff skills for evaluation, and following up on evaluation recommendations. OMB indicated that it intended the Guidelines to focus on elements required by the FATAA legislation. Nevertheless, incorporating these leading practices in the Guidelines can help ensure that all agencies address impediments, effectively manage foreign assistance, and meet their assistance goals. The Guidelines Incorporate Most of the GAO’s Leading Practices for Monitoring, but Do Not Include Risk Assessments, Staff Qualifications, or Close-Out Procedures Based on our review, OMB incorporates 11 of 14 GAO’s leading practices. Figure 1 shows our assessment of the Guidelines with regard to monitoring foreign assistance. The OMB Guidelines do not incorporate practices on developing monitoring plans that are based on risks, ensuring that staff are appropriately qualified to conduct monitoring, and establishing close-out procedures for projects and programs. Developing monitoring plans based on an assessment of risk. The Guidelines do not incorporate GAO’s leading practice of developing monitoring plans based on as assessment of risks related to achieving the defined objectives. Identifying and assessing risks can help agencies determine if impediments exist that they might need to mitigate in order to manage their foreign assistance more effectively. Additionally, determining which activities warrant greater oversight and which require less can also help agencies ensure the appropriate allocation of foreign assistance. Ensuring Staff qualifications for monitoring. The Guidelines do not incorporate GAO’s leading practice for agencies to ensure that staff members responsible for monitoring programs or projects have the relevant knowledge, skills, and training. By having qualified staff for monitoring programs or projects, agencies can help ensure they meet their foreign assistance goals. By hiring qualified staff and providing them the right training, tools, structure, incentives, and responsibilities, agencies can make operational success possible. Establishing close out procedures for projects and programs. The Guidelines do not incorporate GAO’s leading practice for agencies to establish program closeout procedures for all required work and administrative actions completed by the implementing partner. By establishing such procedures, agencies can help ensure their foreign assistance is less susceptible to fraud, waste, and mismanagement; addresses increases to potential costs in fees for maintaining foreign assistance; and increases their ability to redirect foreign assistance to other projects. The Guidelines Incorporate Most of the GAO’s Leading Practices for Evaluation, but Do Not Include Developing Staff Skills and Following Up on Recommendations Based on our review, OMB incorporates 12 of 14 GAO’s leading practices. Figure 2 shows our assessment of the Guidelines with regard to evaluating foreign assistance. The OMB Guidelines do not incorporate some practices such as developing staff skills for evaluation and following up on evaluation recommendations. Developing staff skills regarding evaluating. The Guidelines do not incorporate GAO’s leading practice for agencies to establish requirements that the staff responsible for overseeing and using evaluations should continually undertake the relevant education, training, or supervised practice needed to learn new concepts, techniques, and skills. By having their staff continually undertake such education, training, or supervised practice, agencies can benefit more fully from program evaluations. Following up on recommendations. The Guidelines do not incorporate GAO’s leading practice for agencies to determine whether management or programs have accepted the recommendations made in evaluation reports and taken the actions needed to address them. By developing mechanisms to track recommendations, agencies can better address inefficient, mismanaged, or costly programs or projects. OMB Notes the Guidelines for Monitoring and Evaluation Include Elements Required in the FATAA Legislation The FATAA requires the President to set forth guidelines “according to best practices of monitoring and evaluation” but does not define these best practices. Specifically, FATAA states, “the President shall set forth guidelines, according to best practices of monitoring and evaluation studies and analyses, for the establishment of measurable goals, performance metrics, and monitoring and evaluation plans that agencies can apply with reasonable consistency to covered United States foreign assistance.” OMB staff told us that the Guidelines were intended to focus on elements required by the FATAA legislation but noted that agencies are free to add additional requirements to their own M&E policies. However, we have previously reported that while some of these agencies have incorporated these leading practices, others have not. Furthermore, agencies that have incorporated these practices would not necessarily continue to include them if they are not required in the Guidelines. Regarding leading practices, officials noted that while these practices are important, there is no singular established standard for best monitoring practices. Nevertheless, both OMB’s circulars and recent legislation note the importance of leading practices for M&E. For example, Circular A-123 notes that management should identify internal and external risks that may prevent the organization from meeting its objectives. Additionally, the Foundations for Evidence-Based Policymaking Act of 2018 requires OPM, in consultation with the OMB, to identify skills and competencies needed for program evaluation, establish a new occupational series or update an existing one for program evaluation, and establish a new career path for program evaluation. Most Agencies Have Incorporated OMB’s Guideline Requirements in Their Policies, and All Have Taken Initial Steps to Implement Them Based on our review, most agencies incorporated all of OMB’s Guidelines for monitoring in their policies. However, DOD did not include the requirement to establish roles and responsibilities among agencies that participate in funding transfers or ensure that verifiable, reliable, and timely information is collected and available to monitoring personnel. We also found that agencies incorporate most of OMB’s Guideline requirements for evaluation in their policies, but some did not include the requirement to conduct impact evaluation on all pilot programs. Without incorporating these Guideline requirements, agencies risk losing accountability over their funding and monitoring and evaluating activities. They also risk replicating programs without fully understanding their effectiveness. We also found that all of the agencies we reviewed have taken initial steps to implement their M&E policies. Most Agencies Have Incorporated OMB’s Guideline Requirements for Monitoring Based on our review of agency monitoring policies, all the agencies except DOD incorporated relevant Guideline requirements. All six agencies we reviewed incorporated the requirement to establish monitoring policies that apply to their major foreign assistance programs. For example, State, USAID, and MCC have agency-wide policies for foreign assistance M&E. USDA and HHS have policies relevant to their major foreign assistance programs—for USDA, the Foreign Agriculture Service’s food aid programs, and for HHS, the President’s Emergency Plan for AIDS Relief (PEPFAR). All of the agencies with relevant monitoring policies—DOD, HHS, MCC, State, USAID, and USDA— incorporate the requirement to develop, collect, analyze, and report data on performance indicators. These policies help ensure the measurement of project implementation and progress, and promote the timely analysis and reporting of results that could identify any needed corrections. DOD did not incorporate Guideline requirements to establish agencies’ roles and responsibilities and ensure verifiable data for monitoring activities. Establishing agencies’ roles and responsibilities when funds are transferred. DOD did not include the Guideline requirement for agencies to establish roles and responsibilities in funding transfers. Without defined roles and responsibilities, agencies risk losing accountability over funding and monitoring activities. In addition, agencies could miss opportunities to collaborate and leverage interagency efforts to facilitate decision-making and address barriers across agency boundaries. Ensuring verifiable, reliable, and timely data. DOD did not include the Guideline requirement for agencies to ensure they collect and provide verifiable, reliable, and timely data to monitoring personnel. Without ensuring that such data are available to monitoring personnel, agencies risk employing inappropriate methods, continuing ineffective programs or projects, and making uninformed decisions. DOD officials told us these practices are currently not required because they are still in the process of fully aligning their policy with the Guidelines. Officials explained that working on prioritizing and directing resources towards M&E efforts has been a challenge. Officials noted they expect to update the policy to include these requirements in the future, but they have no specific timelines in place. Agencies Incorporate Most but Not All of OMB’s Guideline Requirements on Evaluation The agencies we reviewed incorporated nearly all relevant Guideline requirements on evaluation. Three of the six agencies—DOD, HHS and USDA—did not include a requirement to conduct impact evaluations on all pilot programs or projects. Figure 4 shows our assessment of agencies’ evaluation policies against the Guidelines. All the agencies we reviewed have established project-specific evaluation plans. For example, HHS implements PEPFAR’s evaluation plan which indicates specific requirements for describing the evaluation component, strategy, or intervention, the reason for the evaluation, the type of evaluation, the key evaluation questions, the data sources, the methods by question, and the dissemination and utilization plan. All the agencies we reviewed also had policies on distributing their evaluation reports internally and publicly reporting them. For example, State and USAID have a web-based, customized Evaluation Registry system that they jointly maintain for bureaus and independent offices to record and track planned, ongoing, and completed evaluations. Conduct impact evaluations for pilot programs or projects. DOD, HHS, and USDA did not include the Guideline requirement for agencies to conduct impact evaluations for pilot programs or to conduct only a performance evaluation and to provide a justification for not conducting an impact evaluation. Without a requirement to conduct impact evaluations of pilot programs, agencies risk duplicating or scaling up programs without fully understanding the factors that could lead to their success or failure. DOD. DOD officials told us they do not require this practice because they are still in the process of fully aligning their policy with the Guidelines. According to DOD, it has determined that impact evaluations are impractical and inappropriate for the planned evaluations; instead, it plans to conduct only performance evaluations and provide justifications for not conducting impact evaluations, as required by the OMB Guidelines. DOD plans to address the evaluation methodology of pilot programs in future updates, according to officials. However, DOD has no specific timelines in place for these updates. HHS. PEPFAR’s M&E documents indicate that PEPFAR teams are encouraged but not required to evaluate all current pilot programs to see which should be taken to scale for specific populations. Officials from HHS and the Office of the U.S. Global Aids Coordinator noted that they conduct their own evaluation of pilot programs and use routine program data to inform scaling of programs. However, PEPFAR policies do not specifically require that such evaluations be like the impact evaluations described in the Guidelines. USDA. FAS’s M&E documents indicate that when selecting projects to undergo impact evaluation the agency will consider pilot projects. USDA officials told us they have no requirement to conduct impact evaluations on all pilot projects because impact evaluations may be cost prohibitive and project lifecycles are short (i.e., 3 to 5 years). Officials further noted that implementing partners can conduct an impact evaluation on pilot programs, but are not required to do so. Although the Guideline requirement indicates that agencies can forgo impact evaluations, they must provide a justification in their M&E policy. USDA officials have not provided such a justification provided in their M&E policy. Establish agencies’ roles and responsibilities for evaluation activities when funds are transferred. DOD did not include the Guideline requirement for agencies to define roles and responsibilities when there are funding transfers between or among U.S. government agencies to ensure accountability for evaluation activities. Without defined roles and responsibilities, agencies risk losing accountability over funding and evaluation activities. In addition, they could miss opportunities to collaborate and leverage interagency efforts to facilitate decision-making and address barriers across agency boundaries. Evaluate all programs at least once whose dollar value equals or exceeds that of a median sized program within the agency. DOD did not include the Guideline requirement for agencies to evaluate all programs, at least once during their existence, whose dollar value equals or exceeds that of a median sized program in the agency. Without a mechanism to evaluate all these types of programs, agencies risk continuing inefficient, mismanaged, or costly projects. DOD officials told us they do not currently require these practices because they are still in the process of fully aligning their policy with the Guidelines. They noted that they expect to update the policy to include these requirements, but they have no specific timelines in place. Agencies Have Taken Initial Steps to Implement Their M&E Policies Since the six agencies we reviewed recently updated their M&E policies to align with the OMB Guidelines, many existing assistance projects and programs may not be governed by these requirements. Nonetheless, the agencies we reviewed have taken initial steps to help ensure implementation of agency M&E policies. In interviews, agencies provided us with the following examples of such steps. State. State developed a guidance document and tool-kit to operationalize and oversee its M&E policy to ensure it implements the Guidelines. According to State officials, they provide classroom training on the M&E policy and are piloting a revised online and classroom evaluation courses for staff. Officials also noted that they have dedicated staff to assist bureaus in implementing the Guidelines, among other agency policies. USAID. USAID has an approval process to ensure key deliverables include Activity plans that meet Guideline requirements. Additionally, USAID’s policy requirements indicate that each mission program office must identify a point of contact for monitoring and evaluation to ensure that USAID and its partners are complying with the agencies policies and foreign assistance M&E guidelines. MCC. MCC also has an approval process through their Department of Policy and Evaluation to ensure implementation of the Guidelines. As part of the process, the MCC Board of Directors or the appropriate partner country must approve initial M&E plans. HHS. Within HHS, the Centers for Disease Control and Prevention (CDC) are responsible for implementing the monitoring and evaluation guidance for their PEPFAR programs. CDC officials told us that they have existing mechanisms and supervisory structures in place to ensure that the Guidelines’ requirements are met in PEPFAR programs. USDA. USDA officials told us that the current M&E policy applies only to food assistance programs within FAS and not for other USDA programs. Officials explained they are trying to develop a structure that allows FAS to ensure all USDA components are implementing the OMB Guidelines. DOD. DOD developed guidance for fiscal year 2020 on implementing its M&E policy. DOD officials we spoke to noted they are working on identifying resources, skills, and capabilities to fully implement DOD’s M&E policy. Conclusions OMB’s Guidelines set forth key principles to guide agencies and to specify requirements, where appropriate, which they must cover in their own policies on M&E of foreign assistance. However, they do not include key leading practices for M&E that GAO identified for ensuring agencies meet their foreign assistance goals and objectives. While OMB allows agencies discretion to include these or other best practices, it is unknown if the agencies will do so. By ensuring that OMB’s government-wide Guidelines include these best practices, agencies can help address impediments, effectively manage foreign assistance, and meet their goals. Although all agencies we reviewed developed or updated their M&E policies to align with the Guidelines, not all of them include important requirements. DOD, HHS, and USDA did not include the requirement for agencies to conduct impact evaluations for pilot programs or to conduct performance evaluations and provide a justification for not doing an impact evaluation. Without a requirement to conduct impact evaluations of pilot programs, agencies risk duplicating or scaling up programs without fully understanding the causes that could lead to their success or failure. Recommendations for Executive Action We are making the following seven recommendations, including one to OMB, four to DOD, one to State, and one to USDA. The Director of the Office of Management of Budget should update the Guidelines to include GAO’s leading practices of developing monitoring plans that are based on risks, ensuring that monitoring staff have appropriate qualifications, establishing procedures to close-out programs, developing staff skills regarding evaluations, and establishing mechanisms for following up on evaluation recommendations. (Recommendation 1) The Secretary of Defense should update the Department’s monitoring and evaluation policies to define roles and responsibilities among agencies that participate in interagency funding transfers. (Recommendation 2) The Secretary of Defense should update the Department’s monitoring and evaluation policies to ensure verifiable, reliable, and timely data are available to monitoring personnel. (Recommendation 3) The Secretary of Defense should update the Department’s monitoring and evaluation policies to ensure that it evaluates all programs, at least once in their lifetimes, whose dollar value equals or exceeds that of the median program in the agency. (Recommendation 4) The Secretary of Defense should update the Department’s monitoring and evaluation policies to require the agency to conduct impact evaluations on all pilot programs before replicating or expanding, or conduct performance evaluations for those programs and provide a justification for not conducting an impact evaluation. (Recommendation 5) The Department of State’s U.S. Global AIDS Coordinator, in collaboration with HHS and other implementing agencies, should update the PEPFAR monitoring and evaluation policies to require these agencies to conduct impact evaluations on all pilot programs before replicating or expanding, or conduct performance evaluations for those programs and provide a justification for not conducting an impact evaluation. (Recommendation 6) The Secretary of Agriculture, in collaboration with the Foreign Agriculture Service, should update their monitoring and evaluation policies to require USDA to conduct impact evaluations on all pilot programs before replicating or expanding, or conduct performance evaluations for those programs and provide a justification for not conducting an impact evaluation. (Recommendation 7) Agency Comments and Our Evaluation We provided a draft of this product to the DOD, HHS, MCC, OMB, State, USDA, and USAID for comment. OMB commented on the draft report in an email from the staff responsible for economic policy, federal financial management, and international affairs. In the email, OMB disagreed with the recommendation to revise the Guidelines. It emphasized that an interagency group had developed the Guidelines and had consulted a number of expert sources on monitoring and evaluation policies and practices, including GAO’s leading practices. OMB also developed the guidelines to achieve the objectives contained in the Foreign Aid Transparency and Accountability Act of 2016 within the context of other existing OMB guidance. OMB suggested that it would be more effective to remind agencies that, in addition to the Guidelines specified in M-18-04, they should follow all guidance OMB had issued affecting monitoring and evaluation activities. This guidance includes policies for closeout procedures in the Uniform Guidance, for the Enterprise Risk Management and Internal Control in A-123, and for the Foundations for Evidence- Based Policymaking Act on using evaluation information and monitoring and evaluation staff skills and qualifications. We acknowledge that relevant monitoring and evaluation guidance is available to agencies in other forms beyond the Guidelines. However, we believe it is important for OMB to incorporate this guidance into its Guidelines, if only by reference, in order to emphasize the importance of these practices in the context of monitoring and evaluation of foreign assistance. This step would help ensure that OMB had integrated this guidance into the management of foreign assistance programs as appropriate. DOD concurred with our recommendations and indicated that it would address many of them in the next iteration of its M&E policy for security assistance (see appendix IV for written comments). DOD noted that two of our recommendations had limited applicability to DOD for security assistance, but described how it would implement them. First, DOD stated that it has not used its authority to transfer funds for security cooperation assistance to other departments and agencies. However, DOD indicated it would implement our recommendation to define roles and responsibilities among agencies that participate in interagency funding transfers, should such transfers become necessary. Second, DOD stated that conducting impact evaluations was not a feasible in the context of security assistance. Instead, DOD plans to conduct only performance evaluations, but it would provide justifications for not conducting impact evaluations, as required by the Guidelines. By documenting these approaches in its M&E policies, DOD would help ensure that those departments conducting M&E for DOD security assistance initiatives implement them as required. State agreed with the intent of the recommendation (see appendix V for written comments). State explained that impact evaluations are often not feasible in the context of assistance provided under PEPFAR and described its alternative approach to evaluating new initiatives. State indicated it would update appropriate PEPFAR policies to clarify when agencies should conduct impact and/or performance evaluations. These clarifications will reflect how State evaluates PEPFAR programs in practice in accordance with OMB guidance and legislation, according to State. USAID provided written comments (see appendix VI). HHS and USDA provided technical comments, which we incorporated as appropriate. MCC did not provide comments. We are sending copies of this report to the appropriate congressional committees, the Director of the Office of Management and Budget; Secretaries of Agriculture, Defense, Health and Human Services, and State; Administrator of the U.S. Agency for International Development; and the Executive Officer of the Millennium Challenge Corporation and other interested parties. In addition, this report will be available at no charge on the GAO website at http://www.gao.gov. If you or your staff have any questions about this report, please contact me at (202) 512-3149 or GootnickD@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made major contributions to this report are listed in appendix VI. Appendix I: Objectives, Scope, and Methodology This report examines the extent to which (1) the Office of Management and Budget’s (OMB) monitoring and evaluation (M&E) Guidelines incorporate GAO leading practices and (2) agencies incorporate the OMB Guidelines in their M&E policies and plans. To address objective one, we examined the OMB Guidelines against GAO’s 28 leading practices—14 for monitoring and 14 for evaluation— identified in GAO-16-861R. In 2016, GAO developed the 28 leading practices. In 2019, we provide specific definitions for each of the practices noted. We made slight modifications to the language to align with the definitions provided. For monitoring, we developed this list of leading practices based on our review of the GPRA Modernization Act of 2010; OMB’s Uniform Administrative Requirements, Cost Principles, and Audit Requirements for Federal Awards; GAO’s Standards for Internal Control in the Federal Government (Greenbook); and others. The list of leading practices for monitoring includes developing monitoring plans; collecting, reviewing, and analyzing monitoring data; and establishing roles and responsibilities of personnel responsible for monitoring. For evaluation, we developed a list of leading practices based on the American Evaluation Association’s (AEA) 2016 An Evaluation Roadmap for a More Effective Government (AEA Roadmap) and Preface to Evaluators’ Ethical Guiding Principles. The list of leading practices for evaluation include developing evaluation plans; ensuring evaluator independence; developing staff skills regarding evaluation and use of evidence; and establishing roles and responsibilities of personnel responsible for evaluation. To perform these analyses, two analysts assessed if the Guidelines incorporated specific GAO leading practices. The analysts worked iteratively, comparing notes and reconciling differences at each stage of the analysis. In addition, GAO staff independent of the two analysts reviewed the final analysis, and made modifications as appropriate. We also interviewed relevant OMB officials in Washington D.C. involved in developing the memorandum and inquired about specific requirements and plans to ensure the implementation of these Guidelines. To address our second objective, we examined U.S. agency M&E policies against the requirements noted in the OMB Guidelines. We identified the six major agencies administering the most foreign assistance funds. The six agencies are the U.S. Agency for International Development (USAID), the Department of State (State), the Millennium Challenge Corporation (MCC), the Department of Health and Human Services (HHS), the U.S. Department of Agriculture (USDA) and the Department of Defense (DOD). We asked these agencies to identify or provide all relevant policies and guidance relating to foreign assistance M&E, including, where appropriate, standard operating procedures or other guidance. For USDA, we reviewed the Foreign Agricultural Service’s food assistance; for HHS, the President’s Emergency Program for AIDS Relief; and for DOD, security cooperation programs. To perform these analyses, two analysts assessed agency M&E policy documents against the requirements in the OMB Guidelines. We identified requirements as phrases that included the following language “required,” “must,” “mandatory,” or “should.” The analysts worked iteratively, comparing notes and reconciling differences at each stage of the analysis. In addition, other GAO staff independent of the two analysts reviewed the final analysis, and made modifications as appropriate. We also interviewed relevant OMB staff and agency officials in Washington D.C. involved in developing and implementing the M&E policies and inquired about specific requirements, and plans to ensure their M&E policies are implemented. We conducted this performance audit from July 2018 to July 2019 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. Appendix II: The Office of Management and Budget Monitoring and Evaluation Guidelines (OMB Memorandum M-18-04) Appendix II: The Office of Management and Budget Monitoring and Evaluation Guidelines (OMB Memorandum M-18-04) In January 2018, the Office of Management and Budget (OMB) released (M-18-04) Monitoring and Evaluation Guidelines for Federal Departments and Agencies that Administer United States Foreign Assistance (the “Guidelines”) in response to the Foreign Aid Transparency and Accountability Act of 2016 (FATAA). Table 4 shows the complete description of the requirements noted in the Guidelines. Appendix III: Assessment of the Foreign Aid Transparency and Accountability Act of 2016 and the Office of Management and Budget Monitoring and Evaluation Guidelines The Foreign Aid Transparency and Accountability Act of 2016 (FATAA) has required objectives on monitoring and evaluation for the Office of Management and Budget (OMB) to include in the Guidelines. We compared the 13 required objectives for the Guidelines set forth in the FATAA legislation with those in the OMB Guidelines. We found that all of the monitoring and evaluation requirements set forth in the legislation are included in the OMB Guidelines. Table 5 shows the FATAA legislation requirements, OMB Guidelines, and our assessment of the alignment between the legislation and OMB’s Guidelines. Appendix IV: Comments from the Department of Defense Appendix V: Comments from the Department of State Appendix VI: Comments from the United States Agency of International Development Appendix VII: GAO Contact and Staff Acknowledgments GAO Contact Staff Acknowledgments In addition to the individual named above, James B. Michels (Assistant Director), Farahnaaz Khakoo-Mausel (Analyst-in-Charge), Paulina Maqueda-Escamilla, Mark Dowling, Martin De Alteriis, Benjamin Licht, John Hussey, Neil Doherty, Aldo Salerno, Carolina Morgan and Michael Simon made key contributions to this report. Related GAO Products Government Auditing Standards 2018 Revision (Supersedes GAO-12-331G. GAO-18-568G. Washington, D.C.: July 17, 2018. Foreign Assistance: Agencies Can Improve the Quality and Dissemination of Program Evaluations. GAO-17-316. Washington, D.C.: March 3, 2017. Foreign Assistance: Selected Agencies’ Monitoring and Evaluation Policies Generally Address Leading Practices. GAO-16-861R. Washington, D.C.: September 27, 2016. Program Evaluation: Some Agencies Reported that Networking, Hiring, and Involving Program Staff Help Build Capacity. GAO-15-25. Washington, D.C.: November 13, 2014. Government Efficiency and Effectiveness: Inconsistent Definitions and Information Limit the Usefulness of Federal Program Inventories. GAO-15-83. Washington, D.C.: October 31, 2014. Standards for Internal Control in the Federal Government. GAO-14-704G. Washington, D.C.: September 10, 2014. State Department: Implementation of Grants Policies Needs Better Oversight. GAO-14-635. Washington, D.C.: July 21, 2014. Program Evaluation: Strategies to Facilitate Agencies’ Use of Evaluation in Program Management and Policy Making. GAO-13-570. Washington, D.C.: June 26, 2013. President’s Emergency Plan for AIDS Relief: Agencies Can Enhance Evaluation Quality, Planning, and Dissemination. GAO-12-673. Washington, D.C.: May 31, 2012. Grants Management: Action Needed to Improve the Timeliness of Grant Closeouts by Federal Agencies. GAO-12-360. Washington, D.C.: April 16, 2012. Designing Evaluations: 2012 Revision. GAO-12-208G. Washington, D.C.: January 31, 2012. International School Feeding: USDA’s Oversight of the McGovern-Dole Food for Education Program Needs Improvement. GAO-11-544. Washington, D.C.: May 19, 2011. Program Evaluation: Experienced Agencies Follow a Similar Model for Prioritizing Research. GAO-11-176. Washington, D.C.: January 14, 2011. Managing for Results: Enhancing Agency Use of Performance Information for Management Decision Making. GAO-05-927. Washington, D.C.: September 9, 2005. Program Evaluation: An Evaluation Culture and Collaborative Partnerships Help Build Agency Capacity. GAO-03-454. Washington, D.C.: May 2, 2003. Managing For Results: Federal Managers’ Views Show Need for Ensuring Top Leadership Skills. GAO-01-127. Washington, D.C.: October 20, 2000. Performance Plans: Selected Approaches for Verification and Validation of Agency Performance Information. GAO/GGD-99-139. Washington, D.C.: July 30, 1999. Agency Performance Plans: Examples of Practices That Can Improve Usefulness to Decisionmakers. GAO/GGD/AIMD-99-69. Washington, D.C.: February 26, 1999. Executive Guide: Effectively Implementing the Government Performance and Results Act. GAO/GGD-96-118. Washington, D.C.: June 1, 1996.
target: The Trump Administration requested $28.5 billion in foreign assistance in fiscal year 2019, to be administered by at least 22 federal agencies. Almost 95 percent of this assistance is administered by six agencies—the Departments of Agriculture (USDA), Defense (DOD), State (State), Health and Human Services (HHS), the Millennium Challenge Corporation (MCC), and the U.S. Agency for International Development (USAID). FATAA required the President to set forth guidelines for M&E of U.S. foreign assistance. In January 2018, OMB issued the required guidelines for federal agencies. FATAA also contained a provision for GAO to analyze the guidelines established by OMB; and assess the implementation of the guidelines by the agencies. In this report, GAO examined the extent to which (1) OMB's M&E Guidelines incorporate GAO leading practices, and (2) agencies incorporate the OMB Guidelines in their M&E policies and plans. GAO assessed the OMB Guidelines against GAO's 28 leading practices identified in GAO-16-861R . GAO also assessed the six agencies' foreign assistance M&E policies against the Guidelines and interviewed OMB and relevant agency officials in Washington, DC. The Office of Management and Budget's (OMB) foreign assistance Guidelines incorporate most of GAO's leading practices for monitoring and evaluation (M&E), but gaps exist (see figure). Monitoring : The Guidelines define monitoring as the continuous tracking of program or project data to determine whether desired results are as expected during implementation. The Guidelines do not require GAO's leading practices on risk assessments, staff qualifications, and program close-out procedures. Evaluation : The Guidelines define evaluation as the systematic collection and analysis of program or project outcomes for making judgments and informing decisions. They do not require GAO's leading practices on developing staff skills and following up on recommendations. OMB officials indicated the Guidelines are focused on elements required in the Foreign Aid Transparency and Accountability Act of 2016 (FATAA), but noted that agencies can add additional requirements to their own M&E policies. FATAA requires the President to set forth guidelines “according to best practices of monitoring and evaluation.” OMB staff acknowledged that GAO's leading practices are important, but stated that there is no singular established standard for best monitoring practices. Nevertheless, all of GAO's leading practices can help agencies address impediments, effectively manage foreign assistance, and meet their goals. When assessing agencies' M&E policies against OMB Guidelines, GAO found that agencies incorporated most of the requirements. However, for monitoring, one of the six agencies GAO reviewed—DOD—did not include the requirements to establish agencies' roles and responsibilities and ensure verifiable data for monitoring activities. For evaluation, agencies required most Guideline requirements, but not all. For example, DOD, HHS, and USDA did not require conducting impact evaluations for pilot programs or projects. Without a clear requirement to do such evaluations, agencies risk duplicating or scaling up programs without fully understanding the factors that could lead to their success or failure. Agencies GAO reviewed have plans or mechanisms in place to oversee the implementation of their M&E policies. For example, State developed a guidance document to operationalize and oversee its M&E policy to ensure the implementation of the Guidelines.