U.S. Assistance to Mexico: State Department Could Improve Its Monitoring of Mérida Initiative Projects
Released: 2020-05-12T00:00:00

Letter

Background

Mérida Initiative Projects

There were 445 State/INL and USAID Mérida Initiative projects active from fiscal year 2014 through fiscal year 2018, which includes some projects that started before this period and some that continued after this period. State/INL funded 388 of the projects, and USAID funded 57. USAID’s projects tended to be larger with higher funding amounts than State/INL projects. State/INL projects generally focused on providing training and assistance to Mexican officials from the justice sector, border security, military, and law enforcement, as well as equipment, including for forensic laboratories, drug detection, and border surveillance. USAID projects were intended to engage with Mexican government institutions, civil society organizations, and the private sector to address corruption, promote trust in government, or prevent crime and violence, such as through skill building for youth, efforts to advance human rights, or technical support for judicial system development. State/INL allocated about $542 million and USAID allocated about $182 million for assistance to Mexico under the Mérida Initiative from fiscal year 2014 through fiscal year 2018.

Mérida Initiative Oversight

State/INL and USAID are the lead U.S. agencies for developing the Mérida Initiative’s programming. In these roles, State/INL and USAID work with Government of Mexico officials to outline plans, goals, and objectives for Mérida Initiative projects. State/INL and USAID both manage and fund the Mérida Initiative with the support of a wide range of project implementers, including DOJ, DHS, and DOD, as well as private contractors, nongovernmental organizations, and international organizations. State/INL and USAID implement Mérida Initiative projects primarily through contracts, grants, and agreements with international organizations. State/INL also implements some Mérida Initiative projects through interagency agreements with other U.S. agencies (e.g., DOJ, DHS, and DOD).

State/INL and USAID contracting, grant, and agreement officers, are responsible for administering and overseeing contracts, grants, and other agreements that the agencies award, including for Mérida Initiative projects. They delegate the day-to-day monitoring responsibilities to agency officials located in Mexico City, particularly State/INL and USAID Contracting Officer Representatives (COR) for contracts, State/INL Grant Officer Representatives (GOR) for grants, State/INL Agreement Officer Representatives (AOR) for interagency agreements or letters of agreement with international organizations, and USAID AORs for grants and cooperative agreements, according to agency officials. Key monitoring responsibilities of the CORs, GORs, and AORs typically include reviewing quarterly, annual, and other progress reports submitted by project implementers; ensuring other required documents are submitted; communicating with the implementers on the status of assistance activities; and conducting site visits, among other things.

Key Practices for Monitoring Foreign Assistance Projects

In 2019, we reported on 14 leading practices for monitoring foreign assistance that agencies should incorporate in their monitoring policies to help ensure that they effectively manage foreign assistance, address impediments, and meet their assistance goals. From these leading practices we derived eight key practices that can help agencies monitor the implementation and performance at the project level. To facilitate discussing these key monitoring practices, we grouped them into three areas: (1) assigning monitoring duties to qualified staff, (2) planning monitoring approach, and (3) monitoring project implementation. (See table 1.) These practices are generally consistent with the Office of Management and Budget’s guidelines for Federal departments and agencies that administer United States foreign assistance and related guidance, as well as State’s and USAID’s monitoring policies.

For Projects We Reviewed, State Generally Followed Key Monitoring Practices About Half of the Time, but Did Not Consistently Track Performance Measures

We reviewed 15 of State/INL’s high–dollar value Mérida Initiative projects to assess the extent to which State/INL followed key practices for monitoring foreign assistance projects in the areas of assigning monitoring duties to qualified staff, planning a monitoring approach, and monitoring project implementation. For these projects, the agency generally followed the key practices about half of the time, as shown in figure 1, and for a subset of four selected projects, it did not consistently track performance data or compare them to established performance measures. State/INL does not have procedures in place for monitoring staff to consistently follow all the key practices. Instead, officials said they focused on tracking implementation of the projects’ activities. Consistently following key monitoring practices would allow State/INL to stay well informed of projects performance, take corrective action when necessary, and help ensure that projects achieve their intended results.

For Projects We Reviewed, State Generally Followed Key Monitoring Practices about Half of the Time

Assigning Monitoring Duties to Qualified Staff

State/INL generally followed key practices for assigning monitoring duties to qualified staff almost always. Assigning staff with the appropriate certification helps ensure that they have the necessary knowledge and skills to perform those duties. Establishing roles and responsibilities helps ensure that the assigned monitoring staff are aware of their monitoring duties. State/INL requires that staff responsible for monitoring Mérida Initiative projects be certified as a COR, GOR, or AOR. State/INL also assigns roles and responsibilities to monitoring staff through a designation letter in which a contract or grant officer designates a COR, GOR, or AOR to oversee each project. However, of the 15 projects we reviewed, one had a gap in the documentation for staff certifications, and four had gaps in the documentation of designation letters. For example, in one case State/INL could not provide documentation to demonstrate that the official responsible for monitoring a project on police training had been officially designated or that the official had a valid certification during the full implementation period of the project. According to State/INL staff, the monitoring staff roles and responsibilities are also outlined in other documents such as the State Department’s Foreign Affairs Manual and the AOR Handbook, of which staff are expected to be aware. Figure 2 illustrates the extent to which State/INL followed each related key practice for assigning monitoring duties.

Planning Monitoring Approach

State/INL generally followed key practices for planning a monitoring approach a third of the time. Two projects—one for helicopter pilot training and the other for aviation maintenance training—did not have monitoring plans and thus did not meet any of the three key practices for planning a monitoring approach. According to a State/INL manager, State/INL is no longer working with this implementer due to long-standing difficulties in obtaining documentation needed to monitor the projects. Most of the other 13 projects partially met the key practices for planning a monitoring approach. For example, goals and objectives were included in planning documents other than the monitoring plan. Furthermore, while only three of the projects had a monitoring plan that addressed risk, we determined that 10 of the projects partially addressed this key practice, because risks were assessed or considered, but the identified risks were not addressed in the monitoring plan. In addition, almost all of the projects had relevant project-level performance measures. Developing a monitoring plan that identifies project objectives helps focus monitoring efforts on assessing projects outcomes. In addition, identifying and addressing risks in that plan helps focus monitoring efforts on those aspects of project implementation that are most likely to threaten the success of the project in meeting its goals. We did not see evidence that State/INL had procedures in place to ensure that monitoring officials consistently follow key practices in the area of planning monitoring approach. Figure 3 illustrates the extent to which State/INL followed each related key practice to planning a monitoring approach.

Monitoring Project Implementation

State/INL provided documentation to demonstrate that monitoring managers generally followed key practices for monitoring project implementation about half of the time. Monitoring project implementation helps ensure that projects are meeting their objectives, so that any necessary adjustments or corrective actions can be taken in a timely manner. We found that State/INL did not generally collect all expected progress reports from implementers for seven projects, and of those seven, it did not collect any reports for three projects. Furthermore, State/INL did not provide documentation for eight projects demonstrating that monitoring staff had generally assessed and approved implementers’ periodic progress reports. We also found that for seven projects, State/INL did not provide documentation demonstrating that monitoring staff had generally conducted site or field monitoring visits or taken other steps to validate the partner’s performance implementing the project. For example, for one project that provided training to Mexican immigration officers on the southern border, State/INL only provided one quarterly progress report of the four we requested for the period of our review. For this project, State/INL also did not provide documentation that monitoring staff had taken steps to review and approve the report or that they had conducted any monitoring site visits. A State/INL official explained that they requested the quarterly reports, but at times implementers did not submit them. Without implementing procedures to consistently collect, assess, and approve performance reports from implementers, monitoring staff may not have sufficient information to assess implementers’ performance and determine whether corrective actions are needed. We did not see evidence that State/INL had procedures in place to ensure that monitoring officials consistently follow key practices in the area of monitoring project implementation. Figure 4 illustrates the extent to which State/INL followed each related key practice for monitoring project implementation.

State/INL Did Not Consistently Track Performance Data against Established Measures for Projects We Reviewed

State/INL monitoring officials did not consistently track performance data against established measures for four Mérida Initiative projects we reviewed; these four projects were a subset of the 15 State/INL projects discussed above. Tracking performance data—a key practice for monitoring project implementation—can provide meaningful information on projects’ progress in achieving intended results. The four projects we reviewed included two grants focused on police professionalization; one interagency agreement focused on assistance to forensic laboratories; and one agreement with an international organization focused on conducting a survey on police standards, training, and professionalization. We reviewed how State/INL tracked performance data for these selected projects as part of its efforts to assess and approve implementing partners’ periodic performance reports and data as outlined in the key monitoring practices. Specifically, we analyzed the extent to which State/INL tracked data contained in quarterly progress reports and compared these data to established performance measures. State/INL and the project implementers outlined these performance measures in monitoring documents that these implementers developed and State/INL approved. Some of these projects’ monitoring documents also included data sources, data collection frequency, and performance targets. State/INL did not track performance data for two of the four selected projects and tracked such data inconsistently for the other two selected projects. As a result, State/INL cannot ensure that it has accurate and reliable performance data for its Mérida Initiative projects. Such data could help State/INL determine whether projects are achieving intended results and take necessary corrective actions to improve project performance over time.

State/INL Did Not Track Performance Measures for Two of the Four State/INL Projects We Reviewed

For the two police professionalization projects we reviewed, State/INL did not track performance data against established performance measures outlined in the project narrative at the start of the projects. Some of these projects’ performance measures reflected outputs—such as the number of participants completing at least 25 hours of police training and the number of citizen surveys conducted on public trust of law enforcement. Other performance measures reflected outcomes—such as the percentage of law enforcement officials who feel ready for promotion after completing training and results of citizen surveys on perceived security where law enforcement trainings were conducted. (See examples in table 2.) However, State/INL did not clearly track or reference such performance measures in these two projects’ quarterly progress reports. Instead, State/INL provided details in these reports on project activities and training that did not clearly link to the projects’ performance measures. For example, State/INL noted the number of participants who took a specific training course on a certain date, but did not provide the total number of participants’ training hours to compare them to the performance measure on the total number of participants who completed at least 25 hours of training. State/INL monitoring officials said they had not systematically tracked data on the performance measures of these projects over time, but instead focused on ensuring the trainings were conducted and the number of training participants were tracked. These officials acknowledged the need to improve their tracking of these projects’ progress against their performance measures.

We also identified information in quarterly progress reports for two projects suggesting that the reports did not accurately reflect project activities in those quarters. For example, for one project, State/INL included identical information in two separate quarterly reports even though the implementer conducted different project activities in those two quarters. Thus, at a minimum, the information in one of the quarterly reports did not accurately reflect the project’s activities conducted in that quarter. We found the same issue with another project’s reports. State/INL officials said they were not aware that the project information in these reports were identical.

State/INL Tracked Some Performance Measures for Two of the Four State/INL Projects We Reviewed, but Did So Inconsistently

For the two other State/INL projects we reviewed (one forensics laboratory accreditation project and one police survey project), State/INL tracked some performance data but did so inconsistently. These projects’ performance measures reflected outputs, such as the number of survey pollsters hired and trained and the number of accredited forensic laboratories that maintain their accreditation. Other performance measures reflected outcomes, such as the percentage of forensic laboratories trainees reporting improved knowledge of subject matter and satisfaction rates for training courses for the forensics laboratory project. (See examples in table 3.) In one of these two projects’ quarterly reports, the project implementers inconsistently described and numbered some of the performance measures, and they did not explain the discrepancies. Also, the implementers mentioned different performance measures in different quarterly progress reports—with some measures dropping off in some quarters and new measures appearing in others—without providing a rationale in the reports. As a result, State/INL could not consistently track progress of some of the performance measures over time. State/INL officials stated that these two implementers only included activities in the quarterly reports that they conducted in that quarter, which would result in different and inconsistent performance measures in each report.

In addition, some of the reported project activities did not consistently and clearly align with the performance measures to allow State/INL to track the project’s progress against these measures. For example, some performance measures reflected percentages (e.g., 90 percent of authorities responsible for forensic laboratories routinely attend regional and national conferences), but the report listed the names of conference participants, dates, and locations in a table next to that performance measure. When asked about these discrepancies, State/ INL officials said that they did not ensure that implementers provided complete information to clearly track the project’s progress against performance measures. However, they said that they also conduct monitoring through informal methods not documented in the progress reports, such as through communication via phone calls and emails with the implementers. Such informal methods do not provide State/INL with the necessary data to assess a project’s performance against its goals.

State/INL Monitoring Management Did Not Ensure Their Staff Tracked Performance Measures

For the four State/INL projects we reviewed, State/INL monitoring managers did not establish procedures to collect and review project performance data, such as the number of people who completed a targeted number of hours of training, or the results of training surveys. These managers said they did not prioritize establishing performance tracking procedures and instead focused on the implementation of the projects’ activities, such as counting the number of participants who attended one training course for a particular month. For example, while some monitoring staff sent monthly emails to their managers describing project activities, State/INL monitoring managers did not establish procedures—such as holding regular meetings with or requiring reporting from monitoring staff—that focused on tracking the projects’ progress against established performance measures.

State/INL Receives Activity Data from Implementers to Monitor Project Implementation

State/INL receives activity data from project implementers that it considers useful in helping the agency monitor the projects’ implementation and activities. State/INL officials told us that project activity data in the quarterly progress reports—such as when trainings were conducted and how many people attended—help keep them informed of and monitor the projects’ implementation.

In addition, since 2015, State/INL Mexico has collected detailed data and information in tracking databases on (1) training events and related surveys on that training, and (2) forensic laboratory accreditations and correctional facility accreditations.

The training tracking database contains data on over 6,000 training events, 100,000 trainee records, and over 20,000 survey responses from training event participants. This database can generate numerous reports covering the number of people who completed a specific trained course, which training courses a specific person completed, training survey results, and which implementer conducted the training, among other information.

State/INL databases also collect information on the status of forensics laboratories and correctional facilities across Mexico that are being accredited through Mérida Initiative projects. The forensics database includes pages for each laboratory with detailed information about the level of accreditation received, and types of trainings conducted, among other things. The correctional facilities database is structured similarly to the laboratories database with pages for each facility with detailed information on accreditation status and timeline, among other things. According to State/INL officials, like the training tracking system, the forensics and correctional facilities databases can generate reports, such as monthly progress reports.

Finally, State/INL Mexico is implementing a new cloud-based monitoring database—called DevResults—that will consolidate and track data on activity, output, and outcome indicators for all Mérida Initiative projects. According to State/INL officials, they implemented DevResults so that State/INL could track a project’s progress and trends in real time against its performance goals. According to State/INL officials, DevResults included data for 84 projects as of February 2020. They also noted that agency officials and implementers have completed training on DevResults, and additional training will be provided as needed. State/INL officials said they plan to continue adding data for past and present Mérida Initiative projects in 2020.

For Projects We Reviewed, USAID Almost Always Followed Key Monitoring Practices and Tracked Performance Data, but Did Not Develop Monitoring Plans That Address Risk

We reviewed five of USAID’s Mérida Initiative projects to assess the extent to which USAID followed key monitoring practices in the areas of assigning monitoring duties to qualified staff, planning a monitoring approach, and monitoring project implementation. For these projects, USAID almost always followed key practices—as shown in figure 5—and for a subset of two selected projects, it consistently tracked project performance. According to USAID officials, USAID management conducted periodic portfolio reviews to ensure that monitoring staff adequately monitored Mérida Initiative projects and followed key practices. However, for all five USAID projects we reviewed, monitoring plans did not address identified risks, which could help the agency allocate monitoring resources to those aspects of the projects that warrant closer scrutiny.

For Projects We Reviewed, USAID Almost Always Followed Key Monitoring Practices

Assigning Monitoring Duties to Qualified Staff

USAID generally established roles and responsibilities for technical staff responsible for monitoring projects, but for two of the five projects we reviewed it did not maintain documentation showing that it assigned staff with appropriate certifications. Like State/INL, USAID requires that staff responsible for monitoring Mérida Initiative projects be certified as CORs or AORs, which typically includes periodic training in monitoring projects. USAID assigns roles and responsibilities to these staff through a designation letter in which a contract or agreement officer designates a COR or AOR, respectively, to conduct technical oversight of each project.

For the five projects we reviewed, USAID properly designated monitoring roles and responsibilities to technical staff, however there were gaps in staff certification documentation for technical staff for two projects. For example, we found that the person responsible for monitoring a project promoting justice reform and rule of law in Mexico did not have a valid certificate for 9 months of the project’s 4-year period of performance. Maintaining complete documentation of monitoring-related activities helps USAID management ensure adequate, continuous monitoring of projects. According to USAID, the gaps in documentation were caused by staff turnover and trouble accessing the government-wide system for recording the certification of staff, which was difficult to access or down from December 2017 to March 2018. Officials said that once the system to record certificates was brought back online, they were able to track certifications. Figure 6 illustrates the extent to which USAID followed each related key practice for assigning monitoring duties.

Planning Monitoring Approach

USAID generally developed monitoring plans that included program goals and objectives and project-level performance measures, but the monitoring plans did not address project risks. All five projects generally had a monitoring plan that identified project goals and objectives, and relevant project-level performance measures. However, none of the monitoring plans generally addressed identified risks related to achieving project objectives. While USAID provided documentation showing that the agency had conducted various assessments considering risk for each project, the results of these assessments were not addressed in the projects’ monitoring plans. For example, for a project to promote justice and rule of law in Mexico, USAID assessed risks relating to terrorism, environmental effects, sustainability, and gender equity in carrying out the project. However, the project’s monitoring plan did not address identified risk levels and related monitoring actions designed to mitigate risks identified in these assessments. USAID explained that they address ongoing monitoring of risk through several other processes, such as project design, procurement actions, financial management, award management and administration, semi-annual project portfolio reviews, and annual risk-based assessments of the USAID’s portfolio in Mexico, among others. However, identifying and addressing risks in the monitoring plan can help ensure that monitoring staff are aware of potential impediments to project success about which they need to be vigilant or take steps to mitigate as they monitor the projects. Additionally, determining which activities warrant greater oversight can also help agencies manage monitoring resources cost effectively. Figure 7 illustrates the extent to which USAID followed each related key practice for planning a monitoring approach.

Monitoring Project Implementation

USAID generally followed key practices for monitoring project implementation about two-thirds of the time. We found that USAID collected all progress reports for four of the five projects we reviewed. For two projects, USAID did not provide documentation demonstrating that monitoring staff had generally assessed and approved implementers’ periodic progress reports. For all five projects, USAID provided documentation demonstrating that monitoring staff had generally validated implementing partners’ performance through site visits. Figure 8 illustrates the extent to which USAID followed each related key practice for monitoring project implementation.

USAID Consistently Tracked Established Performance Measures for the Two Projects We Reviewed

USAID monitoring officials consistently tracked performance data and compared them to established performance measures for the two projects we reviewed; these two projects were a subset of the five USAID projects discussed above. To review the extent to which USAID assessed and approved implementing partners’ periodic reports and data—one of the eight key monitoring practices—we determined whether USAID tracked performance data contained in quarterly or annual progress reports. USAID funds one of the two projects through a cooperative agreement focused on strengthening human rights, and the other project through a contract focused on improving the criminal justice sector.

USAID and project implementers outlined these projects’ performance measures in project-specific monitoring plans that both parties developed at the start of the project or revised after the project was in place. Project implementers developed these plans, and USAID approved them. The plans included details related to the performance measures, such as data sources, data collection frequency, and targets. In accordance with these plans, USAID and project implementers tracked performance measures in annual progress reports, while they primarily tracked detailed project activity in quarterly progress reports.

The two USAID projects’ progress reports included tables that tracked project performance. Some of the projects’ performance measures reflected outcomes, such as prosecution rates of Mexican government prosecution units that received technical support and the number of improved measures to address serious human rights violations. Some performance measures reflected outputs, such as the number of Mexican officials trained in human rights advocacy areas. See table 4 for examples of performance measures and information in the progress reports we reviewed.

When the implementer and USAID changed performance measures, they also revised project-specific monitoring plans to document these changes. For example, for one project we reviewed, the established measures were no longer effective in measuring progress toward the project’s objectives, according to USAID officials. As a result, the implementer and USAID modified the project’s monitoring plan at least twice, revising the performance measures to better align with the project’s objectives. The subsequent progress reports we reviewed for these projects included data on the revised performance measures.

USAID has procedures to help ensure that monitoring staff track performance data. According to USAID officials, USAID began sending out a standard spreadsheet to all Mérida Initiative implementing partners in 2018 that requires them to report performance data on a quarterly or annual basis. USAID uses these spreadsheets to track Mérida Initiative project performance data. Since May 2017, USAID has also conducted 6- month portfolio reviews in which monitoring managers and their staff review project activities and performance data collected for their projects and discuss project successes and challenges. USAID managers told us that they implemented these reviews to help ensure that their staff monitor project performance.

Mexico Shares Indicator Data with State/INL for Monitoring the Efforts Related to the Mérida Initiative

According to State/INL, the Government of Mexico provides data to State/INL that help the agency monitor its Mérida Initiative assistance efforts and provides insights into the implementation of the initiative overall. State/INL also noted that, in 2014, the agency hired a contractor to work with both the U.S. and Mexican governments to develop a comprehensive set of indicators to evaluate the progress and results of the Mérida Initiative. In 2015, Mexico agreed that it would provide data to State/INL on this set of indicators to demonstrate the effects of the Mérida Initiative, according to State/INL officials. These officials told us that they try to obtain the data on an annual basis. They also noted that the purpose of collecting the data from Mexico was to establish a mechanism to share information on the Mérida Initiative’s effects and to improve U.S.- Mexico cooperation on the initiative. According to State/INL officials, various Mexican agencies collect the data, such as the Army, Air Force, Navy, Tax Administration Service/Customs, Attorney General’s Office, and National Institute of Statistics and Geography.

The Mexico data comprise about 170 indicators (data points) related to the overall goals and program areas of the Mérida Initiative: Counternarcotics/Special Investigations, Criminal Prosecutions, Border Security and Ports of Entry, and Security and Law Enforcement. Some data are closely linked to Mérida Initiative–funded projects, such as the accreditation status of Mexican correctional facilities. Other data provide broader context, such as Mexican civil society’s perception of Mexican agencies. In addition, data, such as the number of accredited forensic laboratories and correctional facilities, may reflect progress in institution building. Other data, such as the number of accounts blocked by the Mexican Financial Intelligence Unit, may reflect operational capacity development. See table 5 below for examples of the indicators, as reported by Mexico to State/INL.

State/INL officials said they use the indicator data in discussions with Mexican officials to help monitor the implementation and activities of the Mérida Initiative, including which best practices can be replicated across Mexico. State/INL officials said the data also inform the agency’s internal decision making on which Mérida Initiative programs are effective and which programs it should modify. For example, according to State/INL officials, the indicator data help track the use of equipment donated to Mexico through the Mérida Initiative. If the data show extensive use of equipment, State/INL can use the data to justify a request for additional equipment or to approve maintenance of the equipment, according to agency officials.

Conclusions

For over a decade, the Mérida Initiative has funded programs intended to address serious challenges to security and the rule of law. As the United States continues to support hundreds of Mérida Initiative projects in Mexico, it is important that State/INL monitor these projects carefully and stay well informed of the projects’ performance to ensure that they are as effective as possible. USAID has established procedures that help ensure that it follows most key monitoring practices, including those related to assigning monitoring duties to qualified staff and monitoring project implementation. State/INL management has not established such procedures for the projects we reviewed, limiting its ability to stay well informed of project performance and make course corrections to improve performance when necessary. While State/INL and USAID often conducted assessments to identify risks that may affect the achievement of project objectives, they generally did not address the results of the risk assessments in projects’ monitoring plans. Developing monitoring plans to address risks would help establish the appropriate level of oversight needed for each project, which in turn could lead to more cost-effective management of these projects.

Recommendations for Executive Action

We are making the following two recommendations, one to State and one to USAID: The Secretary of State should ensure that State/INL establishes procedures that verify that monitoring officials for Mérida Initiative projects follow the key practices. (Recommendation 1)

The USAID Administrator should establish procedures to ensure that monitoring officials for Mérida Initiative projects develop monitoring plans that address risks. (Recommendation 2)

Agency Comments

We provided a draft of this report to State, DOD, DHS, DOJ, and USAID for review and comment. Some of the agencies provided technical comments, which we incorporated as appropriate. State and USAID also provided formal comments, which are reproduced in appendixes III and IV.

State agreed with our recommendation to establish procedures for staff monitoring Mérida Initiative projects to follow key practices. State indicated that it is working to create new monitoring and evaluation guidance consolidated across State/INL, based in part on GAO’s leading practices. According to State, the new guidance will address the areas highlighted in this report related to monitoring Mérida Initiative projects. State/INL plans to institute annual program reviews in which monitoring staff will assess project performance, effects, and alignment with current and planned priorities. State indicated that annually reviewing State/INL programming will help identify underperforming projects, give relevant staff a forum to discuss any issues or challenges to implementation and monitoring, and ensure the bureau follows the key monitoring practices outlined in this report.

USAID also agreed with our recommendation to establish procedures to ensure that staff monitoring Merida Initiative projects develop monitoring plans that address risk. USAID indicated that USAID/Mexico is revising its Project and Activity Design Mission Order to incorporate recently issued USAID guidance and address our recommendation. According to USAID, the mission order will provide a framework and guidance to ensure that USAID/Mexico systematically addresses project risks and incorporates them into the respective monitoring plan.

We are sending copies of this report to the appropriate congressional committees, the Secretary of State, and the USAID Administrator. In addition, the report will be available at no charge on the GAO website at http://www.gao.gov.

If you or your staff have any questions about this report, please contact me at (202) 512-2964 or GurkinC@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made contributions to this report are listed in appendix V.

Appendix I: Objectives, Scope, and Methodology

This report (1) examines the extent to which the Department of State (State), Bureau of International Narcotics and Law Enforcement Affairs (State/INL), follows key practices in monitoring Mérida Initiative projects and tracks project performance data against established measures; (2) examines the extent to which the United States Agency for International Development (USAID) follows key practices in monitoring Mérida Initiative projects and tracks project performance data against established measures; and (3) describes how State/INL uses data from the Government of Mexico to help monitor the implementation of Mérida Initiative projects. To address these objectives, we reviewed relevant State and USAID agency documents and interviewed agency officials from the Departments of State (State), Homeland Security (DHS), Defense (DOD), and Justice (DOD), and USAID in Washington, D.C., and officials from State and USAID in Mexico City.

In 2019, we reported on 14 leading practices for monitoring foreign assistance that agencies should incorporate in their monitoring policies to help ensure that they effectively manage foreign assistance, address impediments, and meet their assistance goals. From these leading practices, which are focused on a high-level assessment of agency monitoring policies, we derived eight key practices that can help agencies monitor the implementation and performance at the project level, such as those implemented under the Mérida Initiative. These eight key practices include those that in our judgment directly relate to monitoring project- level performance activities. We did not address monitoring of financial activities, because our review focused on performance monitoring. We made minor modifications to the key practices selected to reflect the focus of our review. We also grouped the selected key monitoring practices into three areas: (1) assigning monitoring duties to qualified staff, (2) planning a monitoring approach, and (3) monitoring project implementation.

To determine the extent to which State/INL and USAID followed key practices in monitoring Mérida Initiative projects, we selected a nongeneralizable sample of 15 high–dollar value State/INL projects and five high–dollar value USAID projects that started between January 1, 2014, and December 31, 2016. (See app. II for details on these 20 projects). Some of these projects were ongoing after fiscal year 2019. We selected the projects from a list provided by State/INL and USAID. State’s list included 388 projects, and USAID’s list included 57 projects for a total of 445 projects under the Mérida Initiative. We selected projects implemented through a variety of mechanisms. For State/INL, we selected two letters of agreement with international organizations, four grants, three contracts, and two interagency agreements implemented by DOD, two interagency agreements implemented by DHS, and two interagency agreements implemented by DOJ. For USAID, we selected two contracts and three grants. The value of the 15 State projects in our sample is about $88 million, and the value of the five USAID projects in our sample is about $107 million. These 15 State/INL projects represent about 25 percent of the total value of the State/INL projects that started during this period. These five USAID projects were the highest value contracts and grants cooperative agreements and represent about 70 percent of the total value of USAID projects that started during this period. Because State/INL implements about 90 percent of all Mérida Initiative projects, we chose a larger State/INL sample than USAID sample.

We assessed the agencies’ monitoring of the 20 selected Mérida Initiative projects against eight key monitoring practices largely derived from GAO’s Leading Practices for Monitoring Foreign Assistance. We reviewed documents to determine the extent to which State/INL and USAID followed the eight key monitoring practices for each of the selected Mérida Initiative projects. Specifically, for each selected project, we requested monitoring plans; work plans; risk assessments; Contract, Grant, or Agreement Officer Representative Certificates; Contract, Grant, or Agreement Officers Representatives Designation Letters; implementer progress reports for the latest year of activity of each project (at the time of our review); samples of field or site visit reports; and samples of monitoring emails between monitoring staff and the implementers. We reviewed available documents as they related to each key practice to determine the extent to which the agency had taken steps to follow and document the key practice for each project.

On the basis of our review, we assessed whether the key practices were “generally followed,” “partially followed,” or “not followed.” We rated the extent to which the agency followed each key practice as “generally followed” if we received evidence that all critical elements of the key practice were conducted and documented to a large or full extent, “partially followed” if we received evidence that some but not all critical elements of the key practice were conducted and documented, and “not followed” if we did not receive evidence that any of the critical elements of the key practice were conducted and documented. To perform these analyses, two analysts reviewed the documents to rate the extent to which each key practice was met. The analysts worked iteratively, comparing notes and reconciling differences at each stage of the analysis. In addition, GAO staff independent of the two analysts reviewed the final analysis, and modified it as appropriate.

To determine the extent State/INL and USAID track project performance, we chose a nongeneralizable subset of the 20 projects listed above. Specifically, we chose six projects—four State/INL projects and two USAID projects—primarily based on their high–dollar values. (See app. II for details on these six projects.) We chose a small subset of State/INL and USAID projects to conduct a detailed analysis of data in the projects’ annual and quarterly reports. Specifically, for the four State/INL projects, we chose high–dollar value projects for each of the following implementing mechanisms: grants, interagency agreements, and agreements with international organizations. We excluded contracts from the State/INL subset sample, because the high–dollar value contracts generally did not have the project-level performance measures needed to assess State’s tracking of performance data. We included a second grant in our sample in place of a contract, because more Mérida Initiative State/INL projects are grants than interagency agreements or agreements with international organizations. As a result, our State/INL sample consisted of two grants, one interagency agreement, and one agreement with an international organization. For the USAID sample, we chose one grant or cooperative agreement and one contract. We did not choose other types of implementing agreements because grants/cooperative agreements and contracts comprise over 98 percent of USAID projects for the timeframe of our review.

For both the State/INL and USAID selected projects, we reviewed project monitoring documents—such as project narratives, workplans, and monitoring plans—and identified the performance measures outlined in these documents for each project. We then reviewed these projects’ latest year of implementer quarterly and annual progress reports (at the time of our review), and assessed the extent to which State/INL and USAID assessed and approved implementing partners’ periodic performance reports and data in accordance with the key monitoring practice of assessing and approving performance information. We also met with State/INL and USAID monitoring officials in Washington, D.C., and Mexico to understand the process for how these officials track the performance of these selected projects, including in the projects’ quarterly and annual reports. We also reviewed the reports to identify any discrepancies or errors.

To describe the type of Government of Mexico data that State/INL uses to monitor Mérida Initiative implementation, we reviewed data from fiscal years 2015-2018 related to Mérida Initiative projects collected by the Government of Mexico and shared with State/INL. We also met with State/INL officials in Washington, D.C., and Mexico City to discuss the data, including how it is used and its reliability. After our discussions with State/INL officials, State/INL selected some unclassified examples of the indicators, which we included in our report. The purpose of this component of our review was to describe the nature and use of the Mexico data.

We conducted this performance audit from November 2018 to May 2020 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Appendix II: Selected State/INL and USAID Mérida Initiative Projects Assessed against Key Monitoring Practices

This appendix provides a list of the 15 Department of State (State), Bureau of International Narcotics and Law Enforcement Affairs (State/INL) Mérida Initiative projects, and five United States Agency for International Development (USAID) Mérida Initiative projects selected for our review. We assessed State/INL and USAID monitoring of these projects against key monitoring practices as described in appendix I. The subset of these projects (four State/INL and two USAID) selected for our analysis of the agencies’ tracking of performance data is noted below. State/INL provided the details in table 6, and USAID provided the details in table 7.

Appendix III: Comments from the Department of State

Appendix IV: Comments from the U.S. Agency for International Development

Appendix V: GAO Staff Acknowledgments

GAO Contact

Staff Acknowledgments

In addition to the contact named above, James Michels (Assistant Director), Francisco Enriquez (Analyst-in-Charge), Terry Allen, Ashley Alley, Lilia Chaidez, Martin De Alteriis, Neil Doherty, Teresa Heger, John Hussey, and Andrew Kincare made key contributions to this report.