Human Trafficking: Monitoring and Evaluation of International Projects Are Limited, but Experts Suggest Improvements
Released: 2007-07-26T00:00:00

Letter

Background

Human trafficking generally involves the use of force, fraud, or coercion to enslave individuals in situations that are exploitative and often illegal and dangerous. Since 2001, the U.S. government has contributed approximately $447 million worldwide to foreign governments, NGOs, and international organizations—such as UNODC, ILO, and IOM—to combat human trafficking. U.S. agency and international organization projects generally aim to prevent trafficking, protect victims, and prosecute traffickers. Organizations have different requirements for monitoring and evaluating their antitrafficking projects.

Human Trafficking Cases Often Follow a Similar Pattern

Although the crime of human trafficking can take different forms in different regions and countries around the world, most human trafficking cases follow a similar pattern—that is, traffickers use acquaintances or false advertisements to recruit men, women, and children in or near their homes, and then transfer them to and exploit them in another city, region, or country.According to the TVPA, victims of severe forms of trafficking include those who are recruited or transported for labor through the use of force, fraud, or coercion and for the purpose of subjecting them to involuntary servitude. The use of fraud, force, or coercion typically distinguishes human trafficking victims from individuals who are smuggled. In some cases, individuals may enter freely into agreement with and pay smugglers to help them cross international borders. After arriving at their destination, however, individuals who do not understand the destination country’s language or culture may be exploited by individuals who take advantage of their vulnerability.

Traffickers control their victims’ living and working conditions by physically confining them, taking away their identity documents, and threatening their families. Traffickers also exploit victims’ fears that authorities will prosecute or deport them if they seek help. Victims may be forced to work in legal or illegal and often dangerous areas, including brothels, sweatshops, agricultural businesses, and people’s homes. Documented human trafficking cases have involved victims such as children forced to beg for money in cities, work in carpet shops, and participate in pornography and sex acts with adults; women held in slavery- like conditions as domestic servants, strip club dancers, and prostitutes; and men forced to perform work in the agricultural sector and on fishing vessels.

U.S. Government and Multilateral Organizations’ Projects Focus on Three Areas

To combat the diverse forms of global human trafficking, 5 U.S. departments, USAID, and at least 15 international organizations have provided assistance to governments and civil society organizations in more than 100 countries. Assistance generally aimed to enhance efforts to 1. prevent human trafficking through public awareness, outreach, education, and advocacy campaigns; 2. protect and assist victims by providing shelters as well as health, psychological, legal, and vocational services; and 3. investigate and prosecute human trafficking crimes by providing training and technical assistance for law enforcement officials, such as police, prosecutors, and judges.

These categories of interrelated victim-centered assistance activities— prevention, protection, and prosecution—are commonly referred to as “the three p’s.” Each type of assistance is viewed as critical for reducing the incidence of human trafficking.

Since 2001, U.S. government agencies have provided approximately $447 million in foreign assistance to international organizations, NGOs, and foreign governments to combat human trafficking (see table 1). Of the U.S. agencies, State, Labor, and USAID have provided the most funding to combat global human trafficking.

As we have previously mentioned, U.S. agencies support antitrafficking projects implemented worldwide by various international organizations, which also receive funding from other donor governments and organizations. As shown in table 2, for UNODC, ILO, and IOM, total resources allocated to combating trafficking since 2000 totaled about $255 million. The U.S. government provided about $122 million to UNODC, ILO, and IOM to combat human trafficking, according to data provided by these organizations and State’s Office to Combat and Monitor Trafficking and Persons (G/TIP). In addition, according to UNICEF’s annual reports, between 2003 and 2005, UNICEF allocated more than $453 million to its worldwide child protection program, which includes projects to combat trafficking and the sexual exploitation of children.

Some U.S. agencies and international organizations have provided assistance that, although not specifically related to human trafficking, may help to combat trafficking by reducing individuals’ vulnerability to becoming victims and by strengthening countries’ judicial systems. For example, USAID, the World Bank, the Asian Development Bank, and the Inter-American Development Bank have implemented projects to root out corruption, build host governments’ legal systems, and improve the economic conditions of vulnerable populations in developing countries. See appendix II for additional information on international organizations’ missions and antitrafficking activities.

U.S. Agency and International Organizations’ Monitoring and Evaluation Requirements Vary

Monitoring and evaluation are important tools for managing projects and should be considered when designing projects. Performance measurement involves developing a logic framework to explain how an intervention is to achieve its intended outcomes. Monitoring helps agencies determine whether the project is meeting its goals, update and adjust interventions and activities as needed, and ensure that funds are used responsibly. Monitoring includes, among other things, the development of indicators that are linked as closely as possible to the variables identified in the logic framework. These indicators are to be used throughout implementation to assess whether the project is likely to achieve the desired results. Monitoring also involves the choice of baseline and target values for each indicator and includes plans for periodic performance reports and data quality reviews. Evaluation is needed to assess a project’s impact or effectiveness. Project evaluation involves the development of a methodology that will be used to assess the project’s impact and describes plans for collecting baseline, interim, and final data on project results.

The TVPA requires the President’s Interagency Task Force to Monitor and Combat Trafficking in Persons to measure and evaluate the progress of the U.S. government’s efforts to combat trafficking. The Trafficking Victims Protection Reauthorization Act of 2003 (TVPRA 2003) includes a congressional finding that additional research is needed to fully understand the phenomenon of trafficking in persons and to determine the most effective strategies for combating trafficking. The TVPRA 2003 also requires an annual report from the U.S. Attorney General to Congress to provide information on U.S. government activities to combat trafficking in persons. In addition to these reports, Justice began preparing annual assessments of U.S. government activities to combat trafficking in persons in 2003. All six U.S. organizations’ guidelines also require implementers to monitor and report on project performance. In addition, Labor also requires independent midterm and final evaluations for all international antitrafficking projects.

Organizations Have Taken Steps to Collaborate, but Continue to Face Challenges

Global, regional, and country-level organizations recognize the importance of collaborating to effectively combat human trafficking. Although organizations involved in combating trafficking—governments, multilateral organizations, and donors—have implemented some practices to strengthen collaboration, to succeed they will need to overcome challenges that can impede collaboration, including varying levels of government commitment and capacity.

Global, Regional, and Country-Level Organizations Recognize the Importance of Collaborating to Effectively Combat Human Trafficking

At the global level, several UN organizations have acknowledged the importance of collaborating to effectively combat human trafficking. The UN Chief Executives Boardrecognized the challenges in countering human trafficking and proposed establishing an interagency mechanism to strengthen coordination in 2005. In July 2006, a UN Economic and Social Committee resolution requested that UNODC organize a meeting to coordinate the technical assistance that UN and other intergovernmental organizations provide. In December 2006, the UN General Assembly adopted a resolution recognizing that broad international cooperation between member states and intergovernmental and nongovernmental organizations is essential for effectively countering the threat of human trafficking, and underlined the importance of bilateral, subregional, and regional partnerships, initiatives, and actions. This resolution further encouraged member states to initiate and develop working-level contacts among countries of origin, transit, and destination, especially among police, prosecutors, and social authorities.

Organizations at the regional level have also recognized the importance of collaboration. For example, in a September 2006 report, the Organization for Security and Cooperation in Europe’s (OSCE) antitrafficking unit noted that, since trafficking is a transnational crime, collaboration is crucial to enable transnational mechanisms of communication and cooperation among governments, law enforcement, judiciary, and NGOs. In addition, six countries in the Mekong subregion of Southeast Asia have called for strengthened cooperation to combat human trafficking under a process called the Coordinated Mekong Ministerial Initiative Against Trafficking (COMMIT).

At the country level, donor governments and UN organizations have also recognized the need to establish coordination mechanisms to, among other things, share information and leverage the comparative advantages of each organization. The Paris Declaration of Aid Effectiveness establishes the importance of donor coordination, stating that excessive fragmentation of aid at the global, country, or sector level impairs its effectiveness. It further states that a pragmatic approach to the division of labor and burden sharing increases complementarity and can reduce costs. Through the declaration, donors committed to make use of their respective comparative advantage at a sector or country level by delegating authority to lead donors for the execution of projects, activities, and tasks. To draw on the collective strengths of the UN agencies and programs operating in a country, UN country teams and the host government develop a national analysis, called a common country assessment. They subsequently produce a national development assistance framework, which describes the collective UN response and the expected results to achieve national priorities.

For U.S. government agencies, a 2002 presidential directive stated that strong coordination among agencies working on domestic and foreign policy is crucial. The directive called for departments and agencies to coordinate U.S. foreign assistance programs, including those that provide funding to governmental or nongovernmental organizations to combat trafficking in persons. State officials told us that this is done through the Senior Policy Operating Group (SPOG), which, among other activities, facilitates a review by SPOG programming agencies of each other’s grant proposals for antitrafficking projects.

Organizations Face Challenges in Collaborating to Combat Trafficking

Organizations—governments, multilateral organizations, and NGOs—face several challenges in collaborating to combat human trafficking, including the following: Government commitment varies. State’s annual Trafficking in Persons Report, which analyzes and ranks foreign governments’ compliance with minimum standards to combat trafficking, as outlined in the TVPA, illustrates governments’ varied efforts to address the issue. For example, governments might not recognize trafficking as a problem. They may treat foreign trafficking victims as illegal immigrants and deport them back to their home countries, rather than protect them. They also may not recognize trafficking within their own borders as a problem. Moreover, some government officials are themselves involved in human trafficking.

Government capacity varies. Governments with greater resources and more established institutions have a greater capacity to address trafficking than countries that are poorer or less stable. In addition, changes in government leadership and personnel, due to elections, coups, assassinations, or other events, may result in the loss of expertise in combating human trafficking. Furthermore, governments may put trafficking under the purview of ministries with limited authority, such as women’s ministries.

Organizations that combat trafficking vary in perspective and may be in competition for limited funds. Combating trafficking involves organizations—at the international and country levels—with expertise in raising awareness, assisting child and adult victims, and investigating trafficking cases, among other areas. Organizations may view trafficking through their own mandates or viewpoints and may perceive each other not only as collaborators but also as competitors for scarce resources. As such, they may not share information, which could lead to duplication and waste of funds.

Understanding of trafficking varies across languages and cultures.

Countries approach trafficking in different ways. For example, some countries’ national antitrafficking strategies do not include men as potential trafficking victims; also, in some countries, parents in villages often sell their children to work as domestic servants in large cities. Furthermore, translation issues can complicate establishing a common understanding of what constitutes trafficking. For example, although the legal term in Spanish for human trafficking is “trata de personas,” English speakers have translated human trafficking into Spanish as “tráfico de personas,” which in fact means human smuggling.

Organizations Have Taken Steps to Strengthen Collaboration, but Challenges Remain at the Global, Regional, and Country Levels

Global and Regional Collaboration Efforts Have Recently Been Initiated

Organizations at the global level have defined a common outcome—to end human trafficking and slavery—and have recently initiated various efforts to strengthen collaboration. In March 2007, UNODC launched the Global Initiative to Fight Human Trafficking to generate political will, an action plan, and financial resources to combat trafficking worldwide. The steering committee of the initiative includes the six leading international organizations involved in combating trafficking in persons: UNODC, ILO, IOM, UNICEF, OSCE, and the Office of the United Nations High Commissioner for Human Rights. UNODC also created the Interagency Cooperation Group Against Trafficking in Persons in September 2006 with the intended outcome of improving coordination between UN agencies and other international organizations, and to facilitate a holistic approach to preventing and combating human trafficking. The group’s functions include exchanging information and promoting the effective and efficient use of resources.

Most existing regional coordination efforts that we reviewed have defined a common outcome and established action plans for carrying out antitrafficking activities. These efforts vary regarding whether they are specifically directed against trafficking or are included in discussions on migration or smuggling. In addition, they vary in number of participating members from as few as 6 countries in the Mekong subregion to as many as 56 countries in Europe, Asia, and North America. For example, the governments involved in a UN project that specifically focused on strengthening cooperation to combat trafficking in the Mekong subregion have agreed to undertake 18 actions to combat human trafficking, as part of an effort called the COMMIT process. Officials stated that this agreement strengthens these countries’ incentives to undertake the actions through mutual accountability. In addition, COMMIT developed an action plan that groups existing activities into a single framework delineating roles and responsibilities of UN organizations, implementing partners, and governments in the six Mekong countries. Officials contrasted the COMMIT process, which includes a small group of countries in one region with interconnected trafficking problems, to larger organizations such as the Bali Process, which approaches trafficking issues in conjunction with issues related to smuggling in the Asia-Pacific region and has a larger geographic scope, consisting of 38 governments across several regions. These officials stated that having a larger geographic scope makes it more difficult for the governments of these countries to hold each other accountable for implementing the regional action plan. In addition to these examples, OSCE initiated the Alliance against Trafficking in Persons, which established a partnership with major actors working to combat human trafficking to, among other things, develop joint strategies and provide OSCE participating states and others with harmonized decision-making aids. In Southeastern Europe, the International Center for Migration Policy Development has begun a project to facilitate the creation of a transnational referral mechanism by 10 governments in the region and national NGOs to improve transnational case management and victim protection. Furthermore, in the western hemisphere, the Organization of American States (OAS) held a conference in 2006 that resulted in guidelines for the 34 member states and OAS to combat human trafficking.

Organizations continue to face obstacles as they work on global and regional initiatives to collaborate to combat trafficking. For example, governments disagree on whether there is a difference between “forced” and “voluntary” prostitution. Such disagreements can hinder collaborative efforts in combating sex trafficking. These global and regional initiatives may eventually implement additional practices to enhance collaboration, but it is too early to determine the success of the current initiatives.

Country-Level Organizations Have Taken Some Steps to Strengthen Collaboration, but Continue to Face Difficult Challenges

Although organizations—including host governments, UN organizations, U.S. government agencies, and other donor governments—in the three countries we visited have implemented practices to strengthen collaboration in combating trafficking, they continue to face challenges. None of the three countries has mechanisms to coordinate the efforts of all organizations involved in combating trafficking. Host governments bear ultimate responsibility for combating trafficking within their borders, and governments of the countries we visited have taken some steps to collaborate. For example, the Indonesian and Thai governments have passed national antitrafficking laws and enacted national action plans that define common outcomes, outline strategies, and assign roles and responsibilities. However, neither government’s action plan includes trafficking of men in its definition of human trafficking. Officials in Indonesia stated that they expect to revise the plan to include the trafficking of men, as well as women and children, based on legislation passed in 2007. Although both the Indonesian and Thai governments hold interagency meetings, the ministries responsible for coordinating antitrafficking efforts have limited authority and operational capacity, according to officials we interviewed. Unlike Indonesia and Thailand, Mexico has neither a national antitrafficking law nor an action plan. Mexican officials stated that they convened one interagency meeting on human trafficking and plan to institute additional coordination mechanisms after the government passes a national antitrafficking law.

In Indonesia and Thailand, UN organizations, in conjunction with host governments, have developed country assessments and assistance frameworks to articulate overall development goals—including combating trafficking—joint strategies, and roles and responsibilities. UN organizations working on trafficking in Thailand also meet as part of the Mekong regional project that we previously discussed; although, according to one donor government official, attendance among UN organizations is sporadic. UN officials in Indonesia told us that they share information informally, but do not meet on a regular basis to discuss trafficking.

According to U.S. government officials in Indonesia and Thailand, donor governments have made sporadic and informal efforts to leverage resources and avoid duplication of effort. For example, as a result of informal coordination, Justice and French government officials worked together on a criminal justice training project in Indonesia. In Thailand, according to a U.S. official, although most of the major donors attend the Mekong regional project’s meetings, these meetings are generally focused on UN activities, with little opportunity for donors to coordinate. Officials in Indonesia and Thailand acknowledged the need to establish regular bilateral donor coordination efforts on trafficking issues and, at the time of our fieldwork, had begun discussing the establishment of regular coordination meetings.

While U.S. government agencies overseas have developed some collaboration mechanisms to combat trafficking, they continue to face challenges in coordinating their efforts. The U.S. embassies in the three countries we visited, Indonesia, Thailand, and Mexico, include trafficking in persons in their mission performance plans, which establish combating trafficking as a component of the U.S. government’s overall strategy in each country. U.S. officials in these countries told us they organize trafficking- specific meetings that include U.S. government agencies and their implementing partners, primarily to share information. The U.S. government officials who were involved in antitrafficking issues were responsible for other issues as well. In Indonesia, for example, the primary U.S. embassy contact for trafficking was also temporarily assigned deputy chief of mission in East Timor. In Mexico, one agency official, whose responsibilities consisted solely of human trafficking issues, had been named to also serve as coordinator and primary contact for all U.S. antitrafficking efforts. After this official's departure in December 2006, her portfolio was added to the existing portfolio of another official from the same U.S. agency. At the time of our visit in February 2007, State, USAID, and DHS officials, who also covered other issues, were holding meetings to coordinate their antitrafficking efforts internally. However, a new U.S. antitrafficking coordinator for Mexico had not been designated. Consequently, some Mexican government officials expressed uncertainty about which U.S. agency official had the role of lead U.S. government coordinator for trafficking in Mexico.

Furthermore, some U.S. government officials noted challenges in coordinating between officials in Washington, D.C., and those overseas. For example, a U.S. official in Thailand was unaware of a U.S.-funded project in the country until that project hosted a conference and requested additional funding from the U.S. embassy. The project had not been included in the list of antitrafficking activities in Thailand that the official received from State’s trafficking office, which is based in Washington. According to some agency officials overseas, not knowing what other agencies and bureaus plan to spend on antitrafficking activities in a particular country makes it difficult for them to determine how much of their budgets to allocate to antitrafficking activities. Although State’s new Office of Foreign Assistance has begun to address the issue of better coordinating all U.S. foreign assistance by bringing together core State and USAID teams to discuss U.S. development priorities in each recipient country, some U.S. officials expressed uncertainty regarding which part of the U.S. government would be responsible for outlining a new country-level strategy and budget for combating trafficking.

U.S. Government- Funded Antitrafficking Projects Lack Some Key Monitoring Elements; Little Is Known about Project Impact Due to Difficulties in Conducting Evaluations

Antitrafficking project documents we reviewed generally include monitoring elements, such as an overarching goal and related activities; however, they often lack other monitoring elements, such as targets for measuring performance. Various other factors also make it difficult to evaluate the impact of antitrafficking projects. These factors include questionable estimates of the number of trafficking victims at the project level, which are needed to evaluate the effectiveness of specific antitrafficking interventions. Certain project elements may further impede evaluations, including short time frames and overly broad objectives. Because of these factors, few evaluations that determine impact have been completed. As a result, little is known about the impact of antitrafficking interventions.

U.S. Government-Funded Antitrafficking Projects Lack Some Important Monitoring Elements; State Is Taking Steps to Strengthen Monitoring

Most of the project documents we reviewed generally include one or more monitoring elements, but lack others. We reviewed documents for 23 U.S. government-funded antitrafficking projects in Indonesia, Thailand, and Mexico, which generally include statements of project goals and a description of activities. However, the majority of these documents lack a logic framework that clearly links activities with project-level goals, indicators, and targets. Specifically: Eighteen of the 23 projects do not clearly explain how activities will achieve stated goals. For example, the project proposal for a State- funded project in Thailand does not provide clear linkages demonstrating how the training of local officials and screenings of a public awareness video will achieve the project’s overall goal of reducing trafficking among vulnerable adolescents and women. In contrast, all 4 Labor-funded projects in Indonesia, Mexico, and Thailand clearly link project goals and activities. For example, 1 goal of a project in Mexico is to develop a system to identify networks of exploiters. Activities to achieve this goal include developing computer software and training government officials in its use.

Twenty-one of the 23 projects identify indicators, but of these, only 10 specify targets by which performance is measured. For example, a State-funded project in Thailand included the “number of victims rescued” and the “number of arrests of traffickers” as performance indicators, but did not set numerical targets for measuring performance. In contrast, a State-funded project implemented by IOM in Indonesia established an expected result that 500 victims of trafficking receive rehabilitation and reintegration assistance. For the 13 projects that did not specify targets, the performance standards to which grantees were held accountable were not clear. Of the 10 projects that did specify targets, 5 explained how targets were set. These 5 projects included all 4 of the Labor projects we reviewed.

Another element of monitoring project performance is to supplement implementing partners’ reporting of programmatic and financial progress with independent review through site visits at the field level. For example, Labor officials reported that the International Labor Affairs Bureau’s Office of Child Labor, Forced Labor, and Human Trafficking has engaged ILO’s external auditor to conduct audits of a sample of ILO’s international program for the elimination of child labor projects. This office also has contracted with a certified public accounting firm to conduct independent attestation engagements of its Education Initiative projects. Among the objectives of these audits and attestation engagements is an assessment of the accuracy and reliability of performance data from grantees’ progress reports.

State G/TIP has 4 Washington, D.C.-based staff members who are responsible for overseeing projects in approximately 70 countries. G/TIP officials stated that because they cannot visit all of the projects, they rely on U.S. embassy staff to provide additional field-level oversight. However, the office has not established written guidance for conducting such oversight. During our fieldwork, an embassy official expressed frustration at the lack of clear guidance and procedures. Embassy officials also told us that they meet with project staff, but have other responsibilities that limit the time they can devote to overseeing antitrafficking projects. According to State’s Bureau of Population, Refugees,and Migration (PRM) officials, the bureau has only 3 staff to monitor and oversee its antitrafficking projects currently being implemented in over 20 countries, and these staff cannot conduct site visits to all projects. PRM staff also stated that they review progress reports submitted by project implementers, but do not have a database or system for compiling the information they receive from the field.

To address some of these concerns, State officials told us that they are taking steps to strengthen monitoring, including developing new indicators and written guidance for monitoring antitrafficking projects. State G/TIP officials told us that they are developing a system of key indicators to better inform management decision making that would be consistently used for each of the three main types of antitrafficking programs— prevention, protection, and prosecution. State PRM and IOM, its key implementing partner, have also been involved in an effort to develop and standardize an indicator framework across IOM missions, although this effort was not completed during the time of our review. Officials stated that this effort will serve as a way to exchange ideas about best practices in different parts of the world and will provide managers with information on implementation needed to make decisions about current or future projects.

Factors Impede Evaluations of Antitrafficking Projects and Little Is Known about Project Impact

Various factors impede impact evaluations of antitrafficking projects. First, data on human trafficking are questionable, including estimates of the number of trafficking victims, making it difficult to determine a preproject baseline. Second, elements of project design, such as overly broad objectives or short project duration, diffuse potential impact. Because of these factors, it has been difficult to evaluate the effectiveness of antitrafficking interventions, and few evaluations that determine impact have been completed. As a result, little is known about the impact of antitrafficking interventions.

Lack of Project-Level Baseline Estimates Limits Evaluations

The lack of accurate baseline estimates of the number of trafficking victims is a limitation to conducting evaluations. As we have previously reported, the accuracy of current estimates is questionable. Without estimates of the scope of human trafficking to use as baselines in project locations, it is very difficult to determine where interventions are most needed or where interventions would have the greatest impact. Developing baseline estimates is difficult for the following reasons: Victims are a hidden population. Victims may be unaware, unwilling, or unable to acknowledge that they are trafficking victims. Therefore, it is difficult to reach them to collect information using standard sampling techniques.

Service providers may be unwilling to share victim data due to confidentiality concerns. For example, the global database maintained by IOM is not publicly available since assisted victims are in a precarious position and revealing their identity could have a detrimental effect on their safety.

The definition of the term “trafficking in persons” is broad and varies in meaning across different languages. As we previously discussed, the understanding of trafficking varies across languages and cultures. These variations in definition hinder the comparability of data across countries and organizations.

There are no commonly agreed-upon criteria for identification of human trafficking victims. This lack of criteria hinders the ability to identify victims, create consistent statistical databases, and design analytical tools for surveys and estimates.

Existing data may not be reliable. Developing countries, which are typically the countries of origin, have limited capacity for data collection, and their governments’ commitment to combating trafficking may be insufficient. Thus, sufficiently reliable data needed for estimating trafficking incidence may not be available.

Elements of Project Design Impede Evaluations

Elements in the design of certain antitrafficking projects, including a tendency to focus on very broad, high-level objectives across a diverse range of activities, diffuse projects’ potential impact and create challenges for evaluations.

When projects focus on overly broad objectives and contain too many types of activities, their potential impact is diffused, which makes them difficult to evaluate. For example, some antitrafficking projects have very high-level goals or objectives, such as “creating an environment for effective action against trafficking,” or “strengthening the initiatives of government and others against human trafficking” and contain activities covering 2 or more of the 3 key types of antitrafficking interventions— prevention, protection, and prosecution. Of 153 U.S. international antitrafficking projects funded in fiscal year 2004, 2005, or 2006, 56 percent included 2 or more of these interventions and 29 percent included all 3 interventions. Activities included public awareness campaigns, victim assistance, and training for law enforcement officials.

While projects funded with greater resources could lead to more noticeable longer-term changes, the impact of a shorter-term, smaller-scale intervention may be difficult to attribute and quantify. Antitrafficking projects vary significantly in terms of their time frames and funding levels. For example, the projects we reviewed varied in duration from 1 year to about 4 years, and projects’ funding levels varied from $15,000 to $6 million. Officials stated that some organizations, such as State G/TIP, tend to fund smaller, shorter-term projects, while Labor usually funds larger, longer-term projects. Overall, experts told us that organizations also have generally limited funding for impact evaluations and for research on the nature and scope of human trafficking.

Experts stated that another factor impeding evaluations is that little attention has generally been devoted at the start of projects to evaluation design. For example, questions related to determining the control group, the type of design for impact evaluation, the data that would be collected, and the analytical methods that would be most suitable are often not addressed before implementation. Furthermore, experts stated that although projects target several groups of beneficiaries, it is generally unclear how they can be reached and to which group they would be compared. As a result, it is difficult to determine what would constitute successful project implementation.

Little Is Known about Project Impact

Because of the difficulties in evaluating antitrafficking projects, the few evaluations that have been completed are qualitative rather than quantitative, focus on process rather than impact, and rarely trace victims over time.

The few evaluations completed used qualitative methods that are valuable for documenting victims’ perceptions and experiences, but, unlike quantitative methods, they cannot be used to generalize results to broader populations. For example, IOM’s Office of Inspector General has carried out impact evaluations of a select number of antitrafficking projects. Our review of these evaluations shows that they covered interventions in various parts of the world funded by different donors. All of the available evaluations applied qualitative methodologies, consisting of document reviews, site visits, interviews, and focus groups with stakeholders. In addition, in a USAID-funded assessment of child trafficking victims residing at a shelter in Nepal, an evaluator used qualitative methods, such as observations and interviews, to collect information on their daily lives, needs, and preparation for reintegration into the community. The evaluator concluded that skill-based training provided by the shelter does not adequately prepare the girls for their reintegration into their communities; however, this qualitative result could not be generalized to the national level.

Completed evaluations also typically focused on process rather than impact. Process evaluations are an important tool for improving service delivery and assessing program effectiveness by assessing whether activities conform to program design. Impact evaluations assess the net effect of a program by comparing program outcomes with an estimate of what would have happened in the absence of the program. However, impact evaluations are an emerging field in antitrafficking interventions. For example, a process evaluation of a Labor-funded project aimed at reducing the victimization of minors who have been trafficked or are at risk of being trafficked in Thailand focused on the project’s planning, implementation, and management. This process evaluation was designed to monitor implementation and assess project outcomes. However, the evaluation did not ultimately assess the project’s impact in reducing the victimization of minors.

Finally, our review identified few evaluations that track individual victims who have been reintegrated into their communities over time. For example, ILO is currently pilot-testing an application of a tracer study methodology in the context of trafficking prevention. In addition, according to Labor officials, in some cases the office has conducted follow-up studies that examine a sample of beneficiaries to document changes that have occurred. Furthermore, IOM officials noted that several of its missions provided statistical information to gauge the success of its victim reintegration efforts over time based on specific indicators. According to experts, such evaluations are costly and time-consuming. Tracking victims by country can also be difficult because country boundaries are permeable and many trafficking routes cross national borders.

Expert Panel Identified Ways to Address the Difficulties of Monitoring and Evaluating Antitrafficking Projects

A GAO-convened panel of experts identified and discussed ways to address the factors that make it difficult to monitor and evaluate antitrafficking projects. Panelists suggested approaches to improve the monitoring and evaluation of antitrafficking projects, by improving information on the severity of human trafficking and addressing weaknesses in the design of antitrafficking projects.

Conduct Research on the Nature and Severity of Human Trafficking

Panelists acknowledged that the lack of existing data on the nature and severity of human trafficking limits evaluations of antitrafficking projects, and suggested ways to improve information on the nature and severity of human trafficking. According to the panelists, researchers need to gather evidence to answer the following questions: What is the nature and severity of human trafficking? Panelists stated that it is necessary to understand the nature of trafficking in terms of its underlying conditions and the types of traffickers and victims. Trafficking is a multidimensional, complex problem that involves a wide range of victims; recruiters, brokers, and intermediaries; and abusive employers and sexual exploiters. Understanding the incentives of the people engaged in trafficking is an important first step.

The severity of human trafficking can be measured by using qualitative and quantitative methodologies. Panel members suggested several sampling methods that have been used to sample other hard-to-reach populations, including the homeless, hidden migrants, missing and exploited children, domestic violence victims, inmates, and drug users. One suggested method is sampling of “hot spots”—an intensive search for victims in areas known to have high concentrations of victims or in areas to which many victims return. Other methods include adaptive cluster, double, indirect, and snowball sampling. (For a more detailed discussion of sampling methods, see app. III.) These methods could be used individually or in combination. Panelists further emphasized that, whenever feasible, it is important to use methodologies that are appropriate for the location sampled. In addition, it is critical to determine whether the results are unique to a certain location or whether they can be generalized to other locations. Panelists recommended that research start at the local level. Such research would identify successful small-scale interventions, increase the knowledge of victims’ needs, and develop meaningful performance measures. Lessons learned at the local level could then be expanded to national and regional levels.

What is the projects’ estimated effect on the nature and severity of human trafficking? Panelists emphasized the necessity of designing rigorous methods to evaluate antitrafficking projects, such as randomized control trials. For example, as baseline estimates in trafficking hot spots are obtained and interventions are undertaken in that location to reduce trafficking, the use of “place-randomized trials” for evaluation would become possible. To determine the interventions’ impact, data obtained from interventions in hot spots could then be compared with locations where there had been no interventions. Although randomized trials may be difficult to execute for many trafficking projects, they are important ways to generate evidence about interventions’ effectiveness. Randomized trials should be pilot-tested in carefully chosen settings so that the evaluator can identify and correct any problems encountered before expanding the trials to larger populations or areas. Panelists pointed out that such rigorous evaluation has occurred in the fields of public health and criminal justice. For example, a randomized study was done of brothels in Thailand to obtain information on the HIV/AIDS rates of prostitutes.

Address Monitoring and Evaluation Weaknesses in the Design of Antitrafficking Projects

To address weaknesses in project design that impede monitoring and evaluation, such as projects with very broad, high-level objectives, panelists made the following recommendations: Develop a logic framework. Given the weaknesses of some antitrafficking projects that have overly broad objectives across a diverse range of activities, panelists suggested that officials design projects with a logic framework that has clear objectives and narrow the focus of interventions. They also recommended that officials design projects that clearly link activities to intended outcomes, identify measurable indicators, and establish procedures for setting and modifying targets. Measurable indicators with mutually agreed-upon targets allow project officials to assess how a project is achieving its overall goals and objectives. For example, a Labor-funded project implemented by ILO in Mexico includes a logic framework that links project activities and outputs to outcome objectives, and includes clear indicators and means of verification. The project’s overall goal of eliminating the commercial sexual exploitation of children in Mexico links to four immediate objectives that, in turn, link back to project activities and outputs. One objective—that at least 300 child victims of sexual exploitation or at-risk children and their families receive assistance—is linked to 28 activities and 8 outputs, such as increasing families' employment opportunities. These 28 activities include disseminating employment promotion programs, organizing employment training, and monitoring and analyzing the impact of these dissemination and training efforts. Panelists emphasized that donors and implementers should agree on the project’s logic framework during the design phase.

Determine whether a project is ready to be evaluated. Given the significant variance in project duration and funding levels, panelists emphasized that evaluators would not be able to evaluate all existing projects, but should first determine which projects are ready to be evaluated. In conducting such “evaluability assessments,” evaluators determine, among other things, whether (1) the project is large enough, has sufficient resources, and has been implemented long enough to make an impact; (2) the project is reaching its target population; (3) project documents specify and clearly link objectives, goals, and activities; and (4) sufficient information exists to determine impact. For example, larger, long-term projects are more likely to have an impact and, thus, may be better candidates for evaluation than smaller, short- term projects. Because larger antitrafficking projects generally include a diverse range of interventions, panelists suggested narrowing the evaluation to focus on discrete interventions or aspects of the project.

Build monitoring and evaluation into project design. Given that evaluation is not generally considered during design, panelists emphasized that project officials should consider how the project will be evaluated before the project is implemented. Most importantly, organizations need to define the project’s intended impact and how that impact will be measured. To do this, organizations would have to determine the project beneficiaries and to which group they would be compared, the data they would have to collect, and how they would analyze those data. Management would not only need to collect data during implementation to monitor if the project works according to plan, but also collect data before and after implementation to determine what works best.

Conclusions

The United States has played an important role in combating global human trafficking and spurring other governments to increase their efforts in doing so. As organizations around the world increasingly collaborate in combating trafficking, their ultimate success will depend on the extent to which they are able to overcome difficult challenges, such as varying levels of government commitment and capacity, that have impeded collaboration in the past. More than 7 years after the passage of the UN protocol, little is known about which interventions have been the most effective in preventing human trafficking, protecting victims, and prosecuting traffickers.

The United States and other governments, international organizations, and NGOs continue their efforts to fight trafficking, but there is little information available to inform their decisions about project implementation and selection. Although antitrafficking projects contain some important elements for monitoring, they often lack other important elements for measuring performance on a real-time basis, such as targets. Such elements are critical in monitoring project performance to determine whether interventions are being implemented as expected, or whether they need to be changed to better combat human trafficking. Evaluation is also important in determining whether antitrafficking projects have been effective. However, few impact evaluations have been completed due to the difficulties involved. As a result, little is known about the impact of antitrafficking interventions.

Given the grave personal suffering of victims and negative impacts on society that human trafficking creates, strengthening collaboration, monitoring performance, and evaluating impact are important to ensure that organizations fund antitrafficking interventions with the greatest impact, where they are most needed, and through the effective and efficient use of resources.

Recommendations for Executive Action

We recommend that the Secretaries of State and Labor and the Administrator of USAID improve the monitoring and evaluation of their projects to combat global human trafficking by considering the following actions, where appropriate: 1.

Improve information about project impact on the nature and severity of human trafficking, including developing better data about the incidence of trafficking at the applying rigorous evaluation methodologies. 2. Address monitoring and evaluation weaknesses in the design of developing a framework that clearly links activities with project-level goals, indicators, and targets; conducting “evaluability assessments” to determine whether a project is ready to be evaluated; and building monitoring and evaluation into project design before the project is implemented.

We are addressing our recommendations to State, Labor, and USAID because they provided the most U.S. funding for projects to combat global human trafficking.

Agency Comments and Our Evaluation

We requested comments on a draft of this report from the Secretaries of State, Justice, Health and Human Services, Homeland Security, and Labor; the Administrator of USAID; and cognizant officials at UNODC, IOM, ILO, OAS, OSCE, the World Bank, IDB, and ADB or their designees. We received written comments from State, Labor, USAID, and HHS, which are reprinted in appendixes IV, V, VI, and VII along with our responses to specific points.

In its comment letter, State noted that it will implement the recommendations in the report in ways that are relevant and appropriate to the mandates of the State offices working on antitrafficking efforts. State further noted that the recommendations are entirely consistent with the department’s current activities and direction. State also generally agreed that the antitrafficking field is well-served by more information about the nature and severity of human trafficking and recognized that effective project design is critical to successful project implementation and program monitoring and can lay the foundation for evaluation. However, State disagreed with our finding that monitoring is limited, stating that monitoring is in place at the department and improving. While we recognize that State and other U.S. agencies have certain elements of monitoring in place, we report that they lack others. For example, we found that for the 23 antitrafficking projects we reviewed, the majority do not have a logic model that clearly explains how activities are linked to project goals. In addition, the majority of these projects do not specify targets that will establish benchmarks for measuring performance. State also emphasized that many of its projects are designed to be of more limited size, scope, and duration than those of other agencies, such as Labor. State further noted that these projects of limited duration are worthy of funding, but are not necessarily appropriate for evaluation. In the report, we state that the impact of a shorter-term, smaller-scale intervention may be difficult to attribute and quantify.

Labor commented that the report provides a good overall assessment of international cooperation and the need to enhance collaboration among key agencies and governments regarding antitrafficking efforts. Labor also commented that the report highlights important areas for improving monitoring and evaluation of U.S.-funded antitrafficking programs. However, Labor stated that the report does not fully reflect efforts particular federal agencies are taking in the monitoring and evaluation of antitrafficking projects. As an example, Labor stated that it uses several mechanisms, including audits and process evaluations, in its monitoring and oversight of international technical assistance projects, including antitrafficking projects, to ensure that U.S. funds lead to planned outputs and results. In response, we made additions or revisions to the text to further clarify Labor’s monitoring and evaluation efforts. We believe the overall monitoring of antitrafficking projects is limited because the projects funded by the other five agencies did not have the elements of monitoring we found in Labor’s projects.

USAID said it appreciates the thoughtfulness of GAO’s report. USAID also commented that it is concerned with the challenges of coordination, monitoring, and evaluation, and that while it has made considerable efforts to coordinate within the U.S. government and with other organizations, it will continue to work within the interagency process in Washington and in the field. USAID also agrees that monitoring and evaluation of its antitrafficking efforts are very important and that they rely on the availability of both human and financial resources. USAID further agrees that the issues inherent in the evaluation of antitrafficking activities are particularly challenging because there is no baseline against which to measure progress.

HHS said the report is a sound document that substantively covers the wide range of programs and services available to combat human trafficking.

We also received technical comments from Justice and DHS, as well as from UNODC, IOM, ILO, OAS, OSCE, the World Bank, IDB, and ADB, which we have incorporated in the report as appropriate.

We are sending copies of this report to interested congressional committees; the Secretaries of State, Justice, Health and Human Services, Homeland Security, and Labor; the Administrator of USAID; ILO; IOM; and UNODC. We will also provide copies to others on request. In addition, the report will be available at no charge on the GAO Web site at http://www.gao.gov.

If you or your staffs have any questions about this report, please contact me at (202) 512-9601. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made major contributions to this report are listed in appendix VIII.

Objectives, Scope, and Methodology

Our objectives were to examine (1) collaboration among organizations involved in international antitrafficking efforts, (2) U.S. government agencies’ monitoring of antitrafficking projects and difficulties in evaluating these projects, and (3) suggestions for strengthening monitoring and evaluation.

To examine collaboration among organizations involved in international antitrafficking efforts, we reviewed relevant planning, funding, and project documents on human trafficking from the Departments of State, Justice, Labor, Homeland Security, and Health and Human Services and the U.S. Agency for International Development (USAID). We reviewed planning and project documents from relevant United Nations (UN) and other international agencies and offices, including the UN Office on Drugs and Crime (UNODC), International Labor Organization (ILO), International Organization for Migration (IOM), United Nations Children’s Fund (UNICEF), and UN High Commissioner for Refugees (UNHCR). We also reviewed UN reports and resolutions that address coordination as well as documents from regional organizations, such as the Organization for Security and Cooperation in Europe (OSCE), the Coordinated Mekong Ministerial Initiative Against Trafficking (COMMIT), and the Regional Conference on Migration.

In addition, we reviewed documents from and discussed international antitrafficking efforts with officials from the above agencies and organizations as well as officials from host government, donor government, and nongovernmental organizations (NGO), in Washington, D.C., and during our fieldwork in Indonesia, Thailand, and Mexico. We also reviewed documents describing these countries’ national mechanisms to combat trafficking, including legislation and action plans, where applicable. We selected Thailand because it is the country with the largest number of international organizations working to combat human trafficking. We selected Indonesia and Mexico because they receive a large amount of U.S. funding for international antitrafficking projects and have a relatively large number of U.S. government agencies and international organizations working in each country. All 3 countries are origin, transit, and destination countries for human trafficking victims and have projects addressing prevention, protection, and prosecution.

To examine organizations’ monitoring and evaluation of antitrafficking programs, we reviewed documentation from 23 projects in the 3 countries we visited and illustrative projects from other countries. We worked with U.S. agency officials in Washington and in the field to identify U.S.-funded antitrafficking projects in Indonesia, Thailand, and Mexico that were ongoing during the time of our review (August 2006 to June 2007). We requested project documents—including proposals, grant agreements, and progress reports—from these U.S. officials for projects identified in these countries. In response to this request, we received documents for 23 antitrafficking projects that were funded or implemented by 6 U.S. agencies involved in international antitrafficking efforts. We examined the project documents for the following elements: statement of goals or objectives, statement of activities, identification of indicators and targets, explanation of how targets were selected, and inclusion of a logic model or framework. While we recognize that these 23 projects may not be the complete universe of antitrafficking projects in these 3 countries, we consider these 23 projects sufficient for the purposes of our review. Although we cannot assume that the issues we identified exist across all projects, they nevertheless represent areas for improvement in monitoring antitrafficking projects. We also reviewed a set of 4 State Office to Combat and Monitor Trafficking in Persons (G/TIP) antitrafficking projects in India, Israel, Afghanistan, and Costa Rica. These projects were selected to illustrate some current monitoring and evaluation practices for existing projects in different parts of the world and different types of exploitation. They showed the variation in monitoring practices that can only increase with sample size. Thus, the projects provided the background information and context needed to understand State G/TIP’s current efforts to standardize monitoring requirements. The reviewed documents for all projects included proposals, applications for assistance, project descriptions, strategy papers, concept papers, causal models, cooperative or grant agreements, and periodic and final reports.

We also reviewed articles and books on monitoring, evaluation, and statistics, such as Sampling by Steven K. Thompson, Wiley Series in Probability and Mathematical Statistics (1992); Adaptive Sampling by Stephen K. Thompson and George A.F., Seber, Wiley Series in Probability and Mathematical Statistics (1996); Better Evaluation for Evidence-Based Policy: Place Randomized Trials in Education, Criminology, Welfare, and Health by Robert Baruch, The Annals of the American Academy of Political and Social Science (2005), vol. 599, 6-18; Program Evaluation Methods: Measurement and Attribution of Program Results, Third Edition, Treasury Board of Canada, Secretariat; and Monitoring and Evaluation: Some Tools, Methods and Approaches, the World Bank (2004).

To examine suggestions for strengthening monitoring and evaluation, we worked with the National Academy of Sciences to organize a 2-day expert panel on challenges and alternative strategies for monitoring and evaluating the results of international antitrafficking programs and projects in April 2007. We invited the following groups of panel participants: Experts with broad-based, subject-area knowledge of human trafficking.

Experts with specialized knowledge of monitoring and evaluation of programs aimed at hidden populations similar to trafficking victims, such as the homeless and irregular migrants. This group included experts with specific knowledge of baseline data estimation of such populations.

Panelists had backgrounds in academia, research, consulting, and project implementation in the field. Using a nominal group technique, panelists chose to focus on the intervention “safe return” as a starting point for discussion. Experts provided presentations and participated in a discussion and cross-fertilization of ideas. Using a nominal group technique, panel members also ranked two topics in order of importance to the human trafficking field—estimating the number of trafficking victims and evaluability assessments. None of the panel members were compensated for their work on this project. The following experts participated in the panel: Richard Berk, Professor of Criminology and Statistics, Department of Criminology, University of Pennsylvania Robert Boruch, University Trustee Chair Professor, Graduate School of Education and Statistics Department, Wharton School, University of Pennsylvania Mario Thomas Gaboury, Professor and Chair of Criminal Justice, Kristiina Kangaspunta, Chief, Anti-Human Trafficking Unit, United Nations Office on Drugs and Crime Jonathan Martens, Counter-Trafficking Project Specialist, Counter- Trafficking Division, International Organization for Migration Jeffrey S. Passel, Senior Research Associate, Pew Hispanic Center Lisa Rende-Taylor, Technical Advisor to the United Nations Inter-Agency Project on Human Trafficking in the Greater Mekong Subregion Peter Reuter, Professor, School of Public Policy, Department of Criminology, University of Maryland W. Courtland Robinson, Assistant Professor, Center for Refugee and Disaster Response, Johns Hopkins Bloomberg School of Public Health Debra Rog, Associate Director, Westat Corporation Jane Nady Sigmon, Senior Coordinator for International Programs, Office to Monitor and Combat Trafficking in Persons, U.S. Department of State We conducted our review from August 2006 to June 2007 in accordance with generally accepted government auditing standards.

Selected International Organizations Involved in Combating Human Trafficking

This appendix describes the general mission and antitrafficking activities of 15 international organizations that implement international antitrafficking projects.

Methods Suggested by Expert Panel to Estimate the Number of Human Trafficking Victims

This appendix describes the sampling methods suggested by a GAO- convened panel of experts to estimate the number of human trafficking victims.

Comments from the Department of State

The following are GAO’s comments on the Department of State letter dated July 13, 2007.

GAO Comments

1. State said that the report’s title is not apt because program monitoring is in place and improving. We believe that monitoring is limited for the following reasons: for the 23 projects we reviewed, the majority did not have a logic model that clearly explains how activities are linked to project goals; the majority of these projects also did not specify targets that establish benchmarks for measuring performance; of the 10 projects that did specify targets, only 5 explained how targets were set; and, finally, State lacked written guidance for field-level oversight. 2. State noted that many of its projects are designed to be of more limited size, scope, and duration, and that such projects are not necessarily suitable for evaluation. They also added that larger and longer-term projects are more suitable candidates for impact evaluation than shorter projects. Finally, State commented that a relatively short time frame for projects is not a design weakness. In the report, we state that the impact of a shorter-term, smaller-scale intervention may be difficult to attribute and quantify. 3. State noted that baseline data are not required for all interventions. We disagree. In the report, we state that baseline and target values of indicators are needed to assess project performance. For example, baselines for a victim assistance program could include the number of victims served or assisted, or the number of vocational training sessions held. Moreover, project-level estimates are needed for baselines by which to evaluate how effectively interventions are reducing trafficking. For this type of baseline, panelists suggested methods for baseline estimation, such as gathering data in trafficking hot spots, that would allow for more rigorous impact evaluations.

Comments from the Department of Labor

The following are GAO’s comments on the Department of Labor letter dated July 13, 2007.

GAO Comments

1. Labor noted that federal agencies are carrying out monitoring and evaluation activities or are making efforts to improve monitoring and evaluation of antitrafficking activities. Labor also provided details of its monitoring policies, procedures, and activities. We recognize that Labor has monitoring procedures and activities in place, and we revised text in several locations to provide further clarification on Labor’s activities. We clarified that all four Labor projects in Indonesia, Mexico, and Thailand clearly link goals and activities, specify targets, and explain how targets were set. We added language stating that Labor engaged ILO’s external auditor to conduct audits of a sample of ILO’s projects to eliminate child labor and contracted with a certified public accounting firm to conduct independent attestation engagements of its Education Initiative projects. We believe the overall monitoring of antitrafficking projects is limited because the projects funded by the other five agencies did not have the elements of monitoring we found in Labor’s projects. 2. Labor stated that it requires midterm and final evaluations for its technical assistance projects, and that, in some cases, it engages in longer-term follow-up studies. We revised the text to further clarify and recognize Labor’s evaluation requirements and activities. In addition, we added text specifying that Labor funds follow-up studies to document changes that have occurred to a sample of beneficiaries over time. 3. Labor agreed with the need to carry out more systematic impact evaluations, but also emphasized that process evaluations are an important prerequisite of impact evaluations. Labor further commented that randomized trials are not appropriate for all projects, and requested that we specify where agencies should conduct randomized trials and where other methods would be more appropriate. We agree that process evaluations are important and revised the text to further differentiate process evaluations from impact evaluations. We believe each agency should determine whether randomized trials are appropriate for the specific project they are evaluating. 4. We made changes to the draft of this report on the basis of Labor’s specific technical comments, where appropriate.

Comments from the United States Agency for International Development

Comments from the Department of Health and Human Services

GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

Cheryl Goodman, Assistant Director; Jeremy Latimer; Christina Werth; Todd M. Anderson; Gergana Danailova-Trainor; Terry Richardson; and Debbie Chung made key contributions to this report. In addition, Elizabeth Curda, Bruce Kutnick, Mary Moutsos, Barbara Stolz, and Susanna Kuebler provided technical or legal assistance.