Human Trafficking: Monitoring and Evaluation of International Projects Are Limited, but Experts Suggest Improvements
Published: 2007-07-26T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: Human trafficking--a worldwide crime involving the exploitation of men, women, and children for others' financial gain--is a violation of human rights. Victims are often lured or abducted and forced to work in involuntary servitude. Since 2001, the U.S. government has provided about $447 million to combat global human trafficking. As GAO previously reported, estimates of the number of trafficking victims are questionable. In this report, GAO examines (1) collaboration among organizations involved in international antitrafficking efforts, (2) U.S. government monitoring of antitrafficking projects and difficulties in evaluating these projects, and (3) suggestions for strengthening monitoring and evaluation. GAO analyzed agency documents; convened an expert panel; interviewed officials; and conducted fieldwork in Indonesia, Thailand, and Mexico.
section_title: What GAO Found
paragraphs: While governments, international organizations, and nongovernmental organizations have recognized the importance of collaborating and have established some coordination mechanisms and practices, they will need to overcome challenges that have impeded collaboration in the past for their efforts to be successful. In two of the three countries GAO visited, it found that host governments--which bear ultimate responsibility for combating trafficking within their borders--have passed national antitrafficking laws and enacted national action plans. However, organizations continue to face numerous challenges when collaborating to combat human trafficking, including varying levels of government commitment and capacity. For example, some governments treat foreign trafficking victims as illegal immigrants and deport rather than protect them. In addition, according to officials in two of the three countries GAO visited, the ministries responsible for coordinating antitrafficking efforts have limited authority and capacity. U.S. government-funded antitrafficking projects often lack some important elements that allow projects to be monitored, and little is known about project impact due to difficulties in conducting evaluations. Project documents GAO reviewed generally include monitoring elements, such as an overarching goal and related activities, but often lack other monitoring elements, such as targets for measuring performance. To oversee projects, State officials supplement their efforts with assistance from U.S. embassy staff, but have not established written guidance for oversight. Officials said that they are working to improve performance measures and develop monitoring guidance. Conducting impact evaluations of antitrafficking projects is difficult due to several factors, including questionable project-level estimates of the number of trafficking victims. These estimates are needed for baselines by which to evaluate how effectively specific interventions are reducing trafficking. Elements in the design of certain projects, such as objectives that are too broad, further impede evaluation. Because of these difficulties, few impact evaluations have been completed, and little is known about the impact of antitrafficking interventions. A GAO-convened panel of experts identified and discussed ways to address the factors that make it difficult to monitor and evaluate antitrafficking projects. Panelists' suggested approaches included improving information on the nature and severity of trafficking and addressing monitoring and evaluation in project design. To improve information on trafficking, panelists suggested methods that have been used to sample other hard-to-reach populations, including domestic violence victims. One suggested method is sampling of "hot spots"--an intensive search for victims in areas known to have high concentrations of victims. To address weaknesses in project design that impede monitoring and evaluation, panelists suggested that officials design projects that clearly link activities to intended outcomes, identify measurable indicators, and establish procedures for setting and modifying targets.

section_title: Letter
paragraphs: Human trafficking—a worldwide crime involving the exploitation of men,  women, and children for others’ financial gain—is a violation of  fundamental human rights. Victims are often lured or abducted from their  homes and subsequently forced, through various means, to work in  prostitution, sweatshops, agricultural settings, or domestic service, among  other types of servitude. In addition to inflicting grave personal damage  upon its victims, trafficking undermines government authority, fuels  organized criminal groups and gangs, and imposes social and public health  costs. As we have previously reported, estimates of the number of  trafficking victims are questionable due to data and methodological  weaknesses.
To combat global human trafficking, the U.S. government directly  implements projects overseas and contributes financial support to the  United Nations (UN) and other international organizations. The U.S.  government has strongly supported antitrafficking efforts and, since 2001,  has provided approximately $447 million in foreign assistance to  nongovernmental organizations (NGO), international organizations, and  foreign governments to combat and help eliminate human trafficking. In  addition, since the mid-1990s, the United States has played a leading role in  putting human trafficking on the international community’s agenda. In  2000, Congress enacted the Trafficking Victims Protection Act (TVPA) to  combat trafficking in persons; in 2003 and 2005, Congress reauthorized the  act, which is due for reauthorization in 2007. In December 2005, the United  States became a party to the United Nations’ Protocol to Prevent, Suppress  and Punish Trafficking in Persons, Especially Women and Children,  Supplementing the United Nations Convention Against Transnational  Organized Crime, which entered into force in 2003 and, as of July 2007,  has 117 signatories and 113 party nations.
This review is part of a larger body of work that you requested on U.S. and  international efforts to combat human trafficking in the United States and  abroad. To review efforts to combat human trafficking, we examined   (1) collaboration among organizations involved in international  antitrafficking efforts, (2) U.S. government agencies’ monitoring of  antitrafficking projects and difficulties in evaluating these projects, and   (3) suggestions for strengthening monitoring and evaluation.
To address these objectives, we reviewed relevant planning, funding, and  project documents on human trafficking from the Departments of State,  Justice, Labor, Homeland Security (DHS), and Health and Human Services  (HHS) and the U.S. Agency for International Development (USAID). We  also reviewed planning and project documents from relevant UN and other  international agencies and offices, including the UN Office on Drugs and  Crime (UNODC), International Labor Organization (ILO), International  Organization for Migration (IOM), United Nations Children’s Fund  (UNICEF), and UN High Commissioner for Refugees (UNHCR). In  addition, we discussed international antitrafficking efforts with officials  from the above agencies and organizations, as well as foreign government  officials and NGO representatives, in Washington, D.C., and during our  fieldwork in Indonesia, Thailand, and Mexico. We selected these countries  on the basis of the amount of U.S. funding provided for international  antitrafficking projects and the number of U.S. government agencies and  international organizations working in each country. In April 2007, we  worked with the National Academy of Sciences to organize a 2-day expert  panel on challenges and alternative strategies for monitoring and  evaluating international antitrafficking projects. We conducted our review  from August 2006 to June 2007 in accordance with generally accepted  government auditing standards. Appendix I provides a detailed description  of our objectives, scope, and methodology.
subsections: section_title: Background
paragraphs: Human trafficking generally involves the use of force, fraud, or coercion to  enslave individuals in situations that are exploitative and often illegal and  dangerous. Since 2001, the U.S. government has contributed approximately  $447 million worldwide to foreign governments, NGOs, and international  organizations—such as UNODC, ILO, and IOM—to combat human  trafficking. U.S. agency and international organization projects generally  aim to prevent trafficking, protect victims, and prosecute traffickers.  Organizations have different requirements for monitoring and evaluating  their antitrafficking projects.
subsections: section_title: Human Trafficking Cases Often Follow a Similar Pattern
paragraphs: Although the crime of human trafficking can take different forms in  different regions and countries around the world, most human trafficking  cases follow a similar pattern—that is, traffickers use acquaintances or  false advertisements to recruit men, women, and children in or near their  homes, and then transfer them to and exploit them in another city, region,  or country.According to the TVPA, victims of severe forms of trafficking  include those who are recruited or transported for labor through the use of  force, fraud, or coercion and for the purpose of subjecting them to  involuntary servitude. The use of fraud, force, or coercion typically  distinguishes human trafficking victims from individuals who are  smuggled. In some cases, individuals may enter freely into agreement with  and pay smugglers to help them cross international borders. After arriving  at their destination, however, individuals who do not understand the  destination country’s language or culture may be exploited by individuals  who take advantage of their vulnerability.
Traffickers control their victims’ living and working conditions by  physically confining them, taking away their identity documents, and  threatening their families. Traffickers also exploit victims’ fears that  authorities will prosecute or deport them if they seek help. Victims may be  forced to work in legal or illegal and often dangerous areas, including  brothels, sweatshops, agricultural businesses, and people’s homes.  Documented human trafficking cases have involved victims such as  children forced to beg for money in cities, work in carpet shops, and  participate in pornography and sex acts with adults; women held in slavery- like conditions as domestic servants, strip club dancers, and prostitutes;  and men forced to perform work in the agricultural sector and on fishing  vessels.
section_title: U.S. Government and Multilateral Organizations’ Projects Focus on Three Areas
paragraphs: To combat the diverse forms of global human trafficking, 5 U.S.  departments, USAID, and at least 15 international organizations have  provided assistance to governments and civil society organizations in more  than 100 countries. Assistance generally aimed to enhance efforts to  1. prevent human trafficking through public awareness, outreach,  education, and advocacy campaigns;  2. protect and assist victims by providing shelters as well as health,  psychological, legal, and vocational services; and  3. investigate and prosecute human trafficking crimes by providing    training and technical assistance for law enforcement officials, such as  police, prosecutors, and judges.
These categories of interrelated victim-centered assistance activities— prevention, protection, and prosecution—are commonly referred to as “the  three p’s.” Each type of assistance is viewed as critical for reducing the  incidence of human trafficking.
Since 2001, U.S. government agencies have provided approximately $447  million in foreign assistance to international organizations, NGOs, and  foreign governments to combat human trafficking (see table 1). Of the U.S.  agencies, State, Labor, and USAID have provided the most funding to  combat global human trafficking.
As we have previously mentioned, U.S. agencies support antitrafficking  projects implemented worldwide by various international organizations,  which also receive funding from other donor governments and  organizations. As shown in table 2, for UNODC, ILO, and IOM, total  resources allocated to combating trafficking since 2000 totaled about $255  million. The U.S. government provided about $122 million to UNODC, ILO,  and IOM to combat human trafficking, according to data provided by these  organizations and State’s Office to Combat and Monitor Trafficking and  Persons (G/TIP). In addition, according to UNICEF’s annual reports,  between 2003 and 2005, UNICEF allocated more than $453 million to its  worldwide child protection program, which includes projects to combat  trafficking and the sexual exploitation of children.
Some U.S. agencies and international organizations have provided  assistance that, although not specifically related to human trafficking, may  help to combat trafficking by reducing individuals’ vulnerability to  becoming victims and by strengthening countries’ judicial systems. For  example, USAID, the World Bank, the Asian Development Bank, and the  Inter-American Development Bank have implemented projects to root out  corruption, build host governments’ legal systems, and improve the  economic conditions of vulnerable populations in developing countries.  See appendix II for additional information on international organizations’  missions and antitrafficking activities.
section_title: U.S. Agency and International Organizations’ Monitoring and Evaluation Requirements Vary
paragraphs: Monitoring and evaluation are important tools for managing projects and  should be considered when designing projects. Performance measurement  involves developing a logic framework to explain how an intervention is to  achieve its intended outcomes. Monitoring helps agencies determine  whether the project is meeting its goals, update and adjust interventions  and activities as needed, and ensure that funds are used responsibly.  Monitoring includes, among other things, the development of indicators  that are linked as closely as possible to the variables identified in the logic  framework. These indicators are to be used throughout implementation to  assess whether the project is likely to achieve the desired results.  Monitoring also involves the choice of baseline and target values for each  indicator and includes plans for periodic performance reports and data  quality reviews. Evaluation is needed to assess a project’s impact or  effectiveness. Project evaluation involves the development of a  methodology that will be used to assess the project’s impact and describes  plans for collecting baseline, interim, and final data on project results.
The TVPA requires the President’s Interagency Task Force to Monitor and  Combat Trafficking in Persons to measure and evaluate the progress of the  U.S. government’s efforts to combat trafficking. The Trafficking Victims  Protection Reauthorization Act of 2003 (TVPRA 2003) includes a  congressional finding that additional research is needed to fully understand  the phenomenon of trafficking in persons and to determine the most  effective strategies for combating trafficking. The TVPRA 2003 also  requires an annual report from the U.S. Attorney General to Congress to  provide information on U.S. government activities to combat trafficking in  persons. In addition to these reports, Justice began preparing annual  assessments of U.S. government activities to combat trafficking in persons  in 2003. All six U.S. organizations’ guidelines also require implementers to  monitor and report on project performance. In addition, Labor also  requires independent midterm and final evaluations for all international  antitrafficking projects.
section_title: Organizations Have Taken Steps to Collaborate, but Continue to Face Challenges
paragraphs: Global, regional, and country-level organizations recognize the importance  of collaborating to effectively combat human trafficking. Although  organizations involved in combating trafficking—governments, multilateral  organizations, and donors—have implemented some practices to  strengthen collaboration, to succeed they will need to overcome challenges  that can impede collaboration, including varying levels of government  commitment and capacity.
subsections: section_title: Global, Regional, and Country-Level Organizations Recognize the Importance of Collaborating to Effectively Combat Human Trafficking
paragraphs: At the global level, several UN organizations have acknowledged the  importance of collaborating to effectively combat human trafficking. The  UN Chief Executives Boardrecognized the challenges in countering  human trafficking and proposed establishing an interagency mechanism to  strengthen coordination in 2005. In July 2006, a UN Economic and Social  Committee resolution requested that UNODC organize a meeting to  coordinate the technical assistance that UN and other intergovernmental  organizations provide. In December 2006, the UN General Assembly  adopted a resolution recognizing that broad international cooperation  between member states and intergovernmental and nongovernmental  organizations is essential for effectively countering the threat of human  trafficking, and underlined the importance of bilateral, subregional, and  regional partnerships, initiatives, and actions. This resolution further  encouraged member states to initiate and develop working-level contacts  among countries of origin, transit, and destination, especially among  police, prosecutors, and social authorities.
Organizations at the regional level have also recognized the importance of  collaboration. For example, in a September 2006 report, the Organization  for Security and Cooperation in Europe’s (OSCE) antitrafficking unit noted  that, since trafficking is a transnational crime, collaboration is crucial to  enable transnational mechanisms of communication and cooperation  among governments, law enforcement, judiciary, and NGOs. In addition, six  countries in the Mekong subregion of Southeast Asia have called for  strengthened cooperation to combat human trafficking under a process  called the Coordinated Mekong Ministerial Initiative Against Trafficking  (COMMIT).
At the country level, donor governments and UN organizations have also  recognized the need to establish coordination mechanisms to, among other  things, share information and leverage the comparative advantages of each  organization. The Paris Declaration of Aid Effectiveness establishes the  importance of donor coordination, stating that excessive fragmentation of  aid at the global, country, or sector level impairs its effectiveness. It further  states that a pragmatic approach to the division of labor and burden  sharing increases complementarity and can reduce costs. Through the  declaration, donors committed to make use of their respective comparative  advantage at a sector or country level by delegating authority to lead  donors for the execution of projects, activities, and tasks. To draw on the  collective strengths of the UN agencies and programs operating in a  country, UN country teams and the host government develop a national  analysis, called a common country assessment. They subsequently produce  a national development assistance framework, which describes the  collective UN response and the expected results to achieve national  priorities.
For U.S. government agencies, a 2002 presidential directive stated that  strong coordination among agencies working on domestic and foreign  policy is crucial. The directive called for departments and agencies to  coordinate U.S. foreign assistance programs, including those that provide  funding to governmental or nongovernmental organizations to combat  trafficking in persons. State officials told us that this is done through the  Senior Policy Operating Group (SPOG), which, among other activities,  facilitates a review by SPOG programming agencies of each other’s grant  proposals for antitrafficking projects.
section_title: Organizations Face Challenges in Collaborating to Combat Trafficking
paragraphs: Organizations—governments, multilateral organizations, and NGOs—face  several challenges in collaborating to combat human trafficking, including  the following:  Government commitment varies. State’s annual Trafficking in Persons  Report, which analyzes and ranks foreign governments’ compliance  with minimum standards to combat trafficking, as outlined in the TVPA,  illustrates governments’ varied efforts to address the issue. For  example, governments might not recognize trafficking as a problem.  They may treat foreign trafficking victims as illegal immigrants and  deport them back to their home countries, rather than protect them.  They also may not recognize trafficking within their own borders as a  problem. Moreover, some government officials are themselves involved  in human trafficking.
Government capacity varies. Governments with greater resources and  more established institutions have a greater capacity to address  trafficking than countries that are poorer or less stable. In addition,  changes in government leadership and personnel, due to elections,  coups, assassinations, or other events, may result in the loss of expertise  in combating human trafficking. Furthermore, governments may put  trafficking under the purview of ministries with limited authority, such  as women’s ministries.
Organizations that combat trafficking vary in perspective and may be  in competition for limited funds. Combating trafficking involves  organizations—at the international and country levels—with expertise  in raising awareness, assisting child and adult victims, and investigating  trafficking cases, among other areas. Organizations may view trafficking  through their own mandates or viewpoints and may perceive each other  not only as collaborators but also as competitors for scarce resources.  As such, they may not share information, which could lead to  duplication and waste of funds.
Understanding of trafficking varies across languages and cultures.
Countries approach trafficking in different ways. For example, some  countries’ national antitrafficking strategies do not include men as  potential trafficking victims; also, in some countries, parents in villages  often sell their children to work as domestic servants in large cities.  Furthermore, translation issues can complicate establishing a common  understanding of what constitutes trafficking. For example, although  the legal term in Spanish for human trafficking is “trata de personas,”  English speakers have translated human trafficking into Spanish as  “tráfico de personas,” which in fact means human smuggling.
section_title: Organizations Have Taken Steps to Strengthen Collaboration, but Challenges Remain at the Global, Regional, and Country Levels
subsections: section_title: Global and Regional Collaboration Efforts Have Recently Been Initiated
paragraphs: Organizations at the global level have defined a common outcome—to end  human trafficking and slavery—and have recently initiated various efforts  to strengthen collaboration. In March 2007, UNODC launched the Global  Initiative to Fight Human Trafficking to generate political will, an action  plan, and financial resources to combat trafficking worldwide. The steering  committee of the initiative includes the six leading international  organizations involved in combating trafficking in persons: UNODC, ILO,  IOM, UNICEF, OSCE, and the Office of the United Nations High  Commissioner for Human Rights. UNODC also created the Interagency  Cooperation Group Against Trafficking in Persons in September 2006 with  the intended outcome of improving coordination between UN agencies and  other international organizations, and to facilitate a holistic approach to  preventing and combating human trafficking. The group’s functions include  exchanging information and promoting the effective and efficient use of  resources.
Most existing regional coordination efforts that we reviewed have defined a  common outcome and established action plans for carrying out  antitrafficking activities. These efforts vary regarding whether they are  specifically directed against trafficking or are included in discussions on  migration or smuggling. In addition, they vary in number of participating  members from as few as 6 countries in the Mekong subregion to as many as  56 countries in Europe, Asia, and North America. For example, the  governments involved in a UN project that specifically focused on  strengthening cooperation to combat trafficking in the Mekong subregion  have agreed to undertake 18 actions to combat human trafficking, as part  of an effort called the COMMIT process. Officials stated that this  agreement strengthens these countries’ incentives to undertake the actions  through mutual accountability. In addition, COMMIT developed an action  plan that groups existing activities into a single framework delineating  roles and responsibilities of UN organizations, implementing partners, and  governments in the six Mekong countries. Officials contrasted the  COMMIT process, which includes a small group of countries in one region  with interconnected trafficking problems, to larger organizations such as  the Bali Process, which approaches trafficking issues in conjunction with  issues related to smuggling in the Asia-Pacific region and has a larger  geographic scope, consisting of 38 governments across several regions.  These officials stated that having a larger geographic scope makes it more  difficult for the governments of these countries to hold each other  accountable for implementing the regional action plan. In addition to these  examples, OSCE initiated the Alliance against Trafficking in Persons, which  established a partnership with major actors working to combat human  trafficking to, among other things, develop joint strategies and provide  OSCE participating states and others with harmonized decision-making  aids. In Southeastern Europe, the International Center for Migration Policy  Development has begun a project to facilitate the creation of a  transnational referral mechanism by 10 governments in the region and  national NGOs to improve transnational case management and victim  protection. Furthermore, in the western hemisphere, the Organization of  American States (OAS) held a conference in 2006 that resulted in guidelines  for the 34 member states and OAS to combat human trafficking.
Organizations continue to face obstacles as they work on global and  regional initiatives to collaborate to combat trafficking. For example,  governments disagree on whether there is a difference between “forced”  and “voluntary” prostitution. Such disagreements can hinder collaborative  efforts in combating sex trafficking. These global and regional initiatives  may eventually implement additional practices to enhance collaboration,  but it is too early to determine the success of the current initiatives.
section_title: Country-Level Organizations Have Taken Some Steps to Strengthen Collaboration, but Continue to Face Difficult Challenges
paragraphs: Although organizations—including host governments, UN organizations,  U.S. government agencies, and other donor governments—in the three  countries we visited have implemented practices to strengthen  collaboration in combating trafficking, they continue to face challenges.  None of the three countries has mechanisms to coordinate the efforts of all  organizations involved in combating trafficking. Host governments bear  ultimate responsibility for combating trafficking within their borders, and  governments of the countries we visited have taken some steps to  collaborate. For example, the Indonesian and Thai governments have  passed national antitrafficking laws and enacted national action plans that  define common outcomes, outline strategies, and assign roles and  responsibilities. However, neither government’s action plan includes  trafficking of men in its definition of human trafficking. Officials in  Indonesia stated that they expect to revise the plan to include the  trafficking of men, as well as women and children, based on legislation  passed in 2007. Although both the Indonesian and Thai governments hold  interagency meetings, the ministries responsible for coordinating  antitrafficking efforts have limited authority and operational capacity,  according to officials we interviewed. Unlike Indonesia and Thailand,  Mexico has neither a national antitrafficking law nor an action plan.  Mexican officials stated that they convened one interagency meeting on  human trafficking and plan to institute additional coordination  mechanisms after the government passes a national antitrafficking law.
In Indonesia and Thailand, UN organizations, in conjunction with host  governments, have developed country assessments and assistance  frameworks to articulate overall development goals—including combating  trafficking—joint strategies, and roles and responsibilities. UN  organizations working on trafficking in Thailand also meet as part of the  Mekong regional project that we previously discussed; although, according  to one donor government official, attendance among UN organizations is  sporadic. UN officials in Indonesia told us that they share information  informally, but do not meet on a regular basis to discuss trafficking.
According to U.S. government officials in Indonesia and Thailand, donor  governments have made sporadic and informal efforts to leverage  resources and avoid duplication of effort. For example, as a result of  informal coordination, Justice and French government officials worked  together on a criminal justice training project in Indonesia. In Thailand,  according to a U.S. official, although most of the major donors attend the  Mekong regional project’s meetings, these meetings are generally focused  on UN activities, with little opportunity for donors to coordinate. Officials  in Indonesia and Thailand acknowledged the need to establish regular  bilateral donor coordination efforts on trafficking issues and, at the time of  our fieldwork, had begun discussing the establishment of regular  coordination meetings.
While U.S. government agencies overseas have developed some  collaboration mechanisms to combat trafficking, they continue to face  challenges in coordinating their efforts. The U.S. embassies in the three  countries we visited, Indonesia, Thailand, and Mexico, include trafficking  in persons in their mission performance plans, which establish combating  trafficking as a component of the U.S. government’s overall strategy in each  country. U.S. officials in these countries told us they organize trafficking- specific meetings that include U.S. government agencies and their  implementing partners, primarily to share information. The U.S.  government officials who were involved in antitrafficking issues were  responsible for other issues as well. In Indonesia, for example, the primary  U.S. embassy contact for trafficking was also temporarily assigned deputy  chief of mission in East Timor. In Mexico, one agency official, whose  responsibilities consisted solely of human trafficking issues, had been  named to also serve as coordinator and primary contact for all U.S.  antitrafficking efforts. After this official's departure in December 2006, her  portfolio was added to the existing portfolio of another official from the  same U.S. agency. At the time of our visit in February 2007, State, USAID,  and DHS officials, who also covered other issues, were holding meetings to  coordinate their antitrafficking efforts internally. However, a new U.S.  antitrafficking coordinator for Mexico had not been designated.  Consequently, some Mexican government officials expressed uncertainty  about which U.S. agency official had the role of lead U.S. government  coordinator for trafficking in Mexico.
Furthermore, some U.S. government officials noted challenges in  coordinating between officials in Washington, D.C., and those overseas.  For example, a U.S. official in Thailand was unaware of a U.S.-funded  project in the country until that project hosted a conference and requested  additional funding from the U.S. embassy. The project had not been  included in the list of antitrafficking activities in Thailand that the official  received from State’s trafficking office, which is based in Washington.  According to some agency officials overseas, not knowing what other  agencies and bureaus plan to spend on antitrafficking activities in a  particular country makes it difficult for them to determine how much of  their budgets to allocate to antitrafficking activities. Although State’s new  Office of Foreign Assistance has begun to address the issue of better  coordinating all U.S. foreign assistance by bringing together core State and  USAID teams to discuss U.S. development priorities in each recipient  country, some U.S. officials expressed uncertainty regarding which part of  the U.S. government would be responsible for outlining a new country-level  strategy and budget for combating trafficking.
section_title: U.S. Government- Funded Antitrafficking Projects Lack Some Key Monitoring Elements; Little Is Known about Project Impact Due to Difficulties in Conducting Evaluations
paragraphs: Antitrafficking project documents we reviewed generally include  monitoring elements, such as an overarching goal and related activities;  however, they often lack other monitoring elements, such as targets for  measuring performance. Various other factors also make it difficult to  evaluate the impact of antitrafficking projects. These factors include  questionable estimates of the number of trafficking victims at the project  level, which are needed to evaluate the effectiveness of specific  antitrafficking interventions. Certain project elements may further impede  evaluations, including short time frames and overly broad objectives.  Because of these factors, few evaluations that determine impact have been  completed. As a result, little is known about the impact of antitrafficking  interventions.
subsections: section_title: U.S. Government-Funded Antitrafficking Projects Lack Some Important Monitoring Elements; State Is Taking Steps to Strengthen Monitoring
paragraphs: Most of the project documents we reviewed generally include one or more  monitoring elements, but lack others. We reviewed documents for 23 U.S.  government-funded antitrafficking projects in Indonesia, Thailand, and  Mexico, which generally include statements of project goals and a  description of activities. However, the majority of these documents lack a  logic framework that clearly links activities with project-level goals,  indicators, and targets. Specifically:  Eighteen of the 23 projects do not clearly explain how activities will  achieve stated goals. For example, the project proposal for a State- funded project in Thailand does not provide clear linkages  demonstrating how the training of local officials and screenings of a  public awareness video will achieve the project’s overall goal of  reducing trafficking among vulnerable adolescents and women. In  contrast, all 4 Labor-funded projects in Indonesia, Mexico, and Thailand  clearly link project goals and activities. For example, 1 goal of a project  in Mexico is to develop a system to identify networks of exploiters.  Activities to achieve this goal include developing computer software  and training government officials in its use.
Twenty-one of the 23 projects identify indicators, but of these, only 10  specify targets by which performance is measured. For example, a  State-funded project in Thailand included the “number of victims  rescued” and the “number of arrests of traffickers” as performance  indicators, but did not set numerical targets for measuring performance.  In contrast, a State-funded project implemented by IOM in Indonesia  established an expected result that 500 victims of trafficking receive  rehabilitation and reintegration assistance. For the 13 projects that did  not specify targets, the performance standards to which grantees were  held accountable were not clear. Of the 10 projects that did specify  targets, 5 explained how targets were set. These 5 projects included all 4  of the Labor projects we reviewed.
Another element of monitoring project performance is to supplement  implementing partners’ reporting of programmatic and financial progress  with independent review through site visits at the field level. For example,  Labor officials reported that the International Labor Affairs Bureau’s Office  of Child Labor, Forced Labor, and Human Trafficking has engaged ILO’s  external auditor to conduct audits of a sample of ILO’s international  program for the elimination of child labor projects. This office also has  contracted with a certified public accounting firm to conduct independent  attestation engagements of its Education Initiative projects. Among the  objectives of these audits and attestation engagements is an assessment of  the accuracy and reliability of performance data from grantees’ progress  reports.
State G/TIP has 4 Washington, D.C.-based staff members who are  responsible for overseeing projects in approximately 70 countries. G/TIP  officials stated that because they cannot visit all of the projects, they rely  on U.S. embassy staff to provide additional field-level oversight. However,  the office has not established written guidance for conducting such  oversight. During our fieldwork, an embassy official expressed frustration  at the lack of clear guidance and procedures. Embassy officials also told us  that they meet with project staff, but have other responsibilities that limit  the time they can devote to overseeing antitrafficking projects. According  to State’s Bureau of Population, Refugees,and Migration (PRM) officials,  the bureau has only 3 staff to monitor and oversee its antitrafficking  projects currently being implemented in over 20 countries, and these staff  cannot conduct site visits to all projects. PRM staff also stated that they  review progress reports submitted by project implementers, but do not  have a database or system for compiling the information they receive from  the field.
To address some of these concerns, State officials told us that they are  taking steps to strengthen monitoring, including developing new indicators  and written guidance for monitoring antitrafficking projects. State G/TIP  officials told us that they are developing a system of key indicators to  better inform management decision making that would be consistently  used for each of the three main types of antitrafficking programs— prevention, protection, and prosecution. State PRM and IOM, its key  implementing partner, have also been involved in an effort to develop and  standardize an indicator framework across IOM missions, although this  effort was not completed during the time of our review. Officials stated that  this effort will serve as a way to exchange ideas about best practices in  different parts of the world and will provide managers with information on  implementation needed to make decisions about current or future projects.
section_title: Factors Impede Evaluations of Antitrafficking Projects and Little Is Known about Project Impact
paragraphs: Various factors impede impact evaluations of antitrafficking projects. First,  data on human trafficking are questionable, including estimates of the  number of trafficking victims, making it difficult to determine a preproject  baseline. Second, elements of project design, such as overly broad  objectives or short project duration, diffuse potential impact. Because of  these factors, it has been difficult to evaluate the effectiveness of  antitrafficking interventions, and few evaluations that determine impact  have been completed. As a result, little is known about the impact of  antitrafficking interventions.
subsections: section_title: Lack of Project-Level Baseline Estimates Limits Evaluations
paragraphs: The lack of accurate baseline estimates of the number of trafficking victims  is a limitation to conducting evaluations. As we have previously reported,  the accuracy of current estimates is questionable. Without estimates of  the scope of human trafficking to use as baselines in project locations, it is  very difficult to determine where interventions are most needed or where  interventions would have the greatest impact. Developing baseline  estimates is difficult for the following reasons:  Victims are a hidden population. Victims may be unaware, unwilling,  or unable to acknowledge that they are trafficking victims. Therefore, it  is difficult to reach them to collect information using standard sampling  techniques.
Service providers may be unwilling to share victim data due to  confidentiality concerns. For example, the global database maintained  by IOM is not publicly available since assisted victims are in a  precarious position and revealing their identity could have a detrimental  effect on their safety.
The definition of the term “trafficking in persons” is broad and varies  in meaning across different languages. As we previously discussed,  the understanding of trafficking varies across languages and cultures.  These variations in definition hinder the comparability of data across  countries and organizations.
There are no commonly agreed-upon criteria for identification of  human trafficking victims. This lack of criteria hinders the ability to  identify victims, create consistent statistical databases, and design  analytical tools for surveys and estimates.
Existing data may not be reliable. Developing countries, which are  typically the countries of origin, have limited capacity for data  collection, and their governments’ commitment to combating trafficking  may be insufficient. Thus, sufficiently reliable data needed for  estimating trafficking incidence may not be available.
section_title: Elements of Project Design Impede Evaluations
paragraphs: Elements in the design of certain antitrafficking projects, including a  tendency to focus on very broad, high-level objectives across a diverse  range of activities, diffuse projects’ potential impact and create challenges  for evaluations.
When projects focus on overly broad objectives and contain too many  types of activities, their potential impact is diffused, which makes them  difficult to evaluate. For example, some antitrafficking projects have very  high-level goals or objectives, such as “creating an environment for  effective action against trafficking,” or “strengthening the initiatives of  government and others against human trafficking” and contain activities  covering 2 or more of the 3 key types of antitrafficking interventions— prevention, protection, and prosecution. Of 153 U.S. international  antitrafficking projects funded in fiscal year 2004, 2005, or 2006, 56 percent  included 2 or more of these interventions and 29 percent included all 3  interventions. Activities included public awareness campaigns, victim  assistance, and training for law enforcement officials.
While projects funded with greater resources could lead to more noticeable  longer-term changes, the impact of a shorter-term, smaller-scale  intervention may be difficult to attribute and quantify. Antitrafficking  projects vary significantly in terms of their time frames and funding levels.  For example, the projects we reviewed varied in duration from 1 year to  about 4 years, and projects’ funding levels varied from $15,000 to $6  million. Officials stated that some organizations, such as State G/TIP, tend  to fund smaller, shorter-term projects, while Labor usually funds larger,  longer-term projects. Overall, experts told us that organizations also have  generally limited funding for impact evaluations and for research on the  nature and scope of human trafficking.
Experts stated that another factor impeding evaluations is that little  attention has generally been devoted at the start of projects to evaluation  design. For example, questions related to determining the control group,  the type of design for impact evaluation, the data that would be collected,  and the analytical methods that would be most suitable are often not  addressed before implementation. Furthermore, experts stated that  although projects target several groups of beneficiaries, it is generally  unclear how they can be reached and to which group they would be  compared. As a result, it is difficult to determine what would constitute  successful project implementation.
section_title: Little Is Known about Project Impact
paragraphs: Because of the difficulties in evaluating antitrafficking projects, the few  evaluations that have been completed are qualitative rather than  quantitative, focus on process rather than impact, and rarely trace victims  over time.
The few evaluations completed used qualitative methods that are valuable  for documenting victims’ perceptions and experiences, but, unlike  quantitative methods, they cannot be used to generalize results to broader  populations. For example, IOM’s Office of Inspector General has carried  out impact evaluations of a select number of antitrafficking projects. Our  review of these evaluations shows that they covered interventions in  various parts of the world funded by different donors. All of the available  evaluations applied qualitative methodologies, consisting of document  reviews, site visits, interviews, and focus groups with stakeholders. In  addition, in a USAID-funded assessment of child trafficking victims  residing at a shelter in Nepal, an evaluator used qualitative methods, such  as observations and interviews, to collect information on their daily lives,  needs, and preparation for reintegration into the community. The evaluator  concluded that skill-based training provided by the shelter does not  adequately prepare the girls for their reintegration into their communities;  however, this qualitative result could not be generalized to the national  level.
Completed evaluations also typically focused on process rather than  impact. Process evaluations are an important tool for improving service  delivery and assessing program effectiveness by assessing whether  activities conform to program design. Impact evaluations assess the net  effect of a program by comparing program outcomes with an estimate of  what would have happened in the absence of the program. However,  impact evaluations are an emerging field in antitrafficking interventions.  For example, a process evaluation of a Labor-funded project aimed at  reducing the victimization of minors who have been trafficked or are at risk  of being trafficked in Thailand focused on the project’s planning,  implementation, and management. This process evaluation was designed to  monitor implementation and assess project outcomes. However, the  evaluation did not ultimately assess the project’s impact in reducing the  victimization of minors.
Finally, our review identified few evaluations that track individual victims  who have been reintegrated into their communities over time. For example,  ILO is currently pilot-testing an application of a tracer study methodology  in the context of trafficking prevention. In addition, according to Labor  officials, in some cases the office has conducted follow-up studies that  examine a sample of beneficiaries to document changes that have  occurred. Furthermore, IOM officials noted that several of its missions  provided statistical information to gauge the success of its victim  reintegration efforts over time based on specific indicators. According to  experts, such evaluations are costly and time-consuming. Tracking victims  by country can also be difficult because country boundaries are permeable  and many trafficking routes cross national borders.
section_title: Expert Panel Identified Ways to Address the Difficulties of Monitoring and Evaluating Antitrafficking Projects
paragraphs: A GAO-convened panel of experts identified and discussed ways to address  the factors that make it difficult to monitor and evaluate antitrafficking  projects. Panelists suggested approaches to improve the monitoring and  evaluation of antitrafficking projects, by improving information on the  severity of human trafficking and addressing weaknesses in the design of  antitrafficking projects.
subsections: section_title: Conduct Research on the Nature and Severity of Human Trafficking
paragraphs: Panelists acknowledged that the lack of existing data on the nature and  severity of human trafficking limits evaluations of antitrafficking projects,  and suggested ways to improve information on the nature and severity of  human trafficking. According to the panelists, researchers need to gather  evidence to answer the following questions:   What is the nature and severity of human trafficking? Panelists stated  that it is necessary to understand the nature of trafficking in terms of its  underlying conditions and the types of traffickers and victims.  Trafficking is a multidimensional, complex problem that involves a wide  range of victims; recruiters, brokers, and intermediaries; and abusive  employers and sexual exploiters. Understanding the incentives of the  people engaged in trafficking is an important first step.
The severity of human trafficking can be measured by using qualitative  and quantitative methodologies. Panel members suggested several  sampling methods that have been used to sample other hard-to-reach  populations, including the homeless, hidden migrants, missing and  exploited children, domestic violence victims, inmates, and drug users.  One suggested method is sampling of “hot spots”—an intensive search  for victims in areas known to have high concentrations of victims or in  areas to which many victims return. Other methods include adaptive  cluster, double, indirect, and snowball sampling. (For a more detailed  discussion of sampling methods, see app. III.) These methods could be  used individually or in combination. Panelists further emphasized that,  whenever feasible, it is important to use methodologies that are  appropriate for the location sampled. In addition, it is critical to  determine whether the results are unique to a certain location or  whether they can be generalized to other locations. Panelists  recommended that research start at the local level. Such research would  identify successful small-scale interventions, increase the knowledge of  victims’ needs, and develop meaningful performance measures. Lessons  learned at the local level could then be expanded to national and  regional levels.
What is the projects’ estimated effect on the nature and severity of  human trafficking? Panelists emphasized the necessity of designing  rigorous methods to evaluate antitrafficking projects, such as  randomized control trials. For example, as baseline estimates in  trafficking hot spots are obtained and interventions are undertaken in  that location to reduce trafficking, the use of “place-randomized trials”  for evaluation would become possible. To determine the interventions’  impact, data obtained from interventions in hot spots could then be  compared with locations where there had been no interventions.  Although randomized trials may be difficult to execute for many  trafficking projects, they are important ways to generate evidence about  interventions’ effectiveness. Randomized trials should be pilot-tested in  carefully chosen settings so that the evaluator can identify and correct  any problems encountered before expanding the trials to larger  populations or areas. Panelists pointed out that such rigorous evaluation  has occurred in the fields of public health and criminal justice. For  example, a randomized study was done of brothels in Thailand to obtain  information on the HIV/AIDS rates of prostitutes.
section_title: Address Monitoring and Evaluation Weaknesses in the Design of Antitrafficking Projects
paragraphs: To address weaknesses in project design that impede monitoring and  evaluation, such as projects with very broad, high-level objectives,  panelists made the following recommendations:  Develop a logic framework. Given the weaknesses of some  antitrafficking projects that have overly broad objectives across a  diverse range of activities, panelists suggested that officials design  projects with a logic framework that has clear objectives and narrow the  focus of interventions. They also recommended that officials design  projects that clearly link activities to intended outcomes, identify  measurable indicators, and establish procedures for setting and  modifying targets. Measurable indicators with mutually agreed-upon  targets allow project officials to assess how a project is achieving its  overall goals and objectives. For example, a Labor-funded project  implemented by ILO in Mexico includes a logic framework that links  project activities and outputs to outcome objectives, and includes clear  indicators and means of verification. The project’s overall goal of  eliminating the commercial sexual exploitation of children in Mexico  links to four immediate objectives that, in turn, link back to project  activities and outputs. One objective—that at least 300 child victims of  sexual exploitation or at-risk children and their families receive  assistance—is linked to 28 activities and 8 outputs, such as increasing  families' employment opportunities. These 28 activities include  disseminating employment promotion programs, organizing  employment training, and monitoring and analyzing the impact of these  dissemination and training efforts. Panelists emphasized that donors  and implementers should agree on the project’s logic framework during  the design phase.
Determine whether a project is ready to be evaluated. Given the  significant variance in project duration and funding levels, panelists  emphasized that evaluators would not be able to evaluate all existing  projects, but should first determine which projects are ready to be  evaluated. In conducting such “evaluability assessments,” evaluators  determine, among other things, whether (1) the project is large enough,  has sufficient resources, and has been implemented long enough to  make an impact; (2) the project is reaching its target population; (3)  project documents specify and clearly link objectives, goals, and  activities; and (4) sufficient information exists to determine impact. For  example, larger, long-term projects are more likely to have an impact  and, thus, may be better candidates for evaluation than smaller, short- term projects. Because larger antitrafficking projects generally include a  diverse range of interventions, panelists suggested narrowing the  evaluation to focus on discrete interventions or aspects of the project.
Build monitoring and evaluation into project design. Given that  evaluation is not generally considered during design, panelists  emphasized that project officials should consider how the project will  be evaluated before the project is implemented. Most importantly,  organizations need to define the project’s intended impact and how that  impact will be measured. To do this, organizations would have to  determine the project beneficiaries and to which group they would be  compared, the data they would have to collect, and how they would  analyze those data. Management would not only need to collect data  during implementation to monitor if the project works according to  plan, but also collect data before and after implementation to determine  what works best.
section_title: Conclusions
paragraphs: The United States has played an important role in combating global human  trafficking and spurring other governments to increase their efforts in  doing so. As organizations around the world increasingly collaborate in  combating trafficking, their ultimate success will depend on the extent to  which they are able to overcome difficult challenges, such as varying levels  of government commitment and capacity, that have impeded collaboration  in the past. More than 7 years after the passage of the UN protocol, little is  known about which interventions have been the most effective in  preventing human trafficking, protecting victims, and prosecuting  traffickers.
The United States and other governments, international organizations, and  NGOs continue their efforts to fight trafficking, but there is little  information available to inform their decisions about project  implementation and selection. Although antitrafficking projects contain  some important elements for monitoring, they often lack other important  elements for measuring performance on a real-time basis, such as targets.  Such elements are critical in monitoring project performance to determine  whether interventions are being implemented as expected, or whether they  need to be changed to better combat human trafficking. Evaluation is also  important in determining whether antitrafficking projects have been  effective. However, few impact evaluations have been completed due to the  difficulties involved. As a result, little is known about the impact of  antitrafficking interventions.
Given the grave personal suffering of victims and negative impacts on  society that human trafficking creates, strengthening collaboration,  monitoring performance, and evaluating impact are important to ensure  that organizations fund antitrafficking interventions with the greatest  impact, where they are most needed, and through the effective and efficient  use of resources.
section_title: Recommendations for Executive Action
paragraphs: We recommend that the Secretaries of State and Labor and the  Administrator of USAID improve the monitoring and evaluation of their  projects to combat global human trafficking by considering the following  actions, where appropriate: 1.
Improve information about project impact on the nature and severity of  human trafficking, including  developing better data about the incidence of trafficking at the   applying rigorous evaluation methodologies. 2. Address monitoring and evaluation weaknesses in the design of   developing a framework that clearly links activities with project-level  goals, indicators, and targets;  conducting “evaluability assessments” to determine whether a  project is ready to be evaluated; and  building monitoring and evaluation into project design before the  project is implemented.
We are addressing our recommendations to State, Labor, and USAID  because they provided the most U.S. funding for projects to combat global  human trafficking.
section_title: Agency Comments and Our Evaluation
paragraphs: We requested comments on a draft of this report from the Secretaries of  State, Justice, Health and Human Services, Homeland Security, and Labor;  the Administrator of USAID; and cognizant officials at UNODC, IOM, ILO,  OAS, OSCE, the World Bank, IDB, and ADB or their designees. We received  written comments from State, Labor, USAID, and HHS, which are reprinted  in appendixes IV, V, VI, and VII along with our responses to specific points.
In its comment letter, State noted that it will implement the  recommendations in the report in ways that are relevant and appropriate to  the mandates of the State offices working on antitrafficking efforts. State  further noted that the recommendations are entirely consistent with the  department’s current activities and direction. State also generally agreed  that the antitrafficking field is well-served by more information about the  nature and severity of human trafficking and recognized that effective  project design is critical to successful project implementation and program  monitoring and can lay the foundation for evaluation. However, State  disagreed with our finding that monitoring is limited, stating that  monitoring is in place at the department and improving. While we  recognize that State and other U.S. agencies have certain elements of  monitoring in place, we report that they lack others. For example, we  found that for the 23 antitrafficking projects we reviewed, the majority do  not have a logic model that clearly explains how activities are linked to  project goals. In addition, the majority of these projects do not specify  targets that will establish benchmarks for measuring performance. State  also emphasized that many of its projects are designed to be of more  limited size, scope, and duration than those of other agencies, such as  Labor. State further noted that these projects of limited duration are  worthy of funding, but are not necessarily appropriate for evaluation. In the  report, we state that the impact of a shorter-term, smaller-scale  intervention may be difficult to attribute and quantify.
Labor commented that the report provides a good overall assessment of  international cooperation and the need to enhance collaboration among  key agencies and governments regarding antitrafficking efforts. Labor also  commented that the report highlights important areas for improving  monitoring and evaluation of U.S.-funded antitrafficking programs.  However, Labor stated that the report does not fully reflect efforts  particular federal agencies are taking in the monitoring and evaluation of  antitrafficking projects. As an example, Labor stated that it uses several  mechanisms, including audits and process evaluations, in its monitoring  and oversight of international technical assistance projects, including  antitrafficking projects, to ensure that U.S. funds lead to planned outputs  and results. In response, we made additions or revisions to the text to  further clarify Labor’s monitoring and evaluation efforts. We believe the  overall monitoring of antitrafficking projects is limited because the  projects funded by the other five agencies did not have the elements of  monitoring we found in Labor’s projects.
USAID said it appreciates the thoughtfulness of GAO’s report. USAID also  commented that it is concerned with the challenges of coordination,  monitoring, and evaluation, and that while it has made considerable efforts  to coordinate within the U.S. government and with other organizations, it  will continue to work within the interagency process in Washington and in  the field. USAID also agrees that monitoring and evaluation of its  antitrafficking efforts are very important and that they rely on the  availability of both human and financial resources. USAID further agrees  that the issues inherent in the evaluation of antitrafficking activities are  particularly challenging because there is no baseline against which to  measure progress.
HHS said the report is a sound document that substantively covers the wide  range of programs and services available to combat human trafficking.
We also received technical comments from Justice and DHS, as well as  from UNODC, IOM, ILO, OAS, OSCE, the World Bank, IDB, and ADB, which  we have incorporated in the report as appropriate.
We are sending copies of this report to interested congressional  committees; the Secretaries of State, Justice, Health and Human Services,  Homeland Security, and Labor; the Administrator of USAID; ILO; IOM; and  UNODC. We will also provide copies to others on request. In addition, the  report will be available at no charge on the GAO Web site at  http://www.gao.gov.
If you or your staffs have any questions about this report, please contact  me at (202) 512-9601. Contact points for our Offices of Congressional  Relations and Public Affairs may be found on the last page of this report.  GAO staff who made major contributions to this report are listed in  appendix VIII.
section_title: Objectives, Scope, and Methodology
paragraphs: Our objectives were to examine (1) collaboration among organizations  involved in international antitrafficking efforts, (2) U.S. government  agencies’ monitoring of antitrafficking projects and difficulties in  evaluating these projects, and (3) suggestions for strengthening monitoring  and evaluation.
To examine collaboration among organizations involved in international  antitrafficking efforts, we reviewed relevant planning, funding, and project  documents on human trafficking from the Departments of State, Justice,  Labor, Homeland Security, and Health and Human Services and the U.S.  Agency for International Development (USAID). We reviewed planning and  project documents from relevant United Nations (UN) and other  international agencies and offices, including the UN Office on Drugs and  Crime (UNODC), International Labor Organization (ILO), International  Organization for Migration (IOM), United Nations Children’s Fund  (UNICEF), and UN High Commissioner for Refugees (UNHCR). We also  reviewed UN reports and resolutions that address coordination as well as  documents from regional organizations, such as the Organization for  Security and Cooperation in Europe (OSCE), the Coordinated Mekong  Ministerial Initiative Against Trafficking (COMMIT), and the Regional  Conference on Migration.
In addition, we reviewed documents from and discussed international  antitrafficking efforts with officials from the above agencies and  organizations as well as officials from host government, donor government,  and nongovernmental organizations (NGO), in Washington, D.C., and  during our fieldwork in Indonesia, Thailand, and Mexico. We also reviewed  documents describing these countries’ national mechanisms to combat  trafficking, including legislation and action plans, where applicable. We  selected Thailand because it is the country with the largest number of  international organizations working to combat human trafficking. We  selected Indonesia and Mexico because they receive a large amount of U.S.  funding for international antitrafficking projects and have a relatively large  number of U.S. government agencies and international organizations  working in each country. All 3 countries are origin, transit, and destination  countries for human trafficking victims and have projects addressing  prevention, protection, and prosecution.
To examine organizations’ monitoring and evaluation of antitrafficking  programs, we reviewed documentation from 23 projects in the 3 countries  we visited and illustrative projects from other countries. We worked with  U.S. agency officials in Washington and in the field to identify U.S.-funded  antitrafficking projects in Indonesia, Thailand, and Mexico that were  ongoing during the time of our review (August 2006 to June 2007). We  requested project documents—including proposals, grant agreements, and  progress reports—from these U.S. officials for projects identified in these  countries. In response to this request, we received documents for 23  antitrafficking projects that were funded or implemented by 6 U.S. agencies  involved in international antitrafficking efforts. We examined the project  documents for the following elements: statement of goals or objectives,  statement of activities, identification of indicators and targets, explanation  of how targets were selected, and inclusion of a logic model or framework.  While we recognize that these 23 projects may not be the complete  universe of antitrafficking projects in these 3 countries, we consider these  23 projects sufficient for the purposes of our review. Although we cannot  assume that the issues we identified exist across all projects, they  nevertheless represent areas for improvement in monitoring antitrafficking  projects. We also reviewed a set of 4 State Office to Combat and Monitor  Trafficking in Persons (G/TIP) antitrafficking projects in India, Israel,  Afghanistan, and Costa Rica. These projects were selected to illustrate  some current monitoring and evaluation practices for existing projects in  different parts of the world and different types of exploitation. They  showed the variation in monitoring practices that can only increase with  sample size. Thus, the projects provided the background information and  context needed to understand State G/TIP’s current efforts to standardize  monitoring requirements. The reviewed documents for all projects  included proposals, applications for assistance, project descriptions,  strategy papers, concept papers, causal models, cooperative or grant  agreements, and periodic and final reports.
We also reviewed articles and books on monitoring, evaluation, and  statistics, such as Sampling by Steven K. Thompson, Wiley Series in  Probability and Mathematical Statistics (1992); Adaptive Sampling by  Stephen K. Thompson and George A.F., Seber, Wiley Series in Probability  and Mathematical Statistics (1996); Better Evaluation for Evidence-Based  Policy: Place Randomized Trials in Education, Criminology, Welfare,  and Health by Robert Baruch, The Annals of the American Academy of  Political and Social Science (2005), vol. 599, 6-18; Program Evaluation  Methods: Measurement and Attribution of Program Results, Third  Edition, Treasury Board of Canada, Secretariat; and Monitoring and  Evaluation: Some Tools, Methods and Approaches, the World Bank (2004).
To examine suggestions for strengthening monitoring and evaluation, we  worked with the National Academy of Sciences to organize a 2-day expert  panel on challenges and alternative strategies for monitoring and  evaluating the results of international antitrafficking programs and projects  in April 2007. We invited the following groups of panel participants:  Experts with broad-based, subject-area knowledge of human trafficking.
Experts with specialized knowledge of monitoring and evaluation of  programs aimed at hidden populations similar to trafficking victims,  such as the homeless and irregular migrants. This group included  experts with specific knowledge of baseline data estimation of such  populations.
Panelists had backgrounds in academia, research, consulting, and project  implementation in the field. Using a nominal group technique, panelists  chose to focus on the intervention “safe return” as a starting point for  discussion. Experts provided presentations and participated in a  discussion and cross-fertilization of ideas. Using a nominal group  technique, panel members also ranked two topics in order of importance to  the human trafficking field—estimating the number of trafficking victims  and evaluability assessments. None of the panel members were  compensated for their work on this project. The following experts  participated in the panel:   Richard Berk, Professor of Criminology and Statistics, Department of  Criminology, University of Pennsylvania  Robert Boruch, University Trustee Chair Professor, Graduate School of  Education and Statistics Department, Wharton School, University of  Pennsylvania  Mario Thomas Gaboury, Professor and Chair of Criminal Justice,   Kristiina Kangaspunta, Chief, Anti-Human Trafficking Unit, United  Nations Office on Drugs and Crime  Jonathan Martens, Counter-Trafficking Project Specialist, Counter- Trafficking Division, International Organization for Migration  Jeffrey S. Passel, Senior Research Associate, Pew Hispanic Center  Lisa Rende-Taylor, Technical Advisor to the United Nations Inter-Agency  Project on Human Trafficking in the Greater Mekong Subregion   Peter Reuter, Professor, School of Public Policy, Department of  Criminology, University of Maryland  W. Courtland Robinson, Assistant Professor, Center for Refugee and  Disaster Response, Johns Hopkins Bloomberg School of Public Health  Debra Rog, Associate Director, Westat Corporation  Jane Nady Sigmon, Senior Coordinator for International Programs,  Office to Monitor and Combat Trafficking in Persons, U.S. Department  of State We conducted our review from August 2006 to June 2007 in accordance  with generally accepted government auditing standards.
section_title: Selected International Organizations Involved in Combating Human Trafficking
paragraphs: This appendix describes the general mission and antitrafficking activities  of 15 international organizations that implement international  antitrafficking projects.
section_title: Methods Suggested by Expert Panel to Estimate the Number of Human Trafficking Victims
paragraphs: This appendix describes the sampling methods suggested by a GAO- convened panel of experts to estimate the number of human trafficking  victims.
section_title: Comments from the Department of State
paragraphs: The following are GAO’s comments on the Department of State letter dated  July 13, 2007.
subsections: section_title: GAO Comments
paragraphs: 1. State said that the report’s title is not apt because program monitoring  is in place and improving. We believe that monitoring is limited for the  following reasons: for the 23 projects we reviewed, the majority did not  have a logic model that clearly explains how activities are linked to  project goals; the majority of these projects also did not specify targets  that establish benchmarks for measuring performance; of the 10  projects that did specify targets, only 5 explained how targets were set;  and, finally, State lacked written guidance for field-level oversight.  2. State noted that many of its projects are designed to be of more limited  size, scope, and duration, and that such projects are not necessarily  suitable for evaluation. They also added that larger and longer-term  projects are more suitable candidates for impact evaluation than  shorter projects. Finally, State commented that a relatively short time  frame for projects is not a design weakness. In the report, we state that  the impact of a shorter-term, smaller-scale intervention may be difficult  to attribute and quantify.  3. State noted that baseline data are not required for all interventions. We  disagree. In the report, we state that baseline and target values of  indicators are needed to assess project performance. For example,  baselines for a victim assistance program could include the number of  victims served or assisted, or the number of vocational training  sessions held. Moreover, project-level estimates are needed for  baselines by which to evaluate how effectively interventions are  reducing trafficking. For this type of baseline, panelists suggested  methods for baseline estimation, such as gathering data in trafficking  hot spots, that would allow for more rigorous impact evaluations.
section_title: Comments from the Department of Labor
paragraphs: The following are GAO’s comments on the Department of Labor letter  dated July 13, 2007.
subsections: section_title: GAO Comments
paragraphs: 1. Labor noted that federal agencies are carrying out monitoring and  evaluation activities or are making efforts to improve monitoring and  evaluation of antitrafficking activities. Labor also provided details of its  monitoring policies, procedures, and activities. We recognize that Labor  has monitoring procedures and activities in place, and we revised text  in several locations to provide further clarification on Labor’s activities.  We clarified that all four Labor projects in Indonesia, Mexico, and  Thailand clearly link goals and activities, specify targets, and explain  how targets were set. We added language stating that Labor engaged  ILO’s external auditor to conduct audits of a sample of ILO’s projects to  eliminate child labor and contracted with a certified public accounting  firm to conduct independent attestation engagements of its Education  Initiative projects. We believe the overall monitoring of antitrafficking  projects is limited because the projects funded by the other five  agencies did not have the elements of monitoring we found in Labor’s  projects. 2. Labor stated that it requires midterm and final evaluations for its  technical assistance projects, and that, in some cases, it engages in  longer-term follow-up studies. We revised the text to further clarify and  recognize Labor’s evaluation requirements and activities. In addition,  we added text specifying that Labor funds follow-up studies to  document changes that have occurred to a sample of beneficiaries over  time.  3. Labor agreed with the need to carry out more systematic impact  evaluations, but also emphasized that process evaluations are an  important prerequisite of impact evaluations. Labor further commented  that randomized trials are not appropriate for all projects, and  requested that we specify where agencies should conduct randomized  trials and where other methods would be more appropriate. We agree  that process evaluations are important and revised the text to further  differentiate process evaluations from impact evaluations. We believe  each agency should determine whether randomized trials are  appropriate for the specific project they are evaluating.  4. We made changes to the draft of this report on the basis of Labor’s  specific technical comments, where appropriate.
section_title: Comments from the United States Agency for International Development
section_title: Comments from the Department of Health and Human Services
section_title: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
section_title: Staff Acknowledgments
paragraphs: Cheryl Goodman, Assistant Director; Jeremy Latimer; Christina Werth;  Todd M. Anderson; Gergana Danailova-Trainor; Terry Richardson; and  Debbie Chung made key contributions to this report. In addition, Elizabeth  Curda, Bruce Kutnick, Mary Moutsos, Barbara Stolz, and Susanna Kuebler  provided technical or legal assistance.