Water and Sanitation Assistance: USAID Has Increased Strategic Focus but Should Improve Monitoring
Released: 2015-10-06T00:00:00

Letter

Background

Legislation and Key Agency Roles Related to WASH Assistance

The 2005 Act made access to safe water and sanitation a U.S. foreign policy objective. Specifically, the act aimed to promote the provision of access to safe water and sanitation to countries, locales, and people with greatest need, including the very poor, women, and other vulnerable populations.

Congress passed the 2014 Act to strengthen the 2005 Act by improving the capacity of the U.S. government to implement, monitor, and evaluate programs that increase access to safe water, sanitation, and hygiene. The 2014 Act requires, among other things, that USAID ensure that WASH projects are designed to promote maximum impact and long-term sustainability. The 2014 Act also calls for rigorous monitoring and evaluation to assess improvements in WASH.

State’s Office of Conservation and Water within the Bureau of Oceans and International Environmental and Scientific Affairs is responsible for the development and implementation of U.S. foreign policy on international water and sanitation assistance. USAID’s Office of Water within the Bureau for Economic Growth, Education and Environment is responsible for coordinating, managing, and overseeing USAID’s response to water policy initiatives, including the 2014 Act.

Annual Spending Requirement for WASH Assistance

Annual appropriations acts funding foreign operations have established a spending requirement for WASH. These spending requirements have ranged from $315 million to $365 million since 2012. USAID and State have provided annual guidance to USAID missions regarding activities for which they may attribute funding to the annual spending requirement. The guidance notes that for attribution, proposed activities must be able to demonstrate an impact on water supply, sanitation, or hygiene through objectively verifiable indicators to measure progress. The 2005 Act required State to submit an annual report to Congress detailing the status of WASH efforts; these reports included information from USAID about funds allocated to WASH activities to meet the spending requirements for WASH. This reporting requirement was repealed by the 2014 Act.

USAID Water and Development Strategy

Improving health outcomes through the provision of sustainable WASH is the first strategic objective of USAID’s Water and Development Strategy, 2013-2018 (Water Strategy). To achieve this objective, the Water Strategy includes three key WASH-related goals: Increase first-time and improved access to sustainable water supply. First-time access refers to access to an improved water source that is gained by previously unserved populations. Improved access refers to enhancing existing access to, and the quality of, an already improved water supply.

Increase first-time and improved access to sustainable sanitation. First-time access to improved sanitation generally refers to access to a pit latrine with a slab, septic system, or similar type of improved sanitary facility. Improved access to sanitation generally refers to improvement of an existing sanitation facility.

Increase adoption of key hygiene behaviors. The Water Strategy recommends the promotion of three hygiene practices with the greatest demonstrated impact on health: (1) hand washing with soap at critical times; (2) safe disposal and management of excreta; and (3) improving household water storage, handling, and treatment.

Figure 1 shows USAID’s definitions for improved and unimproved drinking water and sanitation.

In March 2014, USAID issued the Water Strategy Implementation Field Guide (Field Guide) as a tool to help missions understand and implement the Water Strategy. The Field Guide requires that USAID missions track progress toward the three WASH-related goals using standard indicators and some custom indicators, as shown in table 1.

State’s annual reports to Congress detailing the status of WASH projects have included country-level results for two of the standard WASH indicators: (1) number of people gaining access to an improved drinking water source and (2) number of people gaining access to an improved sanitation facility. This information has also been reported in USAID’s annual report on international water-related assistance, Safeguarding the World’s Water.

The Water Strategy projected that during the next 5 years, at least 10 million persons would receive sustainable access to improved water supply and 6 million persons would receive sustainable access to improved sanitation. These projections included persons receiving first- time and improved access to water supply and sanitation.

According to the Water Strategy and the Field Guide, to achieve the greatest impact, WASH projects should include the following elements: expanded access to “hardware” (e.g., water and sanitation infrastructure and hygiene commodities); required “software” activities to promote behavior changes for sustained improvements in water and sanitation access/service and hygiene practices; and an improved enabling policy and institutional environment, including strengthened financial frameworks and public-private partnerships for WASH.

The Field Guide states that the level of effort in each area may vary depending on local context and other factors. In addition, the Water Strategy and the Field Guide highlight gender issues in the water sector as a key focus. They note that the burden of inadequate access to water and sanitation often falls heavily on women and girls and that WASH activities should promote gender equality and female empowerment to address the needs and opportunities of both men and women.

WASH Sustainability

The Water Strategy emphasizes the importance of sustainable WASH services. USAID policy defines sustainable services as public services in which host country partners and beneficiaries take ownership of development processes and maintain project results beyond the life of a USAID project. The strategy notes that the pillars of sustainability for water projects include integrated water resource management, sound governance, and appropriate environmental design, among other factors. In particular, regarding projects to provide first-time and improved access to quality water supply services, the strategy states that methods for ensuring the sustainability of water quality in the long term should be incorporated into project design and that this may include developing monitoring systems to ensure that water quality and supply are sustained at acceptable levels. Further, regarding sanitation services, the strategy notes that it supports the development and testing of improved, low-cost sanitation and waste management technologies, as well as innovative management and financing approaches, to ensure sustainability and to facilitate more rapid expansion of basic sanitation solutions.

USAID Priority Countries for WASH Assistance

In fiscal year 2014, USAID designated 22 countries as tier 1 or tier 2 priorities for WASH assistance on the basis of their need and the opportunity to achieve significant impact. The 6 tier 1 countries are those where USAID found an opportunity to have a transformative impact on national-level policies and to leverage host country resources for the development and implementation of assistance. The 16 tier 2 countries are those where USAID determined that relatively small investment levels were likely to generate a significant impact in at least one dimension of WASH. USAID reported attributing about $463.1 million to the annual spending requirements for WASH in fiscal years 2012 through 2014 for these priority countries. Figure 2 shows the locations of the tier 1 and 2 priority countries.

USAID Missions’ WASH Activities and Funding Varied

In the nine countries we selected for our review, USAID missions reported $214.5 million in WASH funding for 74 activities, with 6 key focus areas. Each mission’s total funding for WASH activities in these nine countries ranged from about $53.4 million, in Indonesia, to about $4.4 million, in Haiti. Missions reported implementing from as many as 19 WASH activities, in Kenya, to as few as 3 activities, in both Tanzania and Zambia. The most frequent key focus areas for these activities were capacity building and behavior-change communication, followed by infrastructure construction, technical assistance, policy and governance, and financing. Figure 3 provides additional detail about the activities implemented by each selected mission in fiscal years 2012 through 2014.

Table 2 describes the 74 activities’ key focus areas and provides examples of the types of WASH activities in each focus area for the nine selected countries.

Missions noted that some of the 74 reported activities included WASH as one of several other components, such as maternal-child health, nutrition, or natural resources management. For example, the Senegal mission’s Yaajeende Agriculture activity is an agricultural activity that aims to increase nutritional status by diversifying foods produced and eaten. The activity includes a WASH component to expand access to clean drinking water and improved sanitation and also includes water resources management efforts to promote effective small irrigation technologies. In another example, the DRC mission’s Integrated Health Project aims to improve the enabling environment and increase the availability of services, products, and practices for family planning; maternal, newborn, and child health; nutrition, malaria, and tuberculosis; and WASH in targeted health zones.

Additionally, the nine missions reported implementing WASH activities targeting three types of geographic areas: rural, peri-urban (e.g., small towns), and urban. For example, the Tanzania mission’s Integrated Water, Sanitation, and Hygiene activity targeted rural and peri-urban areas (e.g., small towns) to provide support including community piped water schemes, rehabilitated wells, training for community groups to operate and maintain water systems, and school latrines, among other efforts. In contrast, the Indonesia mission’s Indonesia Urban Water, Sanitation, and Hygiene activity aimed to increase access to clean water and improve sanitation facilities for people in urban settings, where about half of Indonesia’s population lives. According to the Indonesia mission, the activity focused on fostering demand for WASH, building capacity, and strengthening the policy and financing environment for WASH through its work with central and local governments, including entities responsible for delivering water and sanitation services.

Table 3 includes information about the geographic focus of the 74 activities in the nine selected countries.

USAID Is Taking Steps to Enhance Strategic WASH Approach and Address WASH Sustainability

In the nine countries we selected for our review, we found that USAID missions are taking steps to develop and implement WASH plans and are incorporating the Water Strategy’s principles and approach into recent and planned WASH activities. The missions are considering sustainability as part of WASH project planning, and USAID’s Office of Water is developing guidance for missions specific to WASH sustainability.

Missions in Selected Countries Have Taken Steps to Develop and Implement WASH Plans and Enhance Strategic WASH Approach

We found that the missions in the nine selected countries have made varying degrees of progress in developing WASH plans. In addition, we found that these missions have generally taken steps to implement WASH plans and enhance the strategic approach in recent WASH projects.

Our review of documents describing these missions’ strategic approach to WASH, as well as interviews with officials at the nine missions, found that five of the missions had completed WASH plans. Missions’ WASH plans may consist of one or more project appraisal documents (PAD), which document a project’s design and expected results. The remaining four missions were in the process of developing or finalizing such plans.

Five of the nine missions—in Ethiopia, Indonesia, Kenya, Tanzania, and Zambia, respectively—had completed WASH plans and taken steps to implement these plans as of July 2015. The Indonesia and Tanzania missions initially developed WASH plans in fiscal year 2009. The Zambia, Kenya, and Ethiopia missions completed WASH plans more recently, in the period from fiscal year 2013 to fiscal year 2015.

Three of the nine missions—in the DRC, Senegal, and Uganda, respectively—were in the process of finalizing WASH plans as of July 2015.

One mission, in Haiti, had begun developing a plan for a more strategic approach to its WASH activities as of February 2015. According to mission officials, the mission had focused its recent water-related activities on agriculture, water resources management, and water productivity. In response to the Water Strategy, the mission has begun to develop a plan for future WASH activities that will consider findings from a fiscal year 2015 mission WASH sector assessment.

Our review of missions’ WASH project documents, as well as semi- structured interviews with officials at the nine missions, also found that these missions have generally taken steps to implement WASH plans and incorporate the Water Strategy’s strategic principles and approach in recent WASH projects, such as by implementing activities to support expanded access to water and sanitation infrastructure along with activities to support capacity building and behavior change. Although four of the nine missions acknowledged that their prior WASH projects lacked strategic focus, all nine missions cited ongoing efforts to develop more focused, strategic, and impactful WASH projects in accordance with the Water Strategy. government support and leadership for WASH were critical to sustainability and continued improvement in the sector. The Tanzania mission’s primary WASH activity, the Tanzania Integrated Water, Sanitation, and Hygiene Program, focused on increasing water supply through a multiple-use water services approach, which the mission’s WASH plan describes as considering domestic and productive water needs to maximize benefits and increase the cost-effectiveness of WASH investments (see sidebar).

Missions that recently completed, or are in the process of finalizing, WASH plans have also taken steps to incorporate the Water Strategy’s principles and approach into recent activities, such as by implementing activities to support expanded access to “hardware” (e.g., water and sanitation infrastructure), as well as to support “software” activities (e.g., efforts to build capacity and promote behavior change). For example, the Zambia mission’s Schools Promoting Learning Achievement through Sanitation and Hygiene activity provided support for latrine construction, rehabilitated water points, and handwashing facilities, as well as hygiene education and capacity building related to operations and maintenance of WASH infrastructure. In addition, missions have supported community-led total sanitation efforts to create demand for improved sanitation. For instance, the Senegal mission’s Millennium Water and Sanitation activity included support for community-led total sanitation, among other components.

Additionally, missions cited examples of steps they have planned to enhance the strategic focus and potential impact of WASH projects going forward, such as by increasing investments in sanitation, encouraging local government ownership of efforts, supporting public-private partnerships and WASH financing options, and enhancing activities’ focus on gender issues. For example, six of the nine missions—in Ethiopia, Indonesia, Senegal, Tanzania, Uganda, and Zambia, respectively—cited plans to increase the focus of their WASH activities on sanitation. The Uganda mission noted that although it had previously implemented several activities that addressed sanitation on a small scale, it plans to focus exclusively on sanitation in its upcoming primary WASH activity. According to the mission, the planned activity will seek to improve access to affordable and acceptable sanitation through public-private partnerships, support for affordable sanitation financing options, and subsidization schemes to reach the poor.

Figure 4 summarizes our findings regarding the status of the nine selected missions’ WASH plans and their recent and planned WASH activities as of July 2015. Appendix II provides additional information about the selected missions’ WASH activities.

Missions Are Considering Sustainability as Part of WASH Planning

Our review of mission documents and information showed that the nine selected missions are generally taking steps to address sustainability as part of their WASH project planning, as described below:

WASH PADs for all five of the missions that had completed these documents—in Indonesia, Kenya, Tanzania, Uganda, and Zambia— included sustainability analyses. For example, the Kenya mission’s WASH PAD included an annex with a two-page sustainability analysis that described the mission’s planned steps to address challenges to sustainable WASH, including challenges related to choice of technologies and technical approaches, weaknesses in governance, long-term financing needs, and the need for sustained behavior changes. The Uganda and Zambia missions’ PADs sustainability analyses did not address WASH specifically but described sustainability considerations in the context of broader project efforts (e.g., health or education).

Missions’ award documents for the 16 activities we reviewed in detail generally described sustainability considerations. For example, the award agreement for a WASH activity in Senegal noted that decades of development experience have shown that merely building water and sanitation infrastructure is not sufficient to deliver adequate service or to ensure that such services are sustainable. The agreement added that WASH efforts would incorporate principles such as optimal balance between hardware and software activities, local ownership and decentralized management of WASH infrastructure and service delivery, and use of appropriate and affordable technologies.

Five of the nine missions conducted more in-depth WASH sustainability assessments to inform planning and design of WASH activities. The Ethiopia mission completed two studies between April 2012 and February 2013 to assess existing water supply schemes and potential strategies to improve sustainability for the mission’s primary ongoing WASH activity, which was implemented in several regions. In addition, in 2013 and 2014, USAID piloted a tool that it developed with Rotary International to provide an in-depth assessment of the sustainability of several WASH activities implemented by missions in three of the nine countries we selected for our review—Indonesia, Kenya, and Tanzania. The assessments included an overview of the WASH sector in each country; a description of steps taken to apply the sustainability tool; results of using the tool, including scores for institutional, management, financial, technical, and environmental sustainability; key findings; and priority areas for action and recommendations by intervention category (i.e., water supply, sanitation, or hygiene). A USAID official noted that for this type of tool to be effective, USAID would need to use it systematically and incorporate a feedback loop to ensure that results are incorporated into future activities. The Indonesia mission incorporated the results of using the tool into its latest WASH plan, while the Kenya and Tanzania missions completed their latest WASH plans before the results of the tool were available. However, Kenya and Tanzania mission officials stated that they are considering results of the assessments in planning their upcoming activities.

Mission officials in the selected countries generally told us that they plan to enhance the focus of future activities on sustainability. For example, officials at the Senegal mission noted that they conducted a sustainability analysis in the process of developing the mission’s forthcoming PAD related to WASH. To enhance its focus on sustainability, the mission is planning, among other efforts, to implement private sector reform activities and more directly engage with the government of Senegal to better ensure ownership of water and sanitation projects. Mission officials in the DRC said that they are in the process of determining the cost of monitoring all of their WASH activities after implementation is completed to better assess sustainability. The officials noted that UNICEF, which is implementing one of the mission’s WASH activities, is using its own resources to monitor the activity to assess sustainability 1 year after completion and that the mission would like to standardize the process for other activities.

USAID’s Office of Water Is Developing Guidance Specific to WASH Sustainability

USAID’s Office of Water has begun to develop guidance to address WASH sustainability. The March 2014 Field Guide states that USAID’s Office of Water will develop guidance for missions to program effectively for sustainable WASH services, including sustainability indicators, monitoring options, and tools for assessing sustainability. To inform future WASH sustainability guidance, in December 2014, USAID’s Office of Water developed a draft technical paper that included recommendations for designing projects to achieve sustainable WASH service and for confirming sustained results through longer-term monitoring. The draft paper described, among other things, the types of indicators that could be used to assess sustainability, including the functionality and reliability of WASH services. Additionally, the draft paper recommended that USAID’s Water Office should supplement its sustainability guidance with questions and resources for missions to consider in WASH project design, adopt sustainability indicators to assess the functionality and reliability develop an approach to post-project monitoring, and conduct an evaluation on the sustainability of WASH services in a sampling of countries.

According to USAID officials, USAID’s Office of Water intends to finalize its guidance for addressing the sustainability of WASH activities in 2015.

The officials stated that the guidance will likely be broad and include a menu of options, tools, and resources that can be tailored to the context in which each mission implements WASH activities.

Monitoring and Reporting for USAID’s WASH Activities in Selected Countries Had Limitations, although Evaluations Were Generally Sound

Our detailed review of 16 WASH activities in the nine selected countries found limitations in the monitoring and reporting of some activities’ performance, although most evaluations we reviewed were sufficiently reliable and methodologically sound for their intended purposes. Monitoring plans for 6 activities did not consistently include annual targets for key WASH indicators in accordance with USAID requirements, limiting the missions’ ability to measure progress toward WASH goals. Also, while the annual reports submitted by the 16 activities’ implementers generally included performance data for the key WASH indicators, the reports for 6 activities did not present data disaggregated by gender as required by USAID policy. Moreover, contrary to agency guidance, missions did not verify beneficiaries for at least 3 activities aimed at increasing access to improved drinking water sources and overstated beneficiaries for 6 activities aimed at increasing access to improved sanitation facilities, calling into the question the accuracy of USAID’s annual reporting about progress toward these WASH goals. The reasons for the missions’ inconsistent adherence to agency guidance regarding annual targets, gender-disaggregated data, and verification of beneficiaries were generally unclear, while USAID officials provided differing reasons for the inaccurate reporting for sanitation activities. In contrast, 12 of the 14 performance evaluations that we reviewed were sufficiently reliable and methodologically sound for their intended purposes.

Monitoring Plans and Reports for Several Activities Lacked Annual Targets, and Reports Did Not Consistently Disaggregate Beneficiaries by Gender

For more than one-third of the 16 activities we reviewed, the monitoring plans did not consistently include annual targets for key WASH indicators, such as the number of people gaining access to an improved drinking water source. USAID requires that at the start of each activity, the implementer establish a monitoring plan with indicators and associated targets to assess progress on an activity. Additionally, State guidance to USAID missions requires that proposed activities demonstrate impact through objectively verifiable indicators to measure progress toward WASH goals, if funds are attributed to the annual congressional spending requirement for international water and sanitation assistance. We found that, of the 16 activities, 3 lacked annual targets for key WASH indicators for the entire duration of the activity and 1 lacked annual targets for key WASH indicators for the remaining 2 years of the activity’s 5-year duration. Additionally, 1 activity included annual targets for the drinking water and sanitation component but not for the hygiene component; for another activity, annual targets were clearly identified for 2 years of its 5- year duration.

While annual reports for the 16 activities we reviewed generally included performance data for key WASH indicators, the reports for more than half of activities with beneficiary indicators did not disaggregate data for these indicators by gender as USAID policy requires. Ten activities had indicators to measure numbers of beneficiaries gaining increased access to an improved water source or improved sanitation facility. However, for 6 of these activities, the missions did not disaggregate performance data by beneficiaries’ gender, making it difficult for USAID to assess the activity’s contributions to gender equality and female empowerment.

Reasons for the inconsistency that we observed in the selected missions’ adherence to agency guidance regarding annual targets and gender- disaggregated data were unclear. USAID officials in Washington, D.C., noted the absence of consistent reasons among the nine missions for a lack of regular reporting on annual targets and of gender-disaggregated data. Officials at the DRC mission noted that staffing constraints had generally limited their ability to monitor their WASH activities and that they planned to hire additional staff to improve monitoring.

Figure 5 summarizes our findings regarding the nine selected missions’ compliance with USAID requirements in documenting targets and reporting performance data for the 16 activities that we reviewed.

Missions Did Not Verify Results for Some Water Activities and Overstated Results for Most Sanitation Activities

For some of the 10 activities with a water component that we reviewed, the missions did not verify the numbers of beneficiaries as required, and for most of the 8 activities with a sanitation component, the missions overstated the numbers of beneficiaries. As a result, because activity performance data contribute to USAID’s annual public reporting of WASH results, the data that USAID uses to report the numbers of people gaining access to an improved drinking water source and to an improved sanitation facility annually may not be accurate. The reasons for lack of verification for water beneficiaries were generally unclear, and mission officials, as discussed later, generally provided varying reasons for the inaccurate reporting for sanitation activities. Figure 6 summarizes our findings regarding the nine selected USAID missions’ compliance with State and USAID requirements for verifying beneficiaries of water activities and reporting beneficiaries of sanitation facilities.

For 7 of the 10 activities that we reviewed, the missions and implementers did not undertake efforts to verify reported beneficiaries or did not document such efforts, calling into question the accuracy of the reported results. State and USAID guidance for the standard indicator of access to an improved drinking water source requires that the implementer or evaluator verify these estimates by assessing factors such as the amount of time the user spent in collecting water and the quantity of water produced by the new or rehabilitated water source. The reasons for the lack of verification, or documentation of verification, of beneficiaries of the 7 activities were generally unclear.

Documents for only 3 of the 10 activities—in Tanzania, Ethiopia, and Indonesia, respectively—evidenced verification efforts and, in one case, corrective actions related to verifying results of activities to provide access to improved drinking water. For 4 other activities, it was unclear, on the basis of documents that the missions provided and mission officials’ responses to our queries, whether the reported results were verified. For example, although officials at the Senegal mission stated that they undertook regular site visits and data quality assessments to verify estimated beneficiaries, they did not provide documentation of such verification. Moreover, the midterm evaluation for this activity noted that estimating the number of beneficiaries on the basis of the Senegalese government’s assumptions was feasible at the planning stage but not during implementation, when the number of beneficiaries could be verified. The evaluation also noted that, absent precise numbers of beneficiaries, it was difficult to determine the cost-effectiveness of the investment.

For the remaining 3 activities—2 in the DRC and 1 in Zambia—mission officials informed us that they had not verified the reported results and cited varying reasons.

In the DRC, annual reports for 2 of the WASH activities we reviewed indicated that more than 900,000 people gained access to an improved drinking water source between fiscal years 2012 and 2014. In USAID’s annual report for fiscal year 2013, the number of beneficiaries reported as gaining access to an improved drinking water source in the DRC constituted more than 13 percent of worldwide beneficiaries (446,989 out of 3,509,090). Officials at the DRC mission informed us that they had been unable to conduct a data quality assessment planned for October 2014 because of challenges that included security concerns and difficulties of traveling in a country with limited roads. Mission officials also noted that estimating numbers of people who gained access to an improved drinking water source was complicated by the displacement of population in the areas where USAID was implementing its WASH activities.

In Zambia, the mission reported in fiscal years 2012 and 2013 that 82,606 and 62,098 people, respectively, gained access to an improved drinking water source as a result of the mission’s two school WASH activities. Officials at the Zambia mission informed us that although they undertook verification of data for the ongoing school WASH activity’s sanitation component, they did not verify all reported beneficiaries of the water component, which included the school population as well as people from the surrounding communities. The officials noted that while beneficiaries gaining access to improved drinking water in schools could be verified, it was not possible to verify beneficiaries from communities surrounding the school, who also have access to water points constructed or rehabilitated under this activity.

Inaccurate Reporting of Beneficiaries Gaining Access to Improved Sanitation Facilities

For six of the eight activities aimed at increasing access to improved sanitation facilities that we reviewed, the activities’ implementers reported beneficiaries for facilities that did not meet USAID’s definition of an improved facility. As a result, the data reported to track progress toward the goal of increasing access to improved sanitation are likely overstated. USAID uses WHO and UNICEF definitions of an improved sanitation facility, which state that a pit latrine without a slab or platform is an unimproved sanitation facility and that only facilities that are not shared or are not public are considered improved.

For six of the eight sanitation activities that we reviewed, implementers tracked and reported numbers of people gaining access to sanitation facilities that included unimproved latrines and shared facilities. Mission officials generally provided differing reasons for the inaccurate reporting for these activities, such as perceived agency emphasis on reporting beneficiaries and adherence to host-government policy or practice. USAID officials in Washington, D.C., noted that, to some extent, the missions’ inconsistency in accurately reporting on the sanitation indicator resulted from inadequate understanding of USAID’s definition of improved sanitation facilities among some mission staff overseeing these activities.

In the DRC, more than 520,000 people gained access to an improved sanitation facility in fiscal years 2012 through 2014 as a result of the mission’s primary WASH activity, according to activity annual reports. However, according to mission officials, the activity’s reported beneficiaries included those who gained access to household-built latrines, including basic pit latrines. USAID officials in Washington, D.C., stated that community-led total sanitation, which reflects the DRC mission’s approach, involves changing people’s behavior regarding use of sanitation facilities by encouraging them to build latrines; however, the officials said that these latrines generally do not meet USAID’s definition of improved sanitation facilities. These officials noted that the results of community-led total sanitation efforts should be reported for USAID’s WASH indicator for a community becoming open-defecation free rather than for the WASH indicator for increased access to an improved sanitation facility. According to officials at the DRC mission, the mission reported results of community-led total sanitation efforts for the indicator for first-time access to an improved sanitation facility in part because of perceived USAID headquarters emphasis on reporting numbers of beneficiaries of WASH assistance. Mission officials noted that headquarters’ emphasis on numbers of beneficiaries had, to some extent, led the mission to focus on activities to increase direct access to water and sanitation rather than on efforts to improve institutions or governance.

In Ethiopia, according to the final report for one of the activities we reviewed, 385,909 people gained access to improved sanitation in fiscal years 2009 through 2013 as a result of this activity. However, the latrines built through this activity included pit latrines without slabs (see fig. 7 for an example), which do not meet USAID’s definition of improved sanitation facilities; consistent with WHO and UNICEF definitions, USAID categorizes a basic pit latrine without a slab as an unimproved sanitation facility. According to Ethiopia mission officials, the activity did not fund the construction of household latrines but instead encouraged households to build their latrines with locally available materials, in consistency with Ethiopian government policy. Furthermore, mission officials noted that the reported results for this activity were based on data from the Ethiopian government’s Health Management Information System, which uses an official Ethiopian government definition of improved sanitation that is not consistent with USAID’s definition.

In Indonesia, Kenya, Tanzania, and Zambia, respectively, the reported beneficiaries of activities to increase access to improved sanitation facilities included people who gained access to shared facilities, such as school toilets. For example, the Zambia mission reported that more than 133,000 people gained access to an improved sanitation facility in fiscal years 2012 through 2014 as a result of the activity focused on increasing WASH access in schools. However, according to USAID guidance, shared sanitation facilities, such as those in schools and hospitals, cannot be included in the results to track progress on the number of people gaining access to improved sanitation facilities, because this indicator is assessed at the household level. According to officials at the Zambia mission, the data they reported included the entire population of schools where sanitation facilities were built; moreover, the ratio of population to sanitation facility exceeded the national standards. Mission officials noted that they focused on schools rather than households because the mission’s WASH efforts are aimed at improving the environment in schools to improve education outcomes.

Evaluations of Selected USAID WASH Activities Were Generally Sound

While we found limitations in the monitoring of several of the 16 activities we reviewed, the WASH activity evaluations that we assessed were, in general, sufficiently reliable and methodologically sound for their intended purposes. Ten of the 14 evaluations in our review assessed 9 of the 16 activities that we selected to examine monitoring. As noted previously in this report, performance monitoring and evaluation are separate activities; evaluations entail collection of additional information from sources that may be different from sources of monitoring data. For example, evaluators of the Ethiopia mission’s primary WASH activity selected a random sample of 6 out of 41 activity sites and obtained information through focus groups with potential beneficiaries, interviews with local government officials, and personal observation of new or rehabilitated water sources. The evaluators used monitoring data where it was available for one indicator (final target and final performance data for beneficiaries gaining access to an improved water source) but independently collected data via surveys to assess the results of hygiene and sanitation component for which the implementer established a final target but did not report on annual targets or performance data. This example also illustrates that while evaluations are distinct from monitoring, they can fill some gaps in monitoring information. (See app. III for USAID’s evaluation findings related to monitoring, outcomes, and sustainability.)

We assessed 14 evaluations of USAID’s WASH-related activities—2 baseline evaluations, 6 midterm evaluations, and 7 final evaluations—that were conducted in fiscal years 2012 through 2014 in the nine selected countries. We assessed the methodological quality of these evaluations on the basis of established evaluation principles, including the appropriateness of the evaluation design; clarity of population selection; clarity of data collection; and adequacy of support for findings, conclusions, and recommendations.

Of the 14 evaluations, 7 had clearly supported findings, conclusions, and recommendations, while 5 had certain limitations in their support for some findings, conclusions, or recommendations. These limitations included insufficient details about data collection and unclear or inappropriate criteria for population selection, given research objectives. However, we determined that, while such limitations can lead to unsupported findings and limit the usefulness of findings, conclusions, and recommendations, the limitations in these 5 evaluations were either clearly stated or otherwise did not substantially detract from the evaluations’ overall purpose or utility.

The 2 remaining evaluations had significant design limitations that suggested a lack of appropriate support for at least one finding, conclusion, or recommendation. For example, the sampling approaches for these evaluations were problematic. Both evaluations selected locations or participants on the basis of convenience, which is a nongeneralizable sampling method. This approach was not appropriate, because the evaluations’ research questions were aimed at generalizing to the entire population or establishing a baseline for the future.

Figure 8 shows the results of our assessment of the 14 evaluations of USAID’s WASH activities.

We found that in some cases, USAID incorporated evaluation results to improve activity monitoring. Our interviews with mission officials and reviews of monitoring documents indicated that the selected USAID missions modified ongoing or planned activities—for at least 6 of the 10 we reviewed that had relevant evaluations—on the basis of evaluation results. For example, the midterm evaluation of the primary WASH activity in Indonesia recommended reducing the activity’s target of 40,000 households willing to pay for sanitation improvements, because of the challenges involved in meeting this goal. As a result, the Indonesia mission modified its agreement with the implementer and reduced the target to 15,000 households. Mission officials noted that the midterm evaluation also provided specific recommendations for future USAID projects that would be taken into consideration during the design process for follow-on WASH investments in Indonesia.

Of the 7 final performance evaluations of WASH activities that we reviewed, 3 were conducted before the activity’s completion, 3 were conducted within 1 month of the activity’s completion, and 1 was conducted within 3 months of the activity’s completion. As a result, these studies were not set up to allow for an assessment or the longer-term sustainability of these projects. To enable assessments of activities’ impact and sustainability, USAID has identified plans to conduct evaluations for some WASH activities several years after project completion. Specifically, USAID’s Water Strategy notes that the agency plans to conduct assessments of WASH sustainability beyond the typical USAID project cycle and to provide support for issues that arise subsequent to completion of WASH activities. Additionally, in an April 2014 document, USAID noted plans to increase its focus on sustaining development outcomes by, among other things, conducting evaluations 3 to 5 years after project conclusion. According to USAID, such long-term evaluations provide opportunities to explore the impact of interventions and may contribute to a deeper understanding of programmatic risk.

Conclusions

Since 2005, USAID has reported that millions have gained access to improved drinking water and sanitation facilities as a result of its assistance. In 2013, USAID issued its first Water Strategy, articulating goals for the provision of sustainable WASH assistance. In response, USAID missions in a number of priority countries are developing a more strategic approach to WASH, and USAID’s Office of Water and missions have begun taking steps to address sustainability of WASH investments. Nevertheless, these efforts are still in the early stages.

However, limitations in some missions’ monitoring and reporting for WASH activities that we reviewed call into question USAID’s ability to reliably assess and report progress toward its strategic WASH goals. Unless USAID identifies and addresses factors contributing to missions’ inconsistent adherence to guidance regarding establishing annual targets for key WASH indicators for all WASH activities and disaggregating activity data by gender, USAID cannot reliably measure these activities’ contributions to achieving WASH goals or toward gender equality and women’s empowerment. Moreover, unless USAID identifies and addresses factors contributing to missions’ inconsistent adherence to guidance for verifying beneficiaries of water activities and accurately reporting beneficiaries of sanitation activities, USAID cannot ensure the accuracy of its annual reports regarding progress in increasing access to safe water and sanitation.

Recommendations for Executive Action

To effectively address limitations in missions’ monitoring and reporting of USAID’s WASH activities, we are making the following two recommendations to the USAID Administrator. Specifically, with respect to inconsistent adherence to agency guidance for establishing annual targets, for reporting gender disaggregated data, for verifying beneficiaries of water activities, and for accurately reporting beneficiaries of sanitation activities, USAID should identify factors contributing to missions’ inconsistent adherence to agency guidance and take steps to address these factors.

Agency Comments and Our Evaluation

We provided a draft of this report to State and USAID. USAID provided written comments, which appear in appendix IV, as well as technical comments that we incorporated as appropriate. State did not provide comments.

In its written comments, USAID generally concurred with our recommendations and outlined steps it is taking to address our second recommendation. Following are highlights of USAID’s comments, with our evaluation: 1. USAID noted that our definition of “obligations” excludes bilateral obligations that have not yet been sub-obligated at the mission-level for WASH activities. As our report states, for the purposes of our review, we defined obligations as orders placed, contracts awarded, and similar transactions during a given period that will require payments during the same or a future period. USAID categorizes these as “sub-obligations,” because it considers these funds to have been obligated through a bilateral agreement with the host country.

In reporting funding for nine selected missions’ WASH activities, we generally reported total allocations for WASH activities ongoing at each mission in fiscal years 2012 through 2014, including, when applicable, funding for years before fiscal year 2012 and through fiscal year 2014. (State and USAID define allocations as the distribution of resources to bureaus and operating units by foreign assistance account.) We included obligations when USAID data did not show WASH activity allocations. 2. USAID stated that it develops targets through multiple processes and that it had provided us with performance plans and reports, which include future-year targets for missions. On the basis of USAID’s technical comments on our draft report, we revised our assessment to acknowledge that the mission-wide performance management plans and reports for fiscal years 2012 and 2013 included annual targets for the two activities we had selected for Zambia (“Schools Promoting Learning Achievement through Sanitation and Hygiene” and “Partnership for Integrated Social Marketing”). Although we included this information in response to USAID’s comments, it is important to note that neither the monitoring plans nor the annual reports for these two activities included annual targets. According to USAID policy (ADS ch. 203), activity-level monitoring plans feed into project-level monitoring plans and mission-wide performance management plans. Therefore, the absence of annual targets in activity-level monitoring plans calls into question the completeness of the Zambia mission- wide performance management plan. Additionally, given that mission- wide performance plans and reports provide only aggregate target and performance data, it may not be possible to identify activity- specific targets or performance data from these reports. 3. USAID stated that our report does not reference certain key documents that USAID had provided. Although USAID’s letter does not specify which key documents it is referring to, in its technical comments, USAID refers to mission-wide performance plans and reports and to State’s annual Water Key Issue Definition guidance. In our report, we refer to performance plans and reports as mission-level annual reporting. In appendix I of our report, we also note that to assess data reliability, we compared implementer reporting on an activity with mission-level annual reporting for the nine selected USAID missions. In addition, our report notes that we reviewed State’s annual Water Key Issue Definition guidance and assessed the extent to which WASH activities in the nine missions generally adhered to this guidance’s requirement regarding activities for which funds can be attributed to the congressional spending requirement for water and sanitation. We described the content of State’s guidance in the background section of our report, where we added a footnote, in response to USAID comments, to more clearly identify the guidance documents we referred to. 4. USAID stated that the inconsistencies identified in our draft report pertaining to the inaccurate categorization of results were the byproduct of isolated incidences of reporting against incorrect indicators. We found inaccuracies in six of eight activities we reviewed that reported on the standard indicator “number of people gaining access to an improved sanitation facility.” While USAID’s comments confirmed these inaccuracies, the agency has not provided any documentation or other support for its statement that they represent isolated incidences. 5. USAID stated that missions are allowed to use custom indicators to track results against water-directive funded activities. We did not intend to imply that use of custom indicators was not in compliance with State and USAID requirements. While we assessed the selected missions’ use of a number of WASH indicators to track progress for 16 selected activities, our review focused in particular on the standard indicators for access to improved water source and improved sanitation. As our report notes, State and USAID have reported overall progress on WASH using these two indicators, and USAID’s Water and Development Strategy (2013-2018) uses these two indicators to project numbers of beneficiaries during the strategy’s 5- year period (i.e., at least 10 million persons would receive sustainable access to improved water supply and 6 million persons would receive sustainable access to improved sanitation). Nevertheless, in our report’s background section, we include a table of standard and custom indicators that USAID allows missions to use in reporting on results of WASH activities. Additionally, on the basis of a follow-up discussion with USAID, we understand that the agency was particularly concerned that our report might seem to imply that its Indonesia mission’s use of a custom indicator for verifying households for the mission’s Water Hibah activity was not in compliance with agency guidance. As a result, we have added a note to figure 6 to clarify that USAID guidance allows missions the flexibility to report using custom indicators. 6. USAID noted that it informed us that verification information could be found in data quality assessments and site visit reports from missions such as Senegal. During our audit work, after the mission informed us that it had conducted verification of beneficiaries through these assessments, we requested data quality assessments from the USAID mission in Senegal. However, the mission did not provide these assessments to us. In the absence of documentation of verification of beneficiaries, we maintain that it is unclear whether the mission verified the number of beneficiaries gaining access to an improved water source.

We are sending copies of this report to the appropriate congressional committees, the USAID Administrator, and the Secretary of State. In addition, the report is available at no charge on the GAO website at http://www.gao.gov.

If you or your staff have any questions about this report, please contact me at (202) 512-3149 or gootnickd@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made major contributions to this report are listed in appendix V.

Appendix I: Objectives, Scope, and Methodology

In this report, we (1) describe the types of water supply, sanitation, and hygiene (WASH) activities the U.S. Agency for International Development (USAID) has implemented in selected countries and the funding it has provided for these activities; (2) assess the extent to which USAID guidance has informed the agency’s efforts to plan and implement WASH activities in these countries; and (3) assess USAID’s monitoring and evaluation of selected WASH activities.

We focused our review on WASH activities that USAID missions had implemented in 9 selected countries: the Democratic Republic of the Congo (DRC), Ethiopia, Haiti, Indonesia, Kenya, Tanzania, Senegal, Uganda, and Zambia. We selected these countries from the list of 22 countries that USAID designated as priority countries for WASH assistance in fiscal year 2014. We based our country selection primarily on the amounts of funding that the missions in these countries attributed to the congressional spending requirement for international water and sanitation assistance in fiscal years 2012 through 2013. During that period, WASH assistance in the 9 selected countries accounted for 53 percent—$155 million—of the funding attributed to the spending requirement by the missions in all tier 1 and tier 2 priority countries. To address our three reporting objectives, we interviewed USAID and Department of State (State) officials and conducted semi-structured telephone interviews with officials at the nine USAID missions regarding their efforts to plan, monitor, and evaluate WASH activities. Because we judgmentally selected the nine USAID missions for our review, our findings from these interviews cannot be generalized to all USAID missions. In addition, we conducted fieldwork in Tanzania and Ethiopia from January 26, 2015, to February 6, 2015. We selected these countries for fieldwork based on factors such as level of funding and type of WASH activities implemented in each country. For example, the Ethiopia mission attributed the most funding to the congressional spending requirement in fiscal years 2012 through 2013 for a range of WASH activities, and the Tanzania mission has implemented one long-standing (since 2009) primary activity focused on WASH.

To describe the types of WASH activities that USAID has implemented and the funding it has provided for these activities, we developed a data collection instrument to obtain funding data and descriptive information from each mission regarding its WASH activities that were ongoing or planned during fiscal years 2012 through 2014. Each mission’s total funding for WASH activities generally represents allocations for WASH components of activities that were ongoing at the mission at any time in fiscal years 2012 through 2014, including, when applicable, WASH funding for years before fiscal year 2012 and through fiscal year 2014. We included obligations in cases for which USAID was not able to provide allocations for WASH activities, such as when a USAID mission obligated unplanned funding to an activity for WASH. To assess the reliability of the data and information we obtained, we reviewed documentation and interviewed agency officials to identify and correct any missing data and any errors. We determined that the data and information we gathered were sufficiently reliable to provide general information about the types of activities implemented and approximate funding provided for these activities.

To assess the extent to which USAID guidance informed planning and implementation of WASH activities, we reviewed USAID guidance, mission-level WASH plans, and other documents. We reviewed USAID guidance including: (1) USAID’s Automated Directives System (ADS), chapter 201, which contains agency policies and procedures and includes guidance on strategic planning and project and activity design; (2) USAID’s Water and Development Strategy, 2013-2018 (Water Strategy); and (3) USAID’s Water and Development Strategy Implementation Field Guide (Field Guide). We assessed the status of the USAID missions’ efforts to develop WASH plans, steps that they have taken to implement these plans, and steps that they have taken to adopt the Water Strategy’s principles and approach for their recent or planned WASH activities. Missions’ WASH plans may consist of one or more project appraisal documents (PAD), which document a project’s design and expected results, as well as a WASH sector assessment or other related documents that describe the mission’s strategic approach to WASH. To assess steps that missions have taken to address sustainability, we reviewed documents that included PADs and sustainability assessments; we also reviewed award documents for selected activities.

We generally selected two activities per country on the basis of factors such as the level of funding allocated and types of WASH activities implemented in fiscal years 2012 through 2014. We based our activity selection on funding data reported in State’s Foreign Assistance Coordination and Tracking System for fiscal years 2012 and 2013, data included in missions’ operational plans for fiscal year 2014, and discussions with mission officials about their WASH activities between fiscal years 2012 through 2014. The activities we selected included those to which the missions in the selected countries had allocated the largest amounts of WASH funding, with one exception: In Uganda, the selected activities did not include the Uganda mission’s activity that received the largest allocations of WASH funding in fiscal years 2012 through 2014, because the mission had allocated the majority of funding for its largest WASH activity before fiscal year 2012, did not initially inform us of this activity, and did not provide the prior-year funding data until after we had made our activity selection. We selected one activity in Tanzania, because the Tanzania mission implemented only one primary WASH activity during the period we reviewed. We initially selected three WASH activities in Haiti. However, the Haiti mission subsequently informed us that two of the selected activities did not have a WASH component and that the mission had incorrectly attributed WASH funds to these activities and planned to take corrective action to ensure use of attributed funds for WASH. As a result, we reviewed one WASH activity for Haiti.

To assess monitoring of WASH activities in the nine countries, we obtained and analyzed documentation for selected WASH activities, including award agreements and modifications, performance management plans, monitoring and evaluation plans, quarterly and annual monitoring reports, and annual funding data. We compared monitoring plans with quarterly and annual reports to determine whether specific WASH indicators had been identified for the activity. Since several activities had multiple WASH indicators, we focused our analysis on standard indicators related to water, sanitation, or hygiene that were used to monitor the activity. When an activity’s monitoring plans included no standard WASH indicators, we identified at least one activity indicator related to access to water, access to sanitation, or hygiene improvement, as relevant. For the purposes of this report, we refer to the indicators that we identified for our review as key WASH indicators. We reviewed each activity’s monitoring plan and monitoring reports to determine whether (1) annual targets were identified for the key WASH indicators and (2) results or performance data for each key WASH indicator were reported on an annual basis, if applicable. Further, we reviewed monitoring reports to assess whether they included gender-disaggregated data for indicators on number of people gaining access to an improved drinking water source or number of people gaining access to an improved sanitation facility. We compared the reported performance data for three key WASH indicators—number of people gaining access to improved drinking water source, number of people gaining access to an improved sanitation facility, and number of liters of drinking water disinfected with point-of-use treatment products as a result of U.S. government assistance—with USAID and State’s guidance for these indicators to assess the extent to which the reported data conformed to the definitions in the guidance document. Although USAID and State guidance required verification of estimated beneficiaries for the indicator for access to an improved water source, the guidance did not require verification of estimated numbers of liters of water purified. In addition to assessing activity performance data against agency guidance, we conducted internal consistency checks to assess the reliability of reported data. For example, to the extent feasible, we compared implementer reporting on an activity with mission-level annual reporting and USAID-wide annual reporting on WASH indicators. Because we found data inconsistencies for several activities, as noted in the report, we did not use the performance data to report the extent to which activities met intended targets.

To assess USAID evaluations for WASH activities in the nine countries, we selected all evaluations for WASH activities completed in fiscal years 2012 through 2014. We identified 14 completed evaluations, including evaluations for 10 of the 16 activities we selected to assess monitoring. To assess the soundness of the evaluations, we reviewed background information about programs and evaluation questions, assessed evaluation design and process, and considered the evaluation results and limitations of each study. Two GAO specialists conducted these assessments independently, using a tool that incorporated key elements of USAID’s scope-of-work checklist for evaluations and considered various aspects of these issues. The two specialists compared the results of their independent assessments and came to agreement about all conclusions. We did not assess selected USAID missions’ compliance with the requirement to conduct evaluations.

To review background information about the evaluated programs, we considered whether the evaluations considered evaluator independence, program objectives and mechanisms, and evaluation goals. We also assessed whether the relationship between the evaluation objectives and program design was clear and appropriate.

To assess evaluation design and process, we considered whether the evaluations clearly described their design; whether appropriate methods were used to select the locations and people covered by the study, including whether the evaluations provided sufficient detail about sampling methods; whether measures used were clearly related to evaluation questions; and whether data collection and analysis were sufficient and appropriate. We also assessed whether selection methods, sample sizes, criteria, measures, data collection, analysis, and overall design were appropriate, given the evaluation objectives.

To consider evaluation results and limitations, we determined whether findings, conclusions, recommendations, and lessons learned were clearly stated, whether stakeholders were given an opportunity to comment on the results, and whether evaluations provided information about how results should be used. We also assessed whether the evaluations clearly and sufficiently described assumptions and limitations of their design and results, including potential biases, confounding variables, unintended consequences, alternative explanations, and methodological limitations. We assessed whether any findings, conclusions, recommendations, or lessons learned were appropriately supported and caveated, given the evaluation design. To summarize evaluation results, we determined whether the sections of evaluations related to program monitoring, outcomes, or sustainability contained descriptive information or appropriately supported findings, conclusions, recommendations, or lessons learned.

We conducted this performance audit from July 2014 to October 2015 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Appendix II: USAID WASH Activities in Nine Selected Countries

Following are funding data and descriptive information about water, sanitation, and hygiene (WASH) activities implemented in fiscal years 2012 through 2014 in the nine countries that we selected for our review— the Democratic Republic of the Congo, Ethiopia, Haiti, Indonesia, Kenya, Tanzania, Senegal, Uganda, and Zambia.

Democratic Republic of the Congo

The U.S. Agency for International Development (USAID) mission in the Democratic Republic of the Congo implemented seven WASH activities in fiscal years 2012 to 2014, with WASH activity funding totaling $26,125,075 (see table 4). The mission reported two primary WASH activities—the Integrated Health Project and Sustainable WASH Interventions: Healthy Villages Program in Two Health Zones—focused on improving access to water, sanitation, and hygiene services in target locations. Allocations of WASH funding for these two activities totaled $22,518,470.

Ethiopia

The USAID mission in Ethiopia implemented 11 WASH activities in fiscal years 2012 through 2014, with WASH activity funding totaling $24,055,770 (see table 5). The mission’s largest WASH activity, called Water, Sanitation, and Hygiene Transformation for Enhanced Resilience, included $10,984,723 in funding for WASH and focused on providing water infrastructure in pastoralist areas.

Figure 9 shows a drinking water reservoir and a water point constructed through the Ethiopia mission’s largest WASH activity.

Haiti

The USAID mission in Haiti implemented six WASH activities in fiscal years 2012 through 2014, with WASH activity funding totaling $4,436,481 (see table 6). The mission’s largest WASH activity, called Santé pour le Développement et la Stabilité d’Haiti, included $1,711,000 in funding for the WASH component of a broader activity that focused on improving the health status of Haitians through improved primary care, referral networks, and management practices at health facilities and in communities.

Indonesia

The USAID mission in Indonesia implemented five WASH activities in fiscal years 2012 through 2014, with WASH activity funding totaling $53,401,700. The mission’s largest WASH activity, called Indonesia Urban Water, Sanitation, and Hygiene, included $38,696,403 in WASH funding and focused on providing access to water and sanitation facilities in urban areas.

Kenya

The USAID mission in Kenya implemented 19 WASH activities in fiscal years 2012 through 2014, with WASH activity funding totaling $18,245,655. The mission’s largest WASH activity, called AIDS, Population and Health Integrated Assistance Program (APHIA) Plus Northern Arid Lands Service Delivery, included $3,024,887 in funding for WASH. The activity focuses on HIV/AIDS, maternal and child health, WASH, and nutrition for orphaned and vulnerable children. The mission reported five similar activities targeting other regions of the country.

Senegal

The USAID mission in Senegal implemented five WASH activities in fiscal years 2012 through 2014, with WASH activity funding totaling $27,616,000. The mission’s largest WASH activity, called Senegal Millennium Water and Sanitation Program, included $20,866,000 in funding for WASH. The activity focused on governance and management, the creation of local business opportunities, increasing demand for clean water and sanitation, the construction of infrastructure, and hygiene.

Tanzania

The USAID mission in Tanzania implemented three WASH activities in fiscal years 2012 through 2014, with WASH activity funding totaling $17,753,586. The mission’s primary WASH activity, called Tanzania Integrated Water, Sanitation and Hygiene Program, included $17,443,586 in funding for WASH. The activity supported community piped and gravity-fed water schemes, rehabilitated wells with rope pumps, training for community groups responsible for operations and maintenance of water schemes, and school latrines. Activity also included a water resource management component.

Figure 10 shows a demonstration rope pump, which the Tanzania Integrated Water, Sanitation and Hygiene Program supported as a cost- effective, easy-to-maintain technology, and a piped water point in the village of Mvumi, Wami-Ruvu River Basin, Tanzania.

Uganda

The USAID mission in Uganda implemented 15 WASH activities in fiscal years 2012 through 2014, with WASH activity funding totaling $15,284,415. The mission’s largest WASH activity, called Northern Uganda Development of Enhanced Local Governance, Infrastructure, and Livelihoods, included $6,225,000 in funding to support local government efforts in northern Uganda to expand basic WASH services. Another activity, called WASHPlus: Supportive Environments for Healthy Communities, included $500,000 in funding for WASH and aimed to build the capacity of district government and USAID implementing partners for WASH efforts to support community-led total sanitation, promoting handwashing in villages, and integrating WASH with nutrition and HIV/AIDS services and programs.

Zambia

The USAID mission in Zambia implemented three WASH activities in fiscal years 2012 through 2014, with WASH activity funding totaling $27,600,000. The mission’s largest WASH activity, called Schools Promoting Learning Achievement through Sanitation and Hygiene, included $15,200,000 in funding for WASH and focused on providing WASH services in schools. Efforts included hygiene education, capacity building for operations and maintenance, and support to establish private- sector spare-parts supply.

Appendix III: USAID Evaluations’ Assessment of Monitoring, Outcomes, and Sustainability

Our review of 14 U.S. Agency for International Development (USAID) evaluations of water, sanitation, and hygiene (WASH) activities in nine selected countries found that the evaluations assessed monitoring, outcomes, and sustainability to varying extents.

Monitoring. Evaluations reported on various issues, including monitoring, WASH indicators, indicators’ limitations, and data related to gender.

All 14 evaluations reported on monitoring of WASH activities.

Ten evaluations reported on one or more of USAID’s indicators related to access to drinking water, sanitation, and hygiene, including whether WASH activities were on track to meet their targets.

Five evaluations discussed limitations in the quality of one or more indicators used to monitor WASH activities. For example, an evaluation of Senegal’s Yaajeende activity noted limitations of the indicator for the number of individuals trained on improved hygiene behaviors. The evaluation stated that, according to USAID’s guidance, the success of training and other interventions related to human and organizational capacity building is to be measured by improvement in organizational output and performance, not simply by the number of individuals trained.

Five evaluations presented and discussed data related to gender, thereby filling some gaps that we had identified related to a lack of disaggregated gender data in monitoring reports. For example, the evaluations for the primary WASH activities in Ethiopia and Senegal assessed women’s participation in community water management committees, which were generally responsible for operations and maintenance (see fig. 11).

Outcomes. Six of the 14 evaluations that we reviewed provided insights into WASH activity outcomes. Specifically, these evaluations had findings related to outcomes such as disease incidence, health expenses, school attendance, time spent getting water, economic impacts, and beneficiary satisfaction. For example, the evaluation of Ethiopia’s Water Sanitation and Hygiene Transformation for Enhanced Resiliency found that the activity resulted in access to safe water at a much closer distance than before and also increased access to safe latrines and improved health practices (such as handwashing). In addition, the evaluation found that activity results included increased time for beneficiaries to participate in other productive and income-generating activities, including more time at school, as well as reduced health expenses.

Sustainability. Twelve of the 14 evaluations that we reviewed addressed the sustainability of WASH activities. Specifically, these evaluations broadly discussed WASH sustainability issues, and 7 of the 12 had findings related to WASH sustainability challenges. These challenges included limitations related to capacity building, a lack of spare parts, and a lack of funding for operations and maintenance. For example, the evaluation of Tanzania’s Integrated Water, Sanitation, and Hygiene Program reported on factors that improved sustainability, such as the ease of repairing a rope pump. The Tanzania evaluation also noted challenges related to capacity building for community water committees. The evaluation assessed 10 of 26 committees as having “fair” usage and maintenance, where community fees were generally not collected or maintenance was spotty, and rated 6 of the 26 as “poor” for underperformance relative to the rest of the project.

Appendix IV: Comments from the U.S. Agency for International Development

Appendix V: GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

In addition to the contact named above, Emil Friberg (Assistant Director), Mona Sehgal (Assistant Director), Lisa Helmer, Mitchell Delaney, Jesse Elrod, Reid Lowe, Bethany Patten, Steven Putansu, and Monica Savoy made significant contributions to this report. Mark Dowling, Jon Melhus, and Ozzy Trevino provided technical assistance.