metadata: authors: 0: author: Andres,Luis Alberto
1: author: Deb,Saubhik
2: author: Gambrill,Martin P.
3: author: Giannone,Elisa
4: author: Joseph,George
5: author: Kannath,Pramod
6: author: Kumar,Manish
7: author: Kurian,P.K.
8: author: Many,Rajesh
9: author: Muwonge,Abdu
count: India
docna: 0: docna: Sustainability of demand responsive
            approaches to rural water supply : the case of Kerala
docty: Policy Research Working Paper
subtopic: Water Supply and Systems,Rural Water Supply and Sanitation,Rural & Small Town Water & Sanitation,Health Economics & Finance,Water and Human Health,Hydrology,Water Supply and Sanitation Economics,Small Private Water Supply Providers,Rural Water,Town Water Supply and Sanitation
teratopic: Water Resources,Water Supply and Sanitation,Rural Development
abstracts: cdata!: This paper presents the findings of an
            impact evaluation to assess the performance and
            sustainability of the demand responsive community-based
            approach toward rural water supply in the state of Kerala,
            India. To achieve the study's objectives, conceptual
            definitions of the "performance" and
            "sustainability" of rural water supply schemes
            were first developed, as were indicators for their
            systematic measurement. Performance and sustainability
            indicators for demand responsive approaches were compared
            with the more conventional supply-based approach to rural
            water supply. The study found that participatory community
            driven water supply schemes were more successful in
            delivering adequate, regular, and quality water supply,
            experienced fewer breakdowns and water shortages, and
            enjoyed higher consumer satisfaction with the quality of
            service delivery. The success of the community-based
            approach demonstrates that people are willing to contribute
            toward the capital costs of the schemes and pay for the
            water they use for a better service delivery. The findings
            of this paper suggest that the community-based approach can
            be a superior alternative to traditional supply driven
            models in expanding and improving water service delivery in
            rural areas.
geo_reg_key: 517191,517212,517198
repnme: repnme: Sustainability of demand responsive
            approaches to rural water supply : the case of Kerala
display_title: Sustainability of demand responsive
            approaches to rural water supply : the case of Kerala
listing_relative_url: /research/2017/04/27329170/sustainability-demand-responsive-approaches-rural-water-supply-case-kerala
topicv3: Political and Institutional Sustainability
            m1326466 1095,Water m1326522 137,Water Supply m1326961 138,Rural Water Supply m1327209 143,Water Scarcity m1504471 3390
publishtoextweb_dt: 2017-04-04T00:00:00Z
historic_topic: Water Resources,Water Supply and Sanitation,Rural Development
no_of_pages: 30
author_keys: 0: author_key: 676776
1: author_key: 916767
2: author_key: 97253
3: author_key: 1338131
4: author_key: 912994
5: author_key: 1260738
6: author_key: 915921
model_extractions: WPS8025 Policy Research Working Paper 8025 Sustainability of Demand Responsive Approaches to Rural Water Supply The Case of Kerala _Luis Andres Saubhik Deb Martin Gambrill Elisa Giannone George Joseph Pramod Kannath Manish Kumar P.K. Kurian Rajesh Many Abdu Muwonge_ Water Global Practice Group April 2017
Policy Research Working Paper 8025

# **Abstract**

This paper presents the findings of an impact evaluation to assess the performance and sustainability of the demand responsive community-based approach toward rural water supply in the state of Kerala, India. To achieve the study’s objectives, conceptual definitions of the “performance” and “sustainability” of rural water supply schemes were first developed, as were indicators for their systematic measurement. Performance and sustainability indicators for demand responsive approaches were compared with the more conventional supply-based approach to rural water supply. The study found that participatory community driven water supply schemes were more successful in delivering adequate, regular, and quality water supply, experienced fewer breakdowns and water shortages, and enjoyed higher consumer satisfaction with the quality of service delivery. The success of the community-based approach demonstrates that people are willing to contribute toward the capital costs of the schemes and pay for the water they use for a better service delivery. The findings of this paper suggest that the community-based approach can be a superior alternative to traditional supply driven models in expanding and improving water service delivery in rural areas.

This paper is a product of the Water Global Practice Group. It is part of a larger effort by the World Bank to provide open access to its research and make a contribution to development policy discussions around the world. Policy Research Working Papers are also posted on the Web at http://econ.worldbank.org. The authors may be contacted at Landres@worldbank.org.

_The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent._ Produced by the Research Support Team
# Sustainability of Demand Responsive Approaches to Rural Water Supply: The Case of Kerala

Luis Andres, Saubhik Deb, Martin Gambrill, Elisa Giannone, George Joseph, Pramod Kannath, Manish Kumar, P.K. Kurian, Rajesh Many, and Abdu Muwonge<sup>1</sup>

## JEL Classification: D02; Q01; Q25

Key Words: Drinking water; rural water schemes; demand responsive approach; sustainability.

> 1 The authors are thankful to Maria Angelica Sotomayor, Srinivasa Rao Podipireddy, Miguel Vargas Ramirez, Smita Misra, and Junaid Kamal Ahmad, Country Director for his invaluable comments.

> This paper is a product of the Water Global Practice. It is part of a larger effort by the World Bank to provide open access to its research and make a contribution to development policy discussions around the world. Policy Research Working Papers are also posted on the Web at http://econ.worldbank.org. Usual disclaimer applies. The authors may be contacted at <u>landres@worldbank.org.</u>
# 1. Introduction

In the early 1990s there was increasing recognition of the failure of supply driven water and sanitation service provision. Rural water schemes built during this period tended to focus more on the construction of facilities to increase access quickly without paying sufficient attention to the post construction phase. This often led to costly reinvestment after the collapse of the water schemes. From the mid- to late1990s, given the pervasive problems relating to the performance and sustainability of rural water infrastructure in the developing world, the World Bank and other development partners worked to develop new approaches to implementing rural water supply schemes which were intended to improve the sustainability of the resultant services. At this time, national and state governments, in their effort to achieve long-term sustainability and improved performance of water supply services, started to focus on institutional arrangements that would ensure the involvement of beneficiaries in system planning, design, construction and management, in order to facilitate cost recovery as well as to improve the operation and maintenance (O&M) of the water schemes. There was growing consensus, both in academic and policy/implementation spheres, that water supply interventions needed to be ‘demand responsive’ – i.e. they needed to include community participation and community contributions towards capital and O&M costs in order to increase the ownership and the sustainability of the schemes and to reduce a community’s dependence on higher levels of government for sustainable rural water supply provision (Sara and Katz, 1997).

In Kerala in southern India, there is a coexistence of different service delivery approaches for the provision of rural water supply within the same state. Until the late 1990s, the Kerala Water Authority (KWA), which followed a ‘supply driven’, or top-down, engineering-based, approach to the provision of water supply, was the only agency responsible for piped water supply and sanitation service delivery in the state’s rural villages and small towns. In 1997, under an initiative to decentralize service delivery, the Government of Kerala (GoK) allowed the Local Self-Government Institutions (LSGIs) to take over existing small water supply schemes from KWA or to implement their own new standalone water supply projects. In 2000, with a view to furthering the decentralization efforts, GoK decided to empower Beneficiary Groups (BGs), created at the community level, to carry out the following tasks related to rural water supply schemes in the state to: (i) make planning and investment decisions, (ii) manage development funds; (iii) lead/supervise construction; and (iv) operate and maintain the resultant systems. The Kerala Rural Water Supply and Sanitation Agency (KRWSA) was set up in 1999 as a Special Purpose Vehicle to plan, implement and supervise a World Bank financed Kerala Rural Water Supply and Sanitation Project, (the ‘ _Jalanidhi Project_ ’). Between 2000 and 2008, GoK successfully implemented the Jalanidhi project which was designed to follow a ‘ _demand responsive approach_ ’ (DRA) to rural water supply, implementation of which encompassed, among other things, beneficiary participation, capital cost contributions from beneficiaries and from the rural local governments ( _Gram_ 2
_Panchayats_ , or GPs), a policy of providing universal household connections within the BGs,<sup>2</sup> and full O&M cost recovery from user fees.

There is a growing volume of literature on the sustainability of community driven rural water supply schemes which mostly investigate the success and the risk factors for the equitable and effective operation of these schemes. Capitalizing upon the coexistence of the different approaches to rural water supply provision in Kerala, namely those following a DRA and those that are supply driven, this paper examines and compares the sustainability and performance of rural water schemes built under various regimes, and provides recommendations for future reforms in the rural water sector. The paper aims to contribute to this literature by: (i) proposing a multidimensional definition of ‘ _sustainability_ ’ and of ‘ _performance_ ’ of rural water schemes; (ii) selecting a matched sample of similar schemes from the demand and the supply approaches in order to ensure comparability between them; and (iii) using impact evaluation techniques for analyzing relative performance, strengths, and weakness of the water supply schemes across various dimensions of performance and sustainability. The paper provides evidence on the effectiveness of DRA vis-à-vis the more traditional supply driven approaches to the implementation of water supply schemes using quasi-experimental matching techniques identifying comparable supply driven and demand driven schemes and analyzing their performances and relative strengths and weaknesses.

The rest of the paper is organized as follows: Section 2 provides a background on DRA followed by a discussion on different types of water supply arrangements in rural Kerala. Issues related to the measurement of performance and sustainability of water supply schemes are addressed in Section 3. Section 4 introduces the empirical strategy including the data sources and the econometric model used for comparing the performances of water supply schemes. This is followed in Section 5 by a discussion of the descriptive data as well as empirical results from the econometric estimations. Section 6 concludes by highlighting the implications of this study for future reforms in the rural water sector in India and, by extension, in other developing countries.

# 2. Background

## 2.1 _Background on the demand responsive approach_

Worldwide, 80 percent of the people who have limited access to drinking water supplies live in rural areas (United Nations, 2010). But the sustainability of rural water infrastructure has been a critical challenge mostly due to the remoteness of rural locations and the lack of financial and technical capabilities to operate and maintain schemes in these areas. One of the United Nations’ 2000

> 2 Kerala is one of the pioneering states to introduce universal household connections for rural water supply under the DRA. It was not previously the norm in Kerala and still is not in most of rural India.

3
Millennium Development Goals (MDGs) was to increase the proportion of the world’s population that has access to safe drinking water and basic sanitation (United Nations, 2010). While the international community has made advancements toward these goal over the past two decades, progress in rural areas is lagging when compared to urban areas (United Nations, 2011). The recent Sustainable Development Goals built on the MDGs proposed a higher measure of access to safely managed water (United Nations, 2015).

Many countries focused on construction of facilities to expand access quickly. However, inadequate attention to post construction O&M led to subsequent collapse of many of these schemes and need for further reinvestment. For instance, a 2009 Water Aid study from Tanzania found that nearly half of improved public water points in rural areas are not functioning, and 25 percent of systems are inoperable after only two years following installation (Taylor, 2009). Similar findings were reported in Nigeria (Andres et al., 2016). These systems fail at such high rates in large part because sustainability of rural water systems in low income countries depends on “the relationship of the user with the life cycle of the water systems” (Jones, 2012).

So to achieve long-term sustainability, governments and development partners started to focus on institutional arrangements that would ensure involvement of users in planning, implementation, and O&M of schemes and financial contribution by users to at least cover the O&M costs. It was important to have infrastructure sustainability, as there was enough evidence that by 2025, nearly 1.4 billion people, amounting to a quarter of the world’s population or a third of the population in developing countries, are destined to face absolute water scarcity (Cosgrove, 2003).

Community driven projects with active beneficiary participation in planning and implementation are likely to be more responsive to the needs of the beneficiaries in creating infrastructure, giving communities control over decisions, improving service delivery, creating ownership, and strengthening the capacity of the communities to undertake other development activities (Chambers, 1983; Sen, 1999; Dongier et. al., 2003). Literature on performance assessment of various community driven, participatory water supply schemes shows that such projects can create effective infrastructure and improve performance of water supply schemes. Participatory-demand-driven models for provision of rural water supply have been found to be successful in delivering well-designed and functioning systems in Ghana and Peru (Thorsten II, 2007). Marks and Davis (2012) reveal that demand based community participation in building drinking water systems increases the community’s sense of ownership for the water system, and improves the functioning of rural water projects in rural Kenya. Isham and Kahkonen (1999) found that greater community participation is associated with improved service delivery in India and Sri Lanka. An assessment of ten community driven projects in Benin, Bolivia, Honduras, Indonesia, Pakistan, and Uganda shows that community driven projects with active beneficiary participation are likely to be more sustainable (Sara and Katz, 1997). Similarly, a more recent study in rural areas of 4
Pakistan found that community participation is crucial for developing ownership and for ensuring longterm sustainability of rural water supply projects (Haq, Hassan and Ahmad, 2014). Several studies have also highlighted the importance of capacity development and institutional support to ensure the longrun sustainability of these projects. An impact evaluation of small community water systems in Bolivia funded by the Bolivian Social Investment Fund found that training and capacity development of communities are crucial for improved performance of these schemes in terms of access and availability of water (Newman et. al., 2002). In Malawi, newer community driven rural water supply schemes were found to be performing better than the older ones, indicating poor sustainability of the schemes due to lack of institutional support (Kleemeier, 2000). In Suriname, socially appropriate technological choice along with involvement and support of the community in general and women in particular were found to be the factors crucial for success of community driven water supply projects (Smith, 2011).

While assessments of various participatory community driven water supply projects have found evidence supporting their success in improving service delivery, there is very little evidence on the relative effectiveness of community driven projects compared to traditional supply driven projects. A study of rural water supply schemes from ten states (including Kerala) in India found that community managed schemes performed ‘somewhat better’ than traditional supply driven schemes (Misra, 2008). Though the study did not focus on a detailed comparison, it found that compared to traditional supply driven schemes, community managed schemes experience less frequent breakdowns and also lesser proportion of beneficiary households report water shortage.

# 2.2 _Various Approaches to Rural Water Supply in Kerala_

According to the 2011 Population Census, Kerala is the home to 33.4 million people which constitutes about 2.8 percent of the total population in India. Although the state receives one of the highest levels of rainfall in the country, with an average of 3,000mm annually, the undulating terrain of the state drains most of the rainwater into the sea. Denudation of tropical forests and vegetation in the aftermath of population explosion adversely affected the natural recharge of aquifers and the water retention capacity of the soil. In addition, the steep and crowded topography provides little opportunity for water storage. With increasing demand for water due to rising population, the groundwater has been over exploited with insufficient recharge thus decreasing water tables.<sup>3</sup> As a result, several districts in Kerala face widespread source failures during summer months and many habitations in midland and highland face near drought conditions. In addition, Kerala also struggles with water quality issues with near universal bacteriological contamination in the open wells; and presence of excess iron, fluoride, salinity, and excess nitrate in ground water.

> 3 10 years ago a bore-well struck water at 80 feet below the ground, but now it touches only after 140 feet (World Bank 2013).

5
KWA was created in 1984 as an autonomous organization under the GoK and was entrusted with the responsibility of providing piped water supply and sanitation services to both urban and rural areas of the state. Under KWA large investments were undertaken in creating infrastructure and expanding piped water supply coverage throughout the state. However, O&M of the infrastructure created received far less attention. As a result, assets created often suffered from lack of proper management and maintenance leading to sub-optimal scheme performance and dysfunctional schemes. Moreover, there was limited scope for large projects in rural hinterlands due to dispersed settlements.

In 1997, the GoK embarked on a major decentralization process which involved transfer of financial resources, decision making authority and the responsibility for the provision of certain essential services, including rural water supply and sanitation to the local governments. Under this, GOK entrusted the local authorities with the responsibility of water and sanitation service delivery and took the decision to transfer all small rural water supply schemes to GPs with concomitant power to levy and collect user charges for providing water services.

In 1999, the Government of India launched substantial sector reforms geared towards DRA for rural water supply and sanitation service provision with the launch of the Sector Reform Project (SRP) which was further scaled up as Swajaldhara in 2002. As a beneficiary of Indian programs, the GoK pioneered rural water reforms by carrying out the World Bank-financed Jalanidhi (‘water is treasure’) project between 2000 and 2009. Jalanidhi covered 112 (11%) of the state’s GPs spanned across 13 districts (out of 14) and implemented 3,694 small water supply schemes (mostly groundwater based) and 16 large surface water based schemes. The project followed a DRA encompassing beneficiary participation, capital cost contributions from beneficiaries and the GPs, universal household connection provision, full O&M cost recovery from user fees, and an integrated strategy to the water, sanitation, environment and health sectors. Major policy reforms were successfully implemented in the Jalanidhi GPs by which RWSS service responsibility was transferred to local governments and BGs with concurrent empowerment and accountability measures. By the end of its implementation period, the project had helped provide access to improved water services to an additional 1.3 million people in the state.

The DRA piloted under Jalanidhi aimed to empower BGs to make investment decisions, manage development funds, and plan, construct and operate water supply schemes. The novel DRA contrasts significantly with the traditional supply driven approach in which projects are designed, implemented, and operated by the KWA that put more emphasis on construction and less on operational and financial sustainability of the schemes built. Jalanidhi facilitated active participation of the community including vulnerable groups, women, and indigenous population in planning, construction and O&M of water supply schemes to ensure sustainability of infrastructure.

6
Around the same time as Jalanidhi, several other community driven projects were implemented in different parts of Kerala. SRP was implemented in Kasaragod and Kollam districts of Kerala by the respective District Panchayats. In Kasaragod, Malappuram, Palakkad, and Thrissur districts, Swajaldhara schemes were implemented through KWA. Whereas Jeevadhara was implemented in Idukki and Alappuzha districts by an NGO, Socio Economic Unit Foundation (SEUF), with financial support from the Government of the Netherlands. Like Jalanidhi, all these projects followed the community based DRA in varying degrees.

In this paper, we compare the performances of the community driven, demand responsive Jalanidhi schemes with the traditional supply driven schemes built and managed by the KWA (KWA-BM). In addition, we analyze the performance of Jalanidhi schemes vis-à-vis other community managed schemes that include SRP, Swajaldhara, Jeevadhara as well as schemes built and transferred by KWA (KWA-BT) to local institutions for operation and management.

# 3. Measuring the Performance of Water Supply Schemes

Measuring the performance of various water supply schemes requires a multidimensional approach that would not only capture performance, i.e. quality of service delivery, but also the factors that are critical for the long-term sustainability of the schemes. Accordingly, six indexes have been identified – three indexes to measure quality of service delivery and one index each to capture operational, financial, and institutional sustainability of water supply schemes. The service delivery indexes include _Availability and Reliability_ index to capture quantity, quality, reliability, and adequacy of water supply; _Household Satisfaction_ index capturing performance rating of water schemes by the household; and _Cost of Service_ Index measuring affordability of service. The sustainability indexes include _O&M_ index measuring the quality of O&M in terms of frequency and length of service disruptions; _O&M Cost Recovery_ index capturing financial sustainability; and _Institutional Sustainability_ index measuring the quality of the institutions (BGs) created for the day to day management of the community based schemes. In addition, an Overall Performance index has been calculated by aggregating the above six indexes.

The selection of various indices and their constituent indicators has been informed by the existing literature (Sara and Katz, 1997; Abrams et al., 1998; Carter et al., 1999; Sugden, 2001; Harvey and Reed, 2004; Mishra, 2008; Montgomery et al., 2009; Mazango and Munjeri, 2009, among others) as well as consultations with water practitioners. A detailed description of the constituent indicators of the six indexes is presented below:

- i) _Availability and Reliability Index:_ (a) Water supplied every day (Yes=1/No=0); (b) No. of days per week water supplied (Days); (c) Adequate water supply (Yes=1/No=0); (d) No irregular supply (Yes=1/No=0); (e) No bad taste (Yes=1/No=0); (f) No Bad odor (Yes=1/No=0); (g) No

7
colored water (Yes=1/No=0); (h) No cloudy water (Yes=1/No=0); (i) No low water pressure (Yes=1/No=0); and (j) No seasonal shortage (Yes=1/No=0).

- ii) _Household Satisfaction Index:_ Household satisfaction with (a) Overall services of water supply; (b) Water quality; (c) Water pressure; (d) Hours of supply; and (e) Regularity of supply. All the indicators are household ratings from 1 (Very Dissatisfied) to 5 (Very Satisfied).

- iii) _Household Cost of Service Index:_ (a) Monthly water charges ( ₹ ); (b) Household opinion on capital contribution/connection charges for the water scheme (High=1, Fair=2, Low=3); and (c) Household opinion on monthly water tariff (High=1, Fair=2, Low=3).

- iv) _Operation and Maintenance Index:_ (a) Yearly laboratory testing for water quality (Yes=1/No=0); (b) Number of water system breakdowns (no water for one day or more) during the last one year; (c) Number of days the longest breakdown lasted during the last one year; and (d) Scheme facing no water shortage anytime during last year (Yes=1 / No=0).

- v) _O&M Cost Recovery Index:_ (a) Annual revenue from water sales as proportion to annual O&M costs; and (b) Proportion of schemes with full O&M cost recovery.

- vi) _Institutional Sustainability Index:_ (a) Proportion of female members in Beneficiary Group (BG) Executive Committee; (b) Female President in BG (Yes=1/No=0); (c) Female Secretary in BG (Yes=1/No=0); (d) Female Treasurer in BG (Yes=1/No=0); (e) Female Pump Operator in BG (Yes=1/No=0); (f) Regular election to the Executive Committee (Yes=1/No=0); (g) Presentation of Annual Report to the General Body (Yes=1/No=0); (h) Maintenance of Records (Yes=1/No=0); (i) Investments in water schemes made by the BG post-Project Completion (Yes=1/No=0); (j) BG with savings bank account (Yes=1/No=0); and (k) Current balance in the savings account.

Finally, to construct the performance and sustainability indexes, continuous variables among the constituent indicators were converted to z-scores. Z-scores are standardized values with a mean of 0 and standard deviation of 1. All these z-scores as well as the binary indicators were aggregated and then standardized again to estimate the index. Constituent indicators for which higher values are associated with worse performance were assigned a negative sign during aggregation to ensure that for the standardized index a positive z-score is associated with a better than average performance. An overall performance index was constructed by aggregating all the six indexes and then converting them to z- scores.

Based on these indexes, a systematic assessment of the performance and sustainability of traditional supply driven approach as well as the more recent community based approaches to rural water supply schemes in Kerala was undertaken. This exercise was intended to examine and compare the sustainability and performance of rural water schemes built under different institutional regimes and provide recommendations for future reforms in the rural water sector.

8
# 4. Empirical Strategy and Data

## 4.1 _Identification of Sample_

Selection of the sample has been guided by the need to identify schemes that are very similar in characteristics but under different types of institutional arrangements so that the differences in performance across scheme types can be solely attributed to their respective institutional arrangements. Accordingly, Jalanidhi schemes have been matched with KWA-BM schemes and other community managed schemes based on four characteristics – water source, age of the scheme, size of the schemes defined by population coverage, and distance between the treatment and control schemes. Since geographical coordinates were not available for the majority of the schemes, latitude and longitude data for the GPs where the schemes are located have been used to calculate the distance between the schemes. In the first step of the matching exercise, schemes from different institutional arrangements that use similar type of water source (river, open dug well, bore well, ponds, lakes) and have similar age profile have been identified. In the next step, matched pairs have been identified by selecting schemes that are closest to each other with a similar population coverage Using the matching exercise, a total of 200 similar and comparable water schemes from the above mentioned three groups have been identified which formed the final matched sample. The final sample consists of 90 Jalanidhi schemes, 44 KWA-BM schemes and 66 other community managed schemes. The distribution of the final matched sample by institutional arrangement is presented in the table below.

**Table 1: Distribution of Final Matched** **<u>Sample by Scheme Type</u>**

|**Scheme**|**Identified**|**Out of**|**Built**|
|---|---|---|---|
|Jalanidhi-I|90|3,710|2001 to 2009|
|KWA-BM|44|395|2004 & 2005|
|Other CommunityManaged|66|750|1999 to 2010|
|**Total**|**200**|**4,855**||

## 4.2 _Data_

Data collection for the selected sample was undertaken using household surveys of beneficiaries, technical and financial audits of water schemes, and institutional assessment of BGs for the Jalanidhi and other community managed schemes. The following is a brief description of the data collection tools used in the study.

_Technical and Financial Audit:_ The technical audit was focused on assessing the current state of water supply infrastructure such as working of the pumps, condition of the reservoir, functioning of the water treatment plan, frequency of breakdowns, frequency of water quality testing, total daily supply of water with respect to the design criteria, water source reliability, quality, and household service level. The 9
financial audit gathered information on O&M cost, water tariff, connection charges, and revenue collection from the water tariff _Household Survey:_ Household surveys were conducted to find out the benefit and satisfaction from the service provided. The survey asked questions related to availability of water at the household level, adequacy of water, the quality of water provided, reliability of service etc. It also included questions related to satisfaction with service quality, and affordability of water tariff and capital contribution/connection charges.

_Institutional Assessment:_ The BG survey was conducted to assess the institutional strength of the community management schemes. The process included interviews of the key stakeholders from the Beneficiary groups as well as focus group discussions of members of the BGs. Institutional assessments collected information on composition of executive committee, frequency of meetings, regularity of election to the Executive Committee, maintenance of records, preparation of annual report, and financial and investment decisions of the BGs.

Field surveys were undertaken from March to June 2014. The survey team was unable to locate some of the schemes. A total of 2,583 households from 157 schemes were surveyed. Technical and financial audits were carried out in 172 schemes. Moreover, 135 BGs were surveyed for the institutional assessment. Table 2 provides a summary of the data collected by scheme type through various survey instruments.

**<u>Table 2: Data Collection by Survey Instrument and Scheme Type</u>**

|**Dataset**|**Household**<br>**Survey**|**Technical**<br>**Audit**|**Financial**<br>**Audit**|**Institutional**<br>**Assessment**|
|---|---|---|---|---|
|Jalanidhi|87|87|87|90|
|KWA BM|22|35|35||
|Other Community Managed<br>Schemes|48|50|50|45|
|Total No of Schemes|157|172|172|135|
|Number of Observations|2,582|172|172|135|

Since the analysis is at the scheme level, data collected through household surveys were aggregated at the scheme level. The technical and financial audits as well as the institutional surveys collected data at the scheme level only. We used imputation technique to handle missing data due to non-response and lack of information. Data imputation at the household level was done only if at least 20 percent of the surveyed households from a particular scheme responded to the question. The missing value was predicted based on the information collected from the 20 percent (or more) households such as education and proxies for income and other characteristics of the schemes. To impute data at the scheme 10
level, we predicted the missing observation with the value obtained by running a regression using other characteristics of the scheme as predictor.

# 4.3 _Methodology_

We compared the means of the indexes and their underlying indicators for Jalanidhi, KWA-BM, and other community managed schemes to assess their relative performance and strength and weakness. This was supplemented by a propensity score matching (PSM) analysis using the data from the technical audits of our sample to estimate the average treatment effect on treated (ATT) of the Jalanidhi schemes vis-à-vis KWA-BM schemes and other community managed schemes. PSM exercise was carried out to improve upon the initial matching exercise which was constrained by the availability of reliable secondary data.

For the PSM exercise, Jalanidhi was defined as the treatment and the control group was selected from the KWA-BM/other community managed schemes. The objective of the PSM was to construct two statistically matched samples from treatment and control groups based on various scheme characteristics. In other words, PSM would select schemes from Jalanidhi and match them with those schemes from KWA-BM that have characteristics similar to the selected Jalanidhi schemes. The matching is done based on propensity scores or probabilities estimated from a Probit model using Jalanidhi schemes as a binary dependent variable and scheme characteristics as regressors. For the Probit specification, we used age of scheme, designed per capita supply of water, availability of water treatment facility, and reliance on dependable sources of water (such as perennial river, deep tube wells, and bore wells) as proxies for scheme characteristics.

For constructing the matched sample, we implemented a radius matching with caliper. Following Stuart (2010), we chose a caliper of 0.2 standard deviation of the estimated propensity scores. We also tested for balance in the matched sample to ensure that the treatment and control groups are comparable. Finally, we estimated the ATT on various performance and sustainability indexes to capture the impact of Jalanidhi schemes vis-à-vis KWA-BM/other community managed schemes.

11
# 5. Empirical Results

In this section, we discuss the summary statistics of various indexes of scheme functionality and their constituent indicators for Jalanidhi, KWA-BM, and other community managed schemes. This is followed by a discussion of the results from the PSM exercise.

## 5.1 _Summary Statistics_

Indicators relating to Water Availability and Reliability are based on data from household surveys aggregated at the scheme level. Around 50 percent of Jalandihi schemes and 60 percent of other community managed schemes are supplying water every day of the week. The number is much lower for the KWA schemes where only 27 percent of the schemes were reported supplying water daily. On average, water is available only for 3.7 days for Jalanidhi and other community managed schemes and for 3.2 days for traditional supply driven KWA schemes. Moreover, the supply was considered adequate for 63 percent of Jalanidhi schemes compared to 62 percent for other community managed schemes and 55 percent for KWA schemes. Regarding regularity of water supply, 36 percent of Jalanidhi schemes were reported supplying water irregularly compared to 43 percent of other community managed schemes and 41 percent of KWA schemes. The majority of the respondents from the community schemes found water pressure in the network adequate. Around 33 percent of Jalanidhi schemes and 38 percent of other community managed schemes were reported having low water pressure in the network. In contrast, beneficiaries of around 70 percent of the traditional supply driven schemes complained about low water pressure. Similarly, beneficiaries of around 70 percent of the traditional supply driven schemes reported facing water shortage during summer months compared to only 38 percent for other community managed schemes and 49 percent of Jalanidhi schemes. The community schemes also perform better in terms of quality of water supplied. Beneficiaries of more than 50 percent of the KWA schemes reported getting colored water compared to only 8 percent for Jalanidhi schemes and 13 percent for other community managed schemes.

So Jalanidhi and other community managed schemes perform better compared to traditional supply driven KWA schemes in terms of availability, adequacy and quality of water supply. However, there is no major systemic difference in performance between Jalanidhi and other community managed schemes. Jalanidhi schemes perform marginally better in areas of regularity and adequacy of water supply whereas other community managed schemes perform better in weekly frequency of water supply and lack of seasonal shortage.

12
**Table** **<u>3: Availability and Reliability Index</u>**

|**Indicators**|**Jalan**<br>**Sche**|**idhi**<br>**mes**|**KWA**<br>**Man**<br>**Sche**|**Built &**<br>**aged**<br>**mes**|**Ot**<br>**Com**<br>**Man**<br>**Sch**|**her**<br>**munity**<br>**aged**<br>**emes**|
|---|---|---|---|---|---|---|
||**Mean**|**SD**|**Mean**|**SD**|**Mean**|**SD**|
|Water Supplied Everyday|0.49|0.5|0.27|0.46|0.62|0.49|
|No. of Days Per Week Water Supplied|3.71|0.86|3.21|0.95|3.78|0.97|
|Adequate Water Supply|0.63|0.49|0.55|0.51|0.62|0.49|
|No Irregular Supply|0.64|0.48|0.59|0.5|0.57|0.5|
|No Bad Taste|0.94|0.23|0.91|0.29|0.96|0.2|
|No Bad Odor|0.95|0.21|0.95|0.21|0.98|0.15|
|No Colored Water|0.92|0.27|0.45|0.51|0.87|0.34|
|No Cloudy Water|0.86|0.35|0.64|0.49|0.83|0.38|
|No Low Pressure|0.67|0.47|0.32|0.48|0.62|0.49|
|No Seasonal Shortage|0.51|0.5|0.32|0.48|0.62|0.49|

The findings of relative performances of schemes in various dimensions of availability, reliability, and adequacy are also corroborated by the household assessments of satisfaction with the performances of these schemes (Table 4). The households were asked to rate on a 5-point scale – from 1 to 5 with 5 being very satisfied – their satisfaction with respect to water quality, water pressure in the network, hours of supply, regularity of supply and overall satisfaction with service delivery. The results show that the Jalanidhi and other community driven schemes have been rated consistently higher than the KWA schemes in all these areas. A comparison of Jalanidhi and other community managed schemes indicates that both these schemes have very similar satisfaction ratings.

**Table 4: Household** **<u>Satisfaction Index</u>**

|**Indicators (Scale of 1-5,**<br>**5 being very satisfied)**|**Jala**<br>**Sch**<br>**Mean**|**nidhi**<br>**emes**<br>**SD**|**KWA**<br>**Manage**<br>**Mean**|**Built &**<br>**d Schemes**<br>**SD**|**Other C**<br>**Manage**<br>**Mean**|**ommunity**<br>**d Schemes**<br>**SD**|
|---|---|---|---|---|---|---|
|Overall Satisfaction|3.59|0.62|3.11|0.62|3.52|0.59|
|Water Quality|3.89|0.65|3.54|0.52|3.88|0.57|
|Water Pressure|3.59|0.61|3.12|0.7|3.62|0.53|
|Hours of Supply|3.47|0.76|3.06|0.76|3.48|0.75|
|Regularity ofSupply|3.47|0.76|2.86|0.74|3.31|0.88|

All the schemes surveyed charge a flat monthly tariff for water. Monthly water charges are lowest for the KWA schemes at around ₹ 41 per month on average and cheaper by more than ₹ 20 compared to Jalanidhi and other community driven schemes (Table 5). Households were also asked to rate the appropriateness of capital cost contributions/connection charges and monthly water tariff. All these water-related charges were considered to be mostly fair by the households irrespective of the scheme type. Interestingly, in spite of having higher tariff, households served by the community managed schemes considered the tariffs to be fair. High ownership, involvement and quality of service associated 13
with the community driven schemes possibly explain the sense of satisfaction with the water tariffs even when they are considerably higher than the traditional supply driven schemes.

# **Table 5: Household Cost of Service Index**

|**Indicators**|**Jala**<br>**Sch**<br>**Mean**|**nidhi**<br>**emes**<br>**SD**|**KWA B**<br>**Managed**<br>**Mean**|**uilt &**<br>**Schemes**<br>**SD**|**Other Com**<br>**Managed S**<br>**Mean**|**munity**<br>**chemes**<br>**SD**|
|---|---|---|---|---|---|---|
|HH Opinion on Contribution<br>(High=1,Fair=2,Low=3)|2.08|0.26|2.53|0.72|2.25|0.53|
|HH Opinion on Tariffs<br>(High=1,Fair=2,Low=3)|2.02|0.28|2.19|0.57|1.95|0.2|
|Monthly Water Charges (₹)|62.88|24.11|41.25|13.15|65.00|24.18|

The O&M index was constructed completely based on data collected through technical audits and aims to capture the operational sustainability of the schemes. Unlike the community managed schemes, the majority of the KWA schemes carried out yearly water quality testing. 64 percent of KWA schemes reported carrying out yearly water quality testing compared to only 21 percent of Jalanidhi schemes and 13 percent of other community managed schemes. However, Jalanidhi schemes on average have fewer breakdowns and fewer days of water outages compared to either KWA schemes or the other community managed schemes. Also fewer percentage of community managed schemes (both Jalanidhi and other community schemes) reported facing water shortage anytime of the year compared to supply driven KWA schemes.

**Table** **<u>6: Operation and Maintenance Index</u>**

|**Indicators**|**Jalan**<br>**Sche**<br>**Mean**|**idhi**<br>**mes**<br>**SD**|**KWA**<br>**Managed**<br>**Mean**|**Built &**<br>**Schemes**<br>**SD**|**Other Co**<br>**Managed**<br>**Mean**|**mmunity**<br>**Schemes**<br>**SD**|
|---|---|---|---|---|---|---|
|Yearly Analysis of Water Quality|0.21|0.41|0.64|0.49|0.13|0.34|
|Number of Breakdowns Last Year|3.16|2.34|5.17|4.87|5.46|5.05|
|Longest Breakdown Last Year (Days)|10.64|14.78|11.94|12.92|12.99|15.02|
|Scheme Facing No Water Shortage<br>Anytime Last Year|0.54|0.5|0.36|0.49|0.47|0.5|

The financial audits indicate that full O&M cost recovery remains a challenge for many of the community driven schemes in spite of the fact that on average revenue from water sales exceeds the O&M costs for both Jalanidhi and other community managed schemes (Table 7). Around 50 percent of the community managed schemes reported to achieve full O&M cost recovery. Revenue as proportion to O&M costs is higher for other community managed schemes compared to Jalanidhi schemes. But these other community managed schemes also show large variations in performance as reflected by high standard deviation. Cost recovery indicators were not reported by most of the KWA schemes. So a comparative analysis with traditional supply driven schemes could not be undertaken.

14
**<u>Table 7: O&M Cost Recovery Index</u>**

|**Indicators**|**Jalani**<br>**Sche**|**dhi**<br>**mes**|**Other Co**<br>**Managed**|**mmunity**<br>**Schemes**|
|---|---|---|---|---|
||**Mean**|**SD**|**Mean**|**SD**|
|Revenue as Proportion to O&M Costs|1.21|0.77|1.74|2.28|
|ProportionofSchemes with FullO&MCostRecovery|0.47|0.5|0.5|0.51|

Performance of Jalanidhi schemes in most of the dimensions of institutional sustainability is better compared to other community driven schemes (Table 8). Almost one-third of the BG executive committee members of Jalanidhi schemes are women with 9 percent of the BGs having a woman president and 14 percent having a woman secretary. Almost half of the Jalanidhi BGs have women treasures compared to one-fourth for the other community managed schemes. Jalanidhi BGs also have more regular elections for the executive committees and 74 percent of these BGs reported preparing and presenting Annual Reports in general body meetings compared to 64 percent of the BGs for the other community managed schemes. Similarly, a larger majority of the Jalanidhi BGs reported having a savings bank account and the average balance in these accounts is almost three times compared to other community managed schemes. However, the average balance even for the Jalanidhi BGs is only ₹ 17,161 which severely limits the ability of the majority of the BGs to undertake big investments when necessary.

**<u>Table 8: Institutional Sustainability Index</u>**

|**Indicators**|**Jalanidhi**<br>**Mean**|**Schemes**<br>**SD**|**Other Co**<br>**Managed**<br>**Mean**|**mmunity**<br>**Schemes**<br>**SD**|
|---|---|---|---|---|
|Proportion of Female in BG Executive|||||
|Committee|0.32|0.19|0.27|0.20|
|Female President in BG|0.09|0.28|0.11|0.32|
|Female Secretary in BG|0.14|0.34|0.16|0.37|
|Female Treasurer in BG|0.49|0.5|0.25|0.44|
|Female Pump Operator in BG|0.13|0.33|0.08|0.27|
|Regular Election (Yes/No)|0.66|0.48|0.41|0.5|
|Annual Report (Yes/No)|0.74|0.44|0.64|0.48|
|Maintenance of Records (Yes/No)|0.16|0.37|0.00|0.00|
|Investments post-Project Completion (Yes/No)|0.13|0.33|0.11|0.31|
|Proportion of BGs with Savings Bank Account|0.70|0.46|0.54|0.50|
|Current Balance in the Savings Account (₹ )|17,161|26,217|6,181|7,472|

Table 9 and Table 10 compare the performance indexes of Jalanidhi schemes with traditional supply driven KWA schemes and the other community managed schemes respectively. The indexes have been computed separately for two groups – Jalanidhi-KWA schemes and Jalanidhi-other community driven schemes. As a result, the mean values of the indexes for Jalanidhi schemes differ across tables. Since the performance indexes are z-scores, the mean of each of these indices is 0 by construction. So construction of indices separately for the two groups makes the comparisons straightforward. Positive 15
values for an index indicate above average performances whereas negative values are associated with below average performances. For Household Cost of Service Index, the definition of the Index also differs across two groups. Monthly water charges were not included in the Index for comparison of Jalanidhi and KWA schemes because of non-availability of data for the majority of KWA schemes. For similar missing data problems, “No of days water supplied per week” was not included in the construction of the availability and reliability index for any of the comparisons. Similarly, comparison of the O&M cost recovery index was not undertaken for either of the two groups because of the large number of missing observations. Moreover, the institutional sustainability index was not included in Jalanidhi-KWA comparison because the index measures the strength of water user groups and is not relevant for the KWA schemes.

A comparison of performance indexes for Jalanidhi and traditional supply driven KWA schemes shows that on average, Jalanidhi schemes perform better than KWA schemes in most of the dimensions including availability and reliability, household satisfaction, O&M as well as overall performance. However, in terms of household costs of service, KWA schemes perform better than Jalanidhi schemes.

16
**<u>Table 9: Comparison of Performance Indices – Jalanidhi & KWA-BM Schemes</u>**

|**Performance Indices**|**Jalanidhi**<br>|**Schemes**<br>|**KWA Built**<br>**Sche**<br>|**& Managed**<br>**mes**<br>|
|---|---|---|---|---|
||**Mean**|**SD**|**Mean**|**SD**|
|Availability & Reliability Index|0.14|0.99|‐0.57|0.85|
|Household Satisfaction Index|0.16|0.97|‐0.63|0.88|
|Household Cost of Service Index|‐0.17|0.70|0.66|1.61|
|Operation & Maintenance Index|0.05|0.97|‐0.18|1.12|
|Overall PerformanceIndex|0.07|0.96|‐0.27|1.11|

**<u>Table 10: Comparison of Performance Indices – Jalanidhi & Other Community Schemes</u>**

|**Performance Indices**|**Jalanidhi**|**Schemes**<br>|**Other Co**<br>**Managed**|**mmunity**<br>**Schemes**<br>|
|---|---|---|---|---|
||**Mean**|**SD**|**Mean**|**SD**|
|Availability & Reliability Index|‐0.01|1.01|0.02|0.99|
|Household Satisfaction Index|0.02|1.02|‐0.04|0.97|
|Household Cost of Service Index|0.00|1.06|‐0.01|0.89|
|Operation & Maintenance Index|0.17|0.86|‐0.30|1.16|
|Institutional Sustainability Index|0.13|0.96|‐0.91|0.81|
|Overall Performance Index|0.13|0.96|‐0.76|0.99|

The relative performances are mixed when Jalanidhi schemes are compared with other community managed schemes. On average, Jalanidhi schemes perform better in areas of household satisfaction, household cost of service, O&M, institutional sustainability, and overall performance. Community managed schemes perform better in availability and reliability of service. However, for many of these indices, the differences in z-scores between these two groups are relatively small.

# 5.2 _Results from Propensity Score Matching_

A propensity score matching exercise was done to assess the performances of Jalanidhi schemes vis-àvis other approaches by using propensity scores to identify comparable treatment and control groups from the surveyed schemes. There were two comparisons done for the analysis. First, the performance of the decentralized demand responsive Jalanidhi schemes were assessed using the traditional supply driven KWA-BM schemes as control group. In the second analysis, Jalanidhi schemes were compared with the other community managed schemes. The PSM analysis was done using the ‘psmatch2’ command in STATA. Since ‘psmatch2’ does not provide the correct standard-errors and t-statistics for the estimates of ATT, the respective standard errors and t-statistics were estimated using bootstrapping. Balance tests were also carried out to check for comparability of treatment and control groups. The “Rubin’s B” and “Rubin’s R” statistics from balance tests indicate overall balance in the sample. The test results are reported in Table A1 and Table A2 in the appendix.

17
Table 11 presents the ATT estimates from comparison of matched Jalanidhi and KWA built and managed schemes for selected indicators.<sup>4</sup> The ATT estimates indicate that although for most of the constituent indicators of availability and reliability index Jalanidhi schemes perform better than the KWA schemes, the differences between them are not significant except for adequate water pressure in the network, no colored water, and no seasonal shortage. In the matched sample, 96 percent of the Jalanidhi schemes reported no colored water compared to 35 percent of the KWA schemes. Similarly, 49 percent of Jalanidhi schemes reported no seasonal shortage and 64 percent of Jalanidhi schemes reported adequate pressure in the network. The respective proportions for KWA schemes are only 4 percent and 15 percent. Jalanidhi schemes also have been rated consistently and significantly higher for all dimensions of household satisfaction. However, for satisfaction with water quality rating, the difference is only significant at the 10 percent level. There are no significant differences between Jalanidhi and KWA schemes in terms of household opinion regarding connection charges and water tariffs. Similarly, there are also no significant differences between these two types of schemes in terms of longest breakdown of the system or the percentages of schemes facing no water shortage anytime last year. However, Jalanidhi schemes experienced significantly lower number of breakdowns than the KWA schemes, whereas significantly higher proportion of KWA schemes carried out yearly testing of water quality compared to Jalanidhi schemes.

**Table 11: PSM Comparison of Jalanidhi with KWA Built & Managed Schemes: Selected** **<u>Indicators</u>**

|**Variable**|**Jalanidhi**|**KWA-**<br>**BM**|**ATT**|**SE**|**P-value**|
|---|---|---|---|---|---|
|**Availability & Reliability Index**||||||
|No Irregular Supply|0.62|0.41|0.20|0.23|0.379|
|No Low Pressure|0.64|0.15|0.49*|0.22|0.022|
|No Seasonal Shortage|0.49|0.04|0.45**|0.14|0.001|
|**Household Satisfaction Index**||||||
|Overall Satisfaction|3.5|2.9|0.64**|0.20|0.001|
|Regularity of Supply|3.4|2.5|0.97***|0.24|0.000|
|**Household Cost of Service Index**||||||
|HH Opinion on Contribution|2.1|2.4|-0.29|0.32|0.362|
|HH Opinion on Tariffs|2.1|2.0|0.05|0.19|0.813|
|**Operation & Maintenance Index**||||||
|Yearly Analysis of Water Quality|0.28|0.69|-0.41+|0.24|0.082|
|Number of Breakdowns Last Year|3.4|8.7|-5.3**|1.93|0.006|
|Scheme Facing No Water Shortage|0.60|0.39|0.20|0.23|0.367|
|� p�0.1, *p<0.05,**p<0.01, *** p<0.001||||||

> 4 ATT estimates for full set of indicators for both Jalanidhi and KWA-BM comparison and Jalanidhi-other community managed schemes comparison are presented in the Appendix (Table A3 and Table A4 respectively).

18
The ATT estimates for comparison of Jalanidhi and other community managed schemes indicate no significant differences across various indicators related to availability and reliability, household satisfaction and household cost of service (Table 12). Jalanidhi schemes, however, experienced fewer breakdowns compared to other community managed schemes. Jalanidhi schemes also perform significantly better in certain dimensions of institutional sustainability. Jalanidhi schemes have a larger proportion of females in executive committees and significantly higher bank balances compared to other community managed schemes. Similarly, a higher proportion of Jalanidhi schemes have female treasurers and maintain records regularly.

**Table 12: PSM Comparison of Jalanidhi with Other Community Managed Schemes: Selected**

**<u>Indicators</u>**

|**Variable**|**Jalanidhi**|**Other**<br>**Bottom-up**|**ATT**|**SE**|**P-value**|
|---|---|---|---|---|---|
|**Availability & Reliability Index**||||||
|No Irregular Supply|0.62|0.53|0.10|0.13|0.443|
|No Low Pressure|0.65|0.55|0.10|0.16|0.539|
|No Seasonal Shortage|0.49|0.54|-0.05|0.13|0.723|
|**Household Satisfaction Index**||||||
|Overall Satisfaction|3.5|3.5|0.04|0.15|0.820|
|Regularity of Supply|3.4|3.2|0.19|0.23|0.415|
|**Household Cost of Service Index**||||||
|HH Opinion on Contribution|2.1|2.2|-0.10|0.14|0.489|
|HH Opinion on Tariffs|2.0|1.9|0.11|0.07|0.108|
|**Operation & Maintenance Index**||||||
|Number of Breakdowns Last Year|3.471|5.9|-2.4***|0.72|0.001|
|Scheme Facing No Water Shortage|0.532|0.462|0.07|0.14|0.601|
|**Institutional Sustainability Index**||||||
|Proportion of Female in BG Executive<br>Committee|0.322|0.251|0.07+|0.04|0.098|
|Maintenance of Records (Yes/No)|0.163|0.006|0.16**|0.05|0.004|
|CurrentBalanceinthe SavingsAccount|15,967|3,096|12,871***|2,568|0.000|

� p�0.1, * p<0.05, ** p<0.01, *** p<0.001 The ATT estimates of the aggregate performance indexes indicate that Jalanidhi schemes perform better than the KWA-BM schemes in Availability and Reliability, Household Satisfaction, O&M and Overall Performance (Table 13). The ATT estimate for availability and reliability Index is around 1.0 which means that that the respective z-score for Jalanidhi scheme are on an average higher than the z-scores of comparable KWA schemes by 1. Since the difference between z-scores is not readily interpretable, the mean z-scores of the matched Jalanidhi and KWA schemes have been converted to percentile rankings. ATT can then be interpreted as the average difference in percentile rankings between similar Jalanidhi and KWA schemes. Figure 1 shows that for overall performance, availability and reliability, household satisfaction and O&M, Jalanidhi schemes on average ranks 30 to 40 percentile points higher 19
than comparable KWA schemes. However, the percentile ranking of KWA schemes is higher but not significant for household cost of service.

**Table 13: PSM Comparison of Jalanidhi with KWA Built & Managed Schemes: Performance Indexes**

|**Index**||**Jalanidhi**|**KWA**|**ATT**|**SE**|**P‐value**<br>|**Percentile**<br>**Rank**<br>**(Jalanidhi)**|**Percentile**<br>**Rank**<br>**(KWA)**|
|---|---|---|---|---|---|---|---|---|
|Availabilit|y & Reliability|0.124|‐0.872|0.997**|<br>0.36|0.006|55.0|19.2|
|Household|Satisfaction|0.117|‐1.002|1.119***|<br>0.33|0.001|54.7|15.8|
|Household|Cost of Service|‐0.092|0.212|‐0.304|0.57|0.595|46.4|58.4|
|Operation|& Maintenance|0.197|‐0.756|0.953*|0.41|0.021|57.9|22.5|
|Overall Pe<br>*p<0.05,**p|rformance<br><0.01, *** p<0.001|0.161|‐0.925|1.086**|<br>0.40|0.007|56.4|17.8|
|60.0|**Figure 1:**<br>**Ja**|**Differenc**<br>**lanidhi a**|**es in Pe**<br>**nd KW**|**rcentile**<br>**A-BM S**|**Ranks**<br>**cheme**|**between**<br>**s**|||
|40.0<br>50.0|35.8|38|.8|||354|38.6||
|‐40.0<br>‐30.0<br>‐20.0<br>‐10.0<br>0.0<br>10.0<br>20.0<br>30.0<br><br>Difference in Percentile Rank|Availability &<br>Reliability Index|Household<br>Satisfaction In|dex<br>House<br>Serv|‐12.0<br>hold Cost of<br>ice Index|Operati<br>Maintenan|.<br>on &<br>ce Index<br>Perf|Overall<br>ormance Index||

PSM comparisons of Jalanidhi and other community managed schemes show no significant differences in performances in terms of availability and Reliability index, household satisfaction index, and household cost of service index (Table 14). The Jalanidhi schemes perform significantly better than the other community managed schemes in the O&M index, institutional sustainability index and overall performance index. Percentile ranks of Jalanidhi schemes for the O&M index are around 20 percentile points higher than similar community managed schemes (Figure 2). Likewise, when the indexes were aggregated into an overall performance index, percentile ranks of Jalanidhi schemes are on average 19 percentile points higher than comparable community managed schemes. However, the institutional sustainability performance of Jalanidhi schemes is significantly stronger than the other community 20
managed schemes. When compared to similar community managed schemes, Jalanidhi schemes ranked 27 percentile points higher on average.

**<u>Table 14: PSM Comparison of Jalanidhi with Other Community Managed Schemes</u>**

|**Index**|**Jalanidhi**|**Other**<br>**Bottom-**<br>**up**|**ATT**|**SE**|**P-value**|**Percentile**<br>**Rank**<br>**(Jalanidhi)**|**Percentile**<br>**Rank**<br>**(Other**<br>**Bottom-up)**|
|---|---|---|---|---|---|---|---|
|Availability & Reliability|-0.032|-0.117|0.085|0.25|0.731|48.7|45.3|
|Household Satisfaction|-0.049|-0.032|-0.018|0.24|0.942|48.0|48.7|
|Household Cost of Service|-0.046|-0.189|0.143|0.28|0.609|48.2|42.5|
|Operation & Maintenance|0.182|-0.325|0.507*|0.22|0.023|57.2|37.3|
|Institutional Sustainability|0.239|-0.452|0.691**|0.23|0.003|59.4|32.6|
|Overall Performance|0.131|-0.341|0.472*|0.22|0.031|55.2|36.6|

* p<0.05, ** p<0.01, *** p<0.001

**Figure 2: Differences in Percentile Ranks between Jalanidhi and Other Bottom-up Schemes**

<!-- Start of picture text --> 40.0<br>35.0<br>30.0 26.9<br>25.0<br>20.0<br>18.5<br>20.0<br>15.0<br>10.0<br>3.4 5.7<br>5.0<br>0.0<br>‐0.7<br>‐5.0<br>‐10.0 Availability & Household Household Operation & Institutional Overall<br>‐15.0 Reliability Satisfaction Cost of Service Maintenance Sustainability Performance<br>Index Index Index Index Index Index<br>Difference in Percentile Rank<br><!-- End of picture text -->

# 6. Conclusion

There is a large volume of literature that looks into the quality, success and risk factors associated with participatory community driven approaches to rural water supply. But very few studies delved into the relative effectiveness of these community driven schemes compared to traditional supply driven schemes, which still remain the dominant approach to service delivery in rural areas in many developing countries. This paper developed a conceptual definition of performance of water schemes in the context of rural water supply schemes in Kerala and compared performances of the flagship demand responsive Jalanidhi schemes with the traditional supply driven KWA schemes as well as other community managed schemes. Our results indicate that Jalanidhi schemes were more successful in delivering 21
adequate, regular and quality water supply in rural areas compared to the KWA schemes. Jalanidhi schemes also reported fewer breakdowns and water shortages indicating better O&M. The demand responsive community based approach of Jalanidhi was more successful in generating higher consumer satisfaction with service delivery and cost of service in spite of charging higher monthly tariffs compared to the KWA schemes. Overall, the Jalanidhi schemes performed better than the KWA schemes in all important dimensions in which comparisons were done.

When compared to other community managed schemes, Jalanidhi schemes performed better in O&M and overall performance. But the main success of Jalanidhi was in creating stronger institutions, which is one of the prerequisites for the long-term sustainability of the community based approach. The substantial time and effort that was spent in mobilizing communities, creating capacities, and involving communities in planning and implementation of the Jalanidhi schemes possibly explain their superior institutional performance compared to other community managed schemes. However, achieving full O&M cost recovery remains an elusive goal for Jalanidhi schemes in particular and all community managed schemes in general. This coupled with relatively low bank balances for the majority of the BGs severely limits their ability to undertake expensive maintenance work when needed, which in turn might threaten the long-term sustainability of the community based approach to rural water supply.

The findings of this paper suggest that the community based approach can be a superior alternative to traditional supply driven models in expanding and improving water service delivery in rural areas. Success of the community based approach demonstrates that the people are willing to contribute towards the capital costs of the schemes and pay for the water that they use for better service delivery. However, to ensure long-term sustainability of the community based schemes, more attention needs to be paid in creating stronger beneficiary level institutions, including capacity development for financial management for successful operation and management of these schemes. In addition, institutions need to be created to provide operational and financial support to these schemes when needed.

# References

Abrams, L., Palmer, I., and Hart, T. (1998), Sustainability Management Guidelines. Pretoria: Department of Water Affairs and Forestry, South Africa Andres, L. and B. Dasgupta (2016), “Functionality of water schemes in Nigeria” World Bank. (Forthcoming).

- Black, Maggie, 1978-1998 Learning What Works, “A 20 Year Retrospective View on International Water and Sanitation Cooperation.” World Bank, “World Bank Policies on Participation”. (2011). Online at: <u>http://web.worldbank.org/WBSITE/EXTERNAL/TOPICS/EXTSOCIALDEVELOPMENT/EX</u> ~ ~ ~

- <u>TPCENG/0,,menuPK:410312 pagePK:149018 piPK:149093 theSitePK:410306,00.html</u>

22
Brikke, F. and Bredero, M. (2003). Linking Technology Choice with Operation and Maintenance in the Context of Community Water Supply and Sanitation. Geneva: World Health Organization and IRC Water and Sanitation Centre.

- Carter, R. C., Tyrrel, S. F., and Howsam, P. (1999), Impact and sustainability of Community water Supply and sanitation Programmes in Developing Countries, Chartered Institution of Water and Environmental Management, 13, 292-296.

Chambers, Robert. 1983. Rural Development: Putting the First Last. London: Longman.

Cosgrove, W. J. (2003): ‘Water Resource Sector in the Coming Decades: Global Perspective’ in K. Prasad (ed.) Water Resource and Sustainable Development: Challenges of 21st Century. New Delhi: Sipra Publications.

- Dongier, P., J. V. Domelen, E. Ostrom, A. Ryan, W. Wakeman, A. Bebbington, S. Alkire, T. Esmail, and M. Polsky. 2003. “Community Driven Development.” In World Bank, PRSP Sourcebook. Vol. 1. Chapter 9, Washington, D.C.

- Haq, M.A, S.M. Hassan, and K. Ahmad (2014), Community Participation and Sustainability of Water Supply Program in District Faisalabad, Pakistan, Journal of Quality and Technology Management, Volume X, Issue II, Page 125 – 137.

Harvey, P. A., and Reed, R. A. (2004). Rural water supply in Africa: Building blocks for hand pump sustainability. WEDC, Loughborough University. United Kingdom.

Interco-operation Social Development India (ISDI), Impact Evaluation of the Kerala Rural Water Supply and Sanitation Project, Jalanidhi Phase 1, 2014.

Isham, J. and Kähkönen, S. (1999) “Institutional Determinants of the Impact of Community-Based Water Services: Evidence from Sri Lanka and India.” Working Paper 236. University of Maryland, Center for Institutional Reform and the Informal Sector.

Jalanidhi: A Project of Government of Kerala added by the World Bank, Kerala Rural Water Supply and Sanitation Agency, Online at: http://www.jalanidhi.kerala.gov.in/ Jones, S.; Abseen, A.; Stacey, N.; Weir, L. A life-cycle approach to improve the sustainability of rural water systems in resource-limited countries. Challenges 2012, 3, 233–260.

Kleemeier, Elizabeth. 2000. “The Impact of Participation on Sustainability: An Analysis of the Malawi Rural Piped Scheme Program.” World Development 28(5): 929–44.

Mazango, N., and Munjeri, C. (2009), Water management in a hyperinflationary environment: Case study of Nkayi district in Zimbabwe, Physics and Chemistry of the Earth, Parts A/B/C, 34(1-2), 28-35.

Misra, S. (2008) Inefficiency of Rural Water Supply Schemes in India. Policy Paper number 44791 extracted from the World Bank Study on Review of Effectiveness of Rural Water Supply Schemes in India, June 2008.

Montgomery, M. A., Bartram, J., and Elimelech, M. (2009), Increasing functional sustainability of water and sanitation supplies in rural sub-saharan Africa, Environmental Engineering Science, 26(5), 1017-1023.

23
Newman, J. et al (2002) “An Impact Evaluation of Education, Health and Water Supply Investments by the Bolivian Social Investment Fund.” World Bank Economic Review, 16(2), 241-74.

- Rosengrant, M, X. Kai and S. Cline (2002): World Water and Food to 2025. Washington, D.C.: International Food Policy Research Institute.

- Rout, Satyapriya, University of Hyderabad, Institutional variations in practice of demand responsive approach: evidence from rural water supply in India, Water Policy, July 2014.

Sara, J. and Katz, T. (1997) Making Rural Water Supply Sustainable: Report on the Impact of Project Rules. Washington D.C.: UNDP-World Bank Water and Sanitation Program.

Seckler, D, R. Barker and U. Amarasinghe (1999): ‘Water Scarcity in the Twenty-first Century’, International Journal of Water Resource Development, 15 (1/2), 29–42.

Sen, Amartya K. 1999. Development as Freedom. New York: Knopf Press.

Sugden, S. (2001). Assessing sustainability-the sustainability snap shot. Paper presented at the 27th WEDC conference, Lusaka, Zambia.

Taylor, B. (2009) Addressing the Sustainability Crisis: Lessons from research on managing rural water projects. Available online at: <u>http://www.wateraid.org/~/media/Publications/sustainability-crisisrural-water-management-tanzania.pdf</u>

- United Nations. (2010). the Millennium Development Goals Report 2010. Retrieved from <u>http://www.un.org/millenniumgoals/</u>

- United Nations. (2011). The Millennium Development Goals Report 2011 Retrieved from <u>http://www.un.org/millenniumgoals</u>

United Nations. (2015). Transforming Our World: The 2030 Agenda for Sustainable Development. Retrieved from http://www.un.org/ga/search/view_doc.asp?symbol=A/RES/70/1&Lang=E Voluntary Organization for Social Action and Rural Development (VOSARD), Conservation of Natural Resources, Drinking Water Management, Jalanidhi Project, Video online at: <u>http://www.vosard.org/conservation%20of%20natural%20resources.html</u> World Bank 2013 A -“World Development Indicators”, Data, Rural population (% of total population), 2010-2014, online at: <u>http://data.worldbank.org/indicator/SP.RUR.TOTL.ZS</u> World Bank 2013 B- World Bank, India: Getting Water on Tap in Rural Kerala, August, 2013, Online at: <u>http://www.worldbank.org/en/news/feature/2013/08/12/india-getting-water-on-tap-in-ruralkerala</u>

- World Bank 2013 C- Project Performance Assessment Report, Republic of India, Kerala Rural Water Supply and Environmental Sanitation Project “Jalanidhi”, June 27, 2013, Independent Evaluation Group Public Sector.

- World Bank 2009- Implementation Completion and Results Report, Republic of India for the Kerala Rural Water Supply and Environmental Sanitation Project, March 9, 2009, South Asia Sustainable Development Unit, The World Bank Group.

- World Bank 2006 - Background Paper, Rural Water Supply and Sanitation, India Water Supply and Sanitation, Bridging the Gap between Infrastructure and Service, January 2006, the World Bank Group.

24
25