Summary of Evaluation
1. Outline of the ProjectCountry: People’s Republic of Bangladesh

Project Title: Safe Motherhood Promotion Project
Phase 2 (SMPP-2)

Issue/ Sector: HealthCooperation Scheme: Technical Cooperation Project

Division in Charge: Human Development Total Cost: 376 million Japanese yenDepartment(as of terminal evaluation)

Period of1 July 2011 – 30 June 2016 Partner Country’s Implementing Organization:

Cooperation:(five years) (RD signed on 29 Ministry of Health and Family Welfare (MoHFW)
May 2011)

Supporting Organization in Japan:
Project Supporting Committee in Japan is organized.

Related(Technical cooperation)

Cooperation

Dispatch of health advisor (2010-2014)

Project:(Loan)
Maternal, Neonatal and Child Health Improvement Project (Phase 1)(Loan agreement
signed on 25 January 2012)(Others)
Japan Overseas Cooperation Volunteers (Nurse, Rural development, Program manager)

1-1. Background of the Project
Safe Motherhood Promotion Project (Phase 1) (SMPP-1), which was launched by the Ministry ofHealth and Family Welfare (MoHFW), the Government of People’s Republic of Bangladesh, and JICA in
2006, terminated its cooperation in June 2011 with a number of improvements in health service
utilization and health seeking behaviors of mothers. SMPP-1 had the combination approach ofcommunity empowerment, medical facility improvement and mobilization of local governments to link
health service

providers and communities.

Having recognized the achievements of SMPP-1, the Government of Bangladesh and JICA
commenced the SMPP-2 to further strengthen maternal and neonatal health (MNH) activities in line withthe “Health, Population, and Nutrition Sector Development Strategy 2011-2016” of Bangladesh byextending their cooperation as follows: 1) to enhance coordination mechanism in maternal and neonatal
health activities among governments, development partners and NGOs at the national level, 2) todisseminate good practices and experiences of SMPP-1 in the country, and 3) to define replicable andeffective maternal and neonatal health approaches based on SMPP-1 and other good practices.
1-2. Project Outline(1) Overall Goal
Maternal and neonatal health status is improved in Bangladesh.
(2) Project Purpose
The approaches to improve MNH services quality and utilization in line with Health, Population,xi

and Nutrition Sector Development Program (HPNSDP) are expanded in Bangladesh.
(3) Outputs
1)

Good practices of MNH services are identified and consolidated in national strategies andguidelines

2)

Mechanism to monitor and support replication of good practices is developed for making
replicated good practices functional3)

A package of MNH interventions under Upazila Health System (UHS) is developed.

1-3. Inputs as of terminal evaluation(1) Experts-Long-Term (108 MM) (Chief Advisor, Project Coordinator)-Short-Term (3.9 MM) (Advanced Facilitation Workshop, Community Mobilization, Health System
Management Introductory Training, TQM, etc.)(2) Trainees received in third country
TQM in Sri Lanka and Tanzania, Leadership in Governance in Kenya(3) Equipment
Equipment for neonatal care, etc. (total 4,050 thousand Bangladesh Taka (6 million Japanese Yen))
2. Evaluation TeamMembers

Bangladesh Side-

Mr. A.M Rejwenul Hoque (Senior Assistant Chief, MoHFW)-

Dr. Mosharraf Hossain (Assistant Director & Program Manager, CBHC, DGHS)-

Dr. A.S.M Nazmul Huq (Deputy Program Manager (TQM), Hospital Services
Management, DGHS )-Dr. M.A. Zulkawsar (Assistant Director & Deputy Program Manager, DGFP)

Japanese Side:-Mr. Hiroyuki Tomita (Senior Representative, JICA Bangladesh)-Dr. Hiroshi Sato (Chief Senior Researcher, Research Planning Department, Institute ofDeveloping Economies, Japan External Trade Organization)-

Dr. Hidechika Akashi (Senior Advisor, Bureau of International Health Cooperation,National Center for Global Health and Medicine)-Dr. Rintaro Mori (Director, Department of Health Policy/Department of Clinical
Epidemiology, National Center for Child Health and Development)-

Mr. Hirofumi Tsuruta (Process Consultant, Namidabashi Lab Co., Ltd)-Mr. Tatsuya Ashida (Advisor, Health Division 4, Human Development Department,JICA)

xii

Period of27 November 2015 – 14 December Type of Evaluation:

Evaluation study 2015

Terminal Evaluation

3. Results of Evaluation
3-1 Achievements of Outputs(1) Output 1
The indicators of the outpout 1 are: the number of cases/themes/areas of collaboration between
SMPP-2 and other MNH stakeholders, the number of good practices incorporated in national MNH
policies, strategies, guidelines, and manuals, and report of analysis on the activities in Narsingdi andChowgacha. The output 1 was achieved because all the three indicators were achieved as of the terminal
evaluation. SMPP-2 has contributed to incorporating some good practices into HPNSDP 2011-16 andother policies through the collaboration with different stakeholders. Major contribution of good practicesof SMPP-2 is a policy development related to CSG and 5S-CQI-TQM.
(2) Output 2
There are two indicators of the output 2: Mechanisms to monitor and support TQM, CSG and District
HLP are established, and training modules formulated, number of training, number of training
participants, number of Agencies SMPP provided technical assistance for implementation ofTQM/CSG/HLP. All the indicators were achieved as of the terminal evaluation. Therefore, the Output 2was achieved. SMPP-2 has contributed to developing and testing the mechanisms, tools and training
modules to monitor and support CSG activities (including “Core Team Strategy”), 5S-CQI-TQM inhospitals and Horizontal Learning Program (HLP). As of the terminal evaluation, the usefulness anduniqueness of these mechanisms and tools has been recognized among hospital staff, Core Teammembers and partners.
Facilitation

External Facilitator
Facilitation

District Core Team
Focal Person
DCS/MO-CS/Sr. HEO
Members-MO-CS-Sr./Jr. HEO-EPI Supervisor-DSI-Public Health Nurse-StatisticianSupervision

Focal Person
MO (CC)/ MO-DC
Members-HI (Selective)-AHI (Selective)-FPI (Selective)-SI-MT-EPI-Statistician

Monitoring & Support

CC, CG, CSG

Upazila Core Team

Monitoring & Support

Figure. Conceptual Framework of Core Team Strategyxiii

(3) Output 3Output 3 is mostly achieved as of the terminal evaluation. The output 3 indicators are: Defined UHS
concept, strategy and implementation mechanism is documented, the number of evaluation reports onMNH interventions, and the key process indicator for MNH such as ANC/PNC uptake, skilled birthattendance rate, and neonatal case fatality rate. The most of the targets related with the indicators havebeen achieved. The activities of the output 3 contributed to developing and operationalizing UHS
concept and a comprehensive MNH intervention package for UHS. Lessons learned in Satkhira district
can be disseminated to contribute to the discussion on implementation of effective MNH interventions
within the UHS.
3-2 Achievement of Project Purpose
The project purpose is achieved. The most indicators achieved its target as of the terminal evaluation
as shown in the below Table. The factors which enabled the CC/CG/CSG activities and 5S-CQI-TQM toexpand throughout the country include the incorporation of approaches introduced by SMPP-2 into
policies (output 1), the establishment of the mechanisms of monitoring and support to replicate the goodpractice (output 2) and a likely integration of a package of MNH interventions into the Upazila Health
System based on the experience in Satkhira (output 3).
Table. Achievements of Project Purpose
Project Purpose: The approaches to improve MNH service quality and utilization in line with Health,Population, and Nutrition Sector Development Program (HPNSDP) are expanded in Bangladesh
Indicators
Achievement
1. The proportion of all the TQM hospitals Achievedin Satkhira and Narsingdi achieved 70% All the 5S-CQI-TQM hospitals in Satkhira andor more on 5S at the MNH service areas Narsingdi achieved over 75% on 5S at MNH services(ANC/PNC corner, delivery room, areas, according to the project document.
operation theater and female ward)
2. The percentage of CSGs functional in Achieved
Satkhira is increased to 70% or more
The percentages of CSGs functional in Satkhira is 82%(540 CSGs out of total 657 CSGs), according to theproject document.
3. The proportion of women with Achieved
complication using EmOC services As of the terminal evaluation, the proportion of women
increases to 80% or more in Satkhira and with complication using EmOC in Satkhira district andKalaroa Upazila was 80.9%.
Kholaroa
4. Proportion of deliveries assisted by Achievedskilled personnel (C-SBA, SSN/FWV The proportion of deliveries assisted by skilledwith midwifery training, MBBS doctor) personnel in Satkhira district increases to 54.8% as of
the terminal evaluation (baseline: 37.4%) and in Kalaroa
increases to 50% or more.
it is increased to 68.1% (baseline: 45.6%)
5. The proportion of established CSGs Mostly Achieved
reaches to 100% in Bangladesh
Number of established CSG is 39,240 in Bangladesh(99.4% of the target: 40,149 as of June 2015), according
to the CBHC documents
In 2013, RCHCIB issued the government order that all
CCs should create CSGs as per the guideline.
6. TQM pilot hospitals expands to more Achieved
The number of the pilot hospitals is 106 in Bangladesh
than initial four hospitals (baseline: 3)
as of the terminal evaluationxiv

(Source) Project documents
3-3 Summary of Evaluation Results(1) Relevance
Improving maternal and neonatal health services targeted by SMPP-2 is one of agendas in the 3rd
HPNSDP 2011-16, which is a key strategic document as of ex-ant evaluation as well as terminal
evaluation. In addition, many challenges still remain in the neonatal and maternal health such as
ensuring skill attendance at birth, service expansion to hard-to-reach areas, assuring quality of care,according to the concept paper of the next health sector program. Moreover, Japan’s Country Assistance
Program for Bangladesh also prioritizes the contribution to the Bangladesh's efforts to realize MDGs insome areas including maternal and child health. Therefore, SMPP-2 is highly relevant with theBangladesh’s health strategy and needs, as well as Japan’s ODA policy. Therefore, its relevance is high.
(2) EffectivenessAs of the terminal evaluation, the CSG and 5S-CQI-TQM have been disseminated in Bangladesh.
The incorporation of approaches introduced by SMPP-2 into policies (output 1), increased feasibility tomaintain and expand the approaches through monitoring and support activities (output 2) anddevelopment of a package of MNH interventions under the Upazila Health System (output 3) weresuccessfully achieved. Thus, SMPP-2 largely achieved its purpose as of the terminal evaluation.
(3) Efficiency
Both of the project cost and project period will be within the plan. Therefore, efficiency of SMPP-2
is high. During the project period, activities could not be implemented as expected due to security
concern caused by political unrest and frequent strikes in Bangladesh but the project outputs were
produced as planned.
(4) Impact
It is possible to achieve overall goal in 3-5 years after the end of the project because MMR is now ontruck to reach the target by 2016, and NMR will be addressed more intensively in the next health sector
program. If the expanded approaches such as CSG and 5S-CQI-TQM are sustained with monitoring andsupport mechanisms, they will contribute the achievements of the target of MMR and NMR.
(5) Sustainability
As of terminal evaluation, some minor concerns have been observed in terms of the organizational,
technical and financial aspects of the implementing agency for the sustainability of project effects. While
responsible bodies were clearly defined for activities such as CSG, 5S-CQI-TQM, organizational
rearrangement might be conducted by MoHFW according to the concept paper for the next health sector
program. In addition, DGHS/DGFP will be able to operate national strategies and guidelines withvarious development partners because resource persons, agencies, documents, training modules willremain with DGHS/DGFP national level to Upazila levels. However, Core Team's activities developedxv

by SMPP-2 might not be taken over by anyone as this strategy has not been incorporated in the nationalpolicy. Moreover, there will be no major problem in terms of financial sustainability if the next healthsector program allocates the same level of amounts of the expenditure with 3rd HPNSDP. It was reported
during the terminal evaluation study that delayed supply of medicines to CC due to interrupterd fundreleasehas threatened the sustainability of activities of CC. There are other uncertainties for thesustainability related to the transition to a new health sector program. Therefore, sustainability of theproject effects is fair.
3-3 Implementation Process
The PDM for the Project was revised based on the recommendation during the mid-term review
process in order to facilitate the understanding of PDM as well as to refelct actual activities of theProject.
In SMPP-2 Project Supporting Committee, comprised of Japanese experts in key areas related withSMPP-2, was established at the beginning of SMPP-2 to provide technical consultation. The committee
has been beneficial to the Project in term ofproviding different viewpoints for the reframing theproject scope and activities.
In adition,,partnership with other development partners contributed to extend and scale-up theexperiencessuch as activities of CSG and 5s-Kaizen-TQM to the entire region.

Also,combination with Yen Loan project promoted to share the acitivities of the Project as well ascoordination with the activities of Japan

Overseas

Cooperation Volunteers facilitated the exchange ofuseful information on site level.
3-4 Conclusion
SMPP-2 is highly relevant with the Bangladesh’s health strategy and needs, as well as Japan’s ODA
policy, because SMPP-2 has focused on maternal and neonatal health prioritized in the 3rd HPNSDP as
well as Japan’s Country Assistance Program. Effectiveness and impact are high because SMPP-2 hasachieved its project purpose “the expansion of new approaches such as CSG and TQM, which influence
the maternal and neonatal indicators positively. Efficiency of SMPP-2 is high because both of the project
cost and project period will be within the plan. Sustainability of SMPP-2’ effects is fair because someminor concerns have been observed in terms of the organizational, technical and financial aspects of theimplementing agency.
In light of the above, this project is evaluated to be highly satisfactory.
3-5 Recommendations
1. On the ground that “Core Team Strategy” has functioned effectively to activate CG and CSG which
contribute to maximum utilization of CC as well as to strengthen the Upazila Health System. The
Evaluation Team recommends to Bangladesh side to expand the “Core Team Strategy” to otherdistricts to promote CC initiative.
2. CBHC should take necessary measure to address issues mentioned in 3-2-2 Output 2; allocation ofbudgets for “Core Team Strategy”, mobilizing resources for external facilitators such as NGOs,xviretaining experiences as institutional memory, and policy endorsement of “Core Team Strategy”.
3. The Evaluation Team confirmed that the mechanism of CG and CSG is applicable for promoting
further utilization of family welfare center (FWC). DGFP should consider creating CSG under theFWC management committee. In addition, data of FWC should be integrated with existing
community clinic data to capture the entire situation of the union.
4. SMPP-2 should document the implementation process, monitoring mechanism and experience ofDistrict HLP. In addition, the document should be shared and finalized with MoHFW, the Horizontal
Learning Center and its partners by June 2016. Documents are expected to be utilized in the newphase of HLP, and for mobilizing Union Parishards to contribute to health sector strengthening.
5. SMPP-2 should share the results and lessons learned of impact studies in Satkhira about women’s
health seeking behavior, social capital and women’s empowerment with MoHFW and sector partners
by the end of SMPP-2 to draw policy recommendation.
6. The Evaluation Team recommends TQM program of DGHS to support hospitals to strengtheninternal PDCA cycle in 5S-CQI-TQM and showcase the good practices of 5S-CQI-TQM foradvancement. The Team also recommends compiling positive changes of quality of hospital services
with data after the introduction of 5S-CQI-TQM.
7. The Evaluation Team recommends SMPP-2 to document 10-year experiences of SMPP, evolution ofNarsingdi model, and process and lessons learned of the scale-up of CSG and 5S-CQI-TQM by June
2016.
8. The Evaluation Team recommends MoHFW to assure responsible departments to take over theachievements of SMPP-2 even in the next health sector program.
3-6 Lessons Learned
1. It was proved that the Government can effectively mobilize the community participation with strongownership in Bangladesh. To formulate appropriate policies, external supporters such as SMPP-2
could advice or offer innovative ideas (e.g. CSG, Core Team strategy, 5S-CQI-TQM.) to thegovernment from the external objective point of view.
2. SMPP-2 focused on policy formulation and operationalization with the Government throughconstructing linkages between these two aspects, instead of increasing the geographical coverage. It
resulted in the formulation of appropriate policy and relevant approaches in the Bangladesh context,
which attracted development partners to get on board.
3. SMPP-2 conducted the third country training such as 5S-CQI-TQM in Sri Lanka and in Tanzania andexchange visits between Bangladesh and Afghanistan/Kenya. As for the Kenyan case, the visit
influenced the development of Community Health Strategy in Kenya. Those activities were able tocreate the chain reaction of knowledge and experiences by utilizing global network of JICA. It had
the positive impacts beyond the national border by utilizing the achievements of other JICA
technical cooperation projects.
4. PDM is a tool to share the direction of the Project as well as mobilize resources for the project
implementation at the design stage of the Project. Based on SMPP-2 experience, project
management tool should allow flexible and timely revision of activities when critical changes of thexvii

technical cooperation takes places and/or when an opportunities arise for achieving the overall goal,
which facilitate dynamic actions to address the overall goal.
5.

It was observed that one of the most critical factors of SMPP’s success was the placement of JICA
expert who committed oneself/herself to the project over 10 years and understands the country
context, the latest landscape of global health as well as JICA’s development values. Furthermore,
JICA internalized SMPP and built full trust with the expert that enabled SMPP to take prompt
decisions and actions, and allowed SMPP to be adaptable to meet changing needs of the country.

xviii