Summary of Final Evaluation
1.Outline of the ProjectCountry：Islamic Republic of Afghanistan

Project title：Reproductive Health Project

Issue/Sector：HealthCooperation Scheme：Technical Cooperation Project

Division in Charge：Health Systems

Total cost（As of April 2009）

Division, Human Development

Total：639,092,000 yen

Department
Period of Cooperation：Sept. 2004 – Sept.

Partner Country’s Implementing Organization：

2009

Ministry of Public Health (MoPH)
Kabul Provincial Health Directorate(KPHD)
Supporting Organization in Japan：International Medical
Center of Japan
Related Cooperation：“Dasht-e Barchi Hospital Training Center” by remnant ofEmergency Grant

1-1Background of the ProjectAfghanistan is facing the world’s worst health conditions, e.g., maternal mortality ratio of 1,600/100,000
live births. This is due to socio-cultural and physical factors that hinder women’s access to reproductive
health services, the shortage (both in number and quality) of female health service providers and the weak
health systems.
Afghanistan’s Ministry of Public Health (MoPH) launched Reproductive Health Directorate (RHD) in
2004, aiming to ameliorate the maternal mortality ratio. In order to achieve the policy objectives (i.e. better
access to quality family planning services, better access to emergency obstetric care, and increase indeliveries with skilled birth attendants), RHD makes efforts such as allocating a reproductive health officer
to each province, training reproductive health service providers at Malalai Maternal Hospital, etc.
With this background, the Government of Afghanistan requested a technical assistance to strengthen
reproductive health services. This project has started in September 2004, with its purpose: “The capacity ofRHOs (Reproductive Health Officers) and RH service providers to provide necessary services based onBPHS (Basic Package of Health Services) and EPHS (Essential Package of Hospital Services) is improved.”
Before the project completion in September 2009, JICA dispatched the terminal evaluation team andconducted a joint evaluation with the Afghan side.
1-2

Project Overview(1)

Overall Goal：The coverage of quality maternal and newborn health services is expanded.

(2)

Project Purpose：The capacity of RHOs (Reproductive Health Officers) and RH service providers toprovide necessary services based on BPHS (Basic Package of Health Services) and EPHS (EssentialPackage of Hospital Services) is improved.
(3)

Output：Output 1
National Reproductive Health Program is established and functioning.
Output 2
In-service training mechanism is established and maintained in an appropriate manner.

xi

Output 3
RH system is functioning in Urban Kabul.
1-3Inputs（As of April 2009）＜Japanese side＞
Long-term expert
Short-term expert

813

Acceptance of trainees (in Japan)

21

Afghan C/P participants in Training in Third Countries (Cambodia, Indonesia, Pakistan)
Equipment supplied
Local Cost

18Vehicles, photocopier, training materials, drug supplies, etc.

97,692,000 (Japanese yen)＜Afghan side＞
Counterparts (C/P)
Local cost

38 (including those who were replaced during the Project)

Personnel cost for C/P, provision of office for Japanese experts, cost for materials andconsumables
2.Evaluation TeamMembers

1. Mr. Kozo Watanabe (Team Leader) Director, Health Systems Division, Health Systems

ofand Reproductive Health Group, Human Development Department, JICA

Evaluation

2.

Ms. Keiko Osaki (Maternal and Child Health)

Team

3.

Ms. Ritsuko Yamagata (Project Planning)

Senior Advisor, JICA

Staff, Health Systems Division, HumanDevelopment Department, JICA
4.

Ms. Erika Tanaka (Evaluation and Analysis)

Researcher, Social Development

Department, Global Link Management
Period of17 April 2009 – 2 May 2009

Type of Evaluation：Terminal evaluation

Evaluation
3.Results of Evaluation
3-1

Summary of Evaluation Results(1)

Relevance
Relevance is high in terms of Afghan development policy, needs in Afghanistan, Japanese official

development assistance (ODA) policy to Afghanistan, and Project design.
The Project Purpose and Overall goal are consistent with Afghan development policy.

In Afghanistan

National Development Strategy (ANDS: 2008-2013), reproductive and child health program is given highpriority.

Improvement of reproductive health is highly needed in Afghanistan. Maternal mortality rateis extremely high, 1600/100,000 live births (Ministry of Public Health, 2004).
The Project is consistent with Japanese ODA policy as well. In the Japanese ODA policy toAfghanistan, improvement of health is included in one of the four priority areas.
The approach that the Project adopt to achieve Project Purpose is appropriate. Given the fact that theRHD was established only two years before the start of the Project, establishment of mechanism of RH
program and human resource development is highly adequate. The Project puts importance on fostering
ownership and capacity of C/Ps both as individual and as institution. Another distinctive approach of thexii

Project is to achieve its goal through strengthening the RH systems in the public sector. As the assistanceto economic and social reconstruction of Afghanistan is a highly important issue in international
community, the Project gives consideration to collaboration with other aid agencies, which is desirable incurrent Afghan situation.
(2)

EffectivenessProject Purpose is being achieved and effectiveness is high.
Outputs are being gradually achieved. National RH program has been developed and the capacity ofRHD (RH Directorate) is improved.

Training mechanism is established at Malalai Maternity Hospital

Training Center and is being established at Dasht-e Barchi District Hospital Training Center as well.
Some tasks to be achieved are; developed training manuals are still under procedure of official approval;
training database is not developed although reporting system is established.

Objectively Verifiable

Indicators of Output 1 and Output 2 in PDM are generally achieved.

As to Output 3, RH system inKabul City has been remarkably improved since the start of the Project.

Not all the PDM Indicators arefully achieved due to complicated setting of Indicators. To make RH system in Kabul City fully function,
further capacity development of KPHD and establishment of referral system is necessary.
Project Purpose is being achieved.

The administrative capacity of RHOs is greatly enhanced ascompared to the beginning of the Project and RH program is appropriately implemented.
of RH service providers is also strengthened and the RH services are also improved.

The capacity

To further improvethe capacity of RHOs and RH service providers it is necessary to strengthen supervision and training
system.
Three Outputs are logically related to Project Purpose and contributed to achieve Project Purpose.
Output 1 and Output 2, establishment of RH program and in-service training mechanism are essential toachieve Project Purpose. Output 3 is to strengthen capacity of RHD, RHO, and service providers tomake RH system function in Kabul City.

In Kabul City, BPHS is not contracted-out and it is expectedthat establishing RH system there will serve as a setting to practically implement the outcome of trainingof RHD, RHO, and service providers. Thus Output 3 is also related to Project Purpose and contributed toachieve Project Purpose.
(3)

Efficiency
Efficiency is generally high.

Most part of Inputs was implemented as planned and contributed toproduce sufficient Outputs.
Most Japanese experts had adequate knowledge and experiences and helped capacity development ofAfghan C/Ps.
Training in Japan was effective to strengthen capacity of C/Ps.

Participants in Training in Japanacquired skills and method of RH administration and utilize the fruit of the Training in their daily duties.
Technical Exchange with projects in other countries was proved to be effective.

Some C/Ps participatedin Third Country Training provided by JICA in countries with situation similar to that of Afghanistan, e.g.
Indonesia as Islamic country, and Cambodia in a reconstruction process. These training courses also
helped to enhance knowledge and administrative skills of RHO as they can share similar experiences.
Equipment and materials provided by Japanese side are adequate in terms of specifications andquantity. Malalai Maternity Hospital Training Center provided by the Project serves well as a basis oftraining activities.

Dasht-e Barchi District Hospital Training Center opened only in May 2008 and is notxiii

fully utilized yet at the time of terminal evaluation. Recently the organogram in regard to Dasht-e Barchi
District Hospital Training Center was approved and the Training Center is placed under the Dasht-e
Barchi District Hospital and KPHD. However, the Training Center is not positioned as regional training
center.
All the national staff of the Project has high capacity and strong commitment and made essential
contribution to the Project implementation.
The Afghan side allocated C/Ps with sufficient capacity and ownership to implement the Project. The
budget for operational cost by the Afghan side is limited.

It was difficult for MoPH to provide expenses

of maintenance cost for equipment and materials, and the cost for supervision and training was basicallyborn by the Japanese side.
As a part of partnership, the Project successfully planned provision of drugs in collaboration with otherdonors.

This collaborative support made smooth progress in the beginning. However, the share ofpartner donors was not implemented as planned.
Inputs were generally well utilized and contributed to implement planned activities and to produce
Outputs.
The Project implementation process was affected by security to some extent.
As to Important Assumptions, replacement of C/Ps occurred quite frequently in the first part of theProject, causing inconvenience in smooth Project implementation sometimes.

After the Mid-Term

Review, the C/Ps remain stable in their positions.
(4)

ImpactOverall Goal is expected to be achieved if the current benefit of the Project is sustained and activities

are expanded after the Project period. The foundation of RH administration is being established throughcapacity development of RHD and RHO, and the capacity of service providers is being improved throughestablishment of training mechanism.

If the capacity of RHO and service providers is continuously

improved, quality maternal and newborn health services are expected to be expanded.

It should benoted, however, that achievement of Objectively Verifiable Indicators of Overall Goal, i.e., the number ofpeople benefiting from the RH services, depends on conditions outside of framework of the Project.

Inorder that people enjoy benefit of the RH services, some important factors other than capacity of RHOand service providers should be fulfilled: access to health facilities and security, to name a few.
It is expected that this Project will make great contribution to human security in Afghanistan as
reproductive health is an important issue in human security.
Unexpected positive impact is observed.
health in Kabul City as Output 3.

The Project aims to establish referral system in reproductive

During implementation of activities, those involved in the Project

realized that it is necessary to develop policy and strategy to build more comprehensive health systems inurban Kabul.
One negative impact is pointed out.

The Project made suggestion to convert policlinics to DH(District Hospital) and CHC+ (Comprehensive Health Center with 24-hour service) in Kabul City.
of nine policlinics, two were converted to DH and one to CHC+.
CHC+, the number of staff was reduced.

Out

When the policlinic was converted toAccording to interview, it is estimated that 30 to 40

employees, including workers, lost their job.

xiv

(5)

Sustainability
Sustainability is generally high in terms of policy, technical, and institutional aspects, but financial

sustainability is not expected without external support.
The policy to put importance on reproductive health is expected to continue. In ANDS, which describes
policy until 2013, reproductive health is a prioritized area.
Technical sustainability is quite high. Those who were trained by the Project, including RHOs, training
coordinators, trainers, and service providers have acquired knowledge and skills to implement their duties
on their own. Most of them have strong commitment to execute their responsibilities as well.
enhance sustainability, continuous capacity development is required.

To

It is also important to conduct

training when staff is replaced.
Institutional sustainability is also high. RHD was strengthened as an organization and RH program
was developed.

Training Centers were constructed by Japanese support at Malalai District Hospital and

Dasht-e Barchi District Hospital. Training Department was established at Malalai Maternity Hospital andtraining center regulations were developed. Activities introduced by the Project are expected to besustained.
Financial sustainability is the concern. MoPH is not provided sufficient budget to implement its
responsibilities.

The majority of the budget is allocated to personnel cost and running cost is limited.

There is not an established mechanism to allocate budget to training and supervision. Therefore, theMalalai Hospital Training Center is not provided with enough budget to conduct training courses andsupervision. KPHD is not provided with sufficient budget either. To conduct training and supervision
introduced by the Project, it is necessary to secure the budget, especially in terms of running cost.
3-2Factors that promoted realization of effects

One of contributing factors to achieve Project Purpose is strong commitment of both Japanese experts andAfghan C/Ps. Another contributing factor is the situation of RHD at the start of the Project.

The Project

was launched soon after MoPH set up RHD and appointed Deputy Minister in charge of reproductive health.
Almost simultaneous start of the Project promoted smooth implementation of the Project.
3-3

Factors that impeded realization of effects

One concern to fully achieve Project Purpose is that the trained service providers sometimes leave theirposition.
3-4

Conclusion

The Project has been implemented appropriately in general and the Project Purpose is being achieved inspite of difficult situations in Afghanistan. Project implementation process was promoted by strong
commitment of C/P organizations led by Deputy Minister. As a result, the capacity of RHOs and RH
service providers has been significantly increased and it is expected that they will be able to continue theirtasks on their own to some extent.
The three Outputs are also being achieved. National Reproductive Health Program is established although
it is not yet in documentation.

National Reproductive Health Program is established but further capacity

development of RHD is required.

In-service training mechanism is being established but it is stillnecessary to strengthen the training management mechanism and the capacity of training coordinators,trainers, and service providers.

RH system in Kabul City has been clearly improved as well.

xv

However, asthe RH system covers a variety of components, such as referral system and drug distribution, it is stillnecessary to further enhance the capacity of KPHD and strengthen RH system in Kabul City.
3-5

Recommendations

Based on the results of evaluation, recommendations on actions to be taken during the remaining periodof the Project are as follows.
- MoPH will compile National Reproductive Health Program in documentation.
- MoPH will complete the procedure for approval of developed training manuals and in-service training
guideline.
- MoPH will compile and analyze HMIS data collected after the first quarter of 2008, and make decisions
based on the results.

MoPH will strengthen the capacity of RHD staff in data analysis.

- MoPH will translate Annual Report 2008 into English.
- MoPH will have Dasht-e Barchi District Hospital Training Center function as regional training center andimprove the capacity of Dasht-e Barchi District Hospital as well.
- MoPH will compile information on NGOs in collaboration with Grant and Contract Management Unit tofacilitate joint supervision with NGOs.
- MoPH will make continuous effort to estimate, secure, and distribute budget to cover running cost for
management of training and supervision.
During Evaluation, the following suggestion was raised. It may not be a matter to be considered during theremaining Project period, but it is desirable to give consideration.
- Although Dasht-e Barchi District Hospital meets the requirement of number of staff specified by BPHS,
the staff is not sufficient to adequately conduct increasing deliveries.
appropriate in urban situation.

The criteria of BPHS may not be

It is recommended to review BPHS and revise BPHS regulations to beadopted in urban area.
3-6

Lessons Learned

Through the Terminal Evaluation of the Project, following lessons learned were extracted.
- When a certain input is implemented in collaboration with other donor agency, it is important to assure that
expected input by partner agency will be surely implemented.
- Personnel replacement occurs frequently in a country in the process of reconstruction like Afghanistan.
In this case it is effective to establish the results of activities in documentation to sustain the activities andachievements of the project.

xvi