Evaluation Summary
1. Outline of the ProjectCountry:
Indonesia

The

Republic

Issue/Sector: Healthcare
medical treatmentof Project Title: The Project to Enhance Surveillance System for
Avian Influenza
and Cooperation Scheme: Technical Cooperation Project

Division in charge: Health Total Cost： 197million JPY
Division 2, Health Group 3,
Human Development Department(R/D):
Partner Country’s Implementing Organization: immunization
20/Oct/2008-19/Oct/20 and Quarantine, the Ministry of Hea lth, South S ulawesi
Provincial Health Office and District Health Offices
Period of 11
Cooperat
Supporting Organization in Japan: National Center for Global
ion
Health and Medicine, National Institute of Infectious Diseases
Other Related Projects: N/A
1-1 Background of the Project
Since the first human case of Avian Influenza H5N1 (AI) was detected in July 2005 in the Republicof Indonesia (hereinafter referred to as “Indonesia”), one-hundred and seventy-seven (177) AI
human cases has been reported continuously and resulting in 146cases of the world’s highest
number of human fatalities as of May 13, 20111. The situation raises deep concern about emerging
novel influenza virus attributed to the mutation of AI virus in the process of spread of AI humaninfection, and pandemic not only in Indonesia but also all over the world. Thus, it is required for not
only Indonesia but also international society to minimize the social influence of novel influenza
through the early detection and rapid response.
Especially in the South Sulawesi Province, greater emphasis was placed upon the reinforcement ofsurveillance system due to the following reasons:
1.

Less international assistances to the South Sulawesi Province, which is classified as hi-risk
Province in comparison with others by the Government of Indonesia due to high densities ofpoultry and population,

2.

Less reported suspected AI human cases in comparison with high number of avian cases,
which implies insufficient surveillance capability for the detection of AI human cases, and3.

South Sulawesi Province has been a key junction as a gateway to eastern Indonesia.

For these reasons, the Government of Indonesia submitted an application for technical cooperation
to the Government of Japan with regard to the strengthening of surveillance system for AI humancases at South Sulawesi as a target area of a project.
Based on the request of technical assistance by the government of Indonesia to the Japanese
government, Japan International Cooperation Agency (JICA) launched the technical cooperation
project entitled “the Project to en hance surveillance system for Avian Influenza” (hereinafterreferred to as ‘the Pro ject’) with the Project Purpose of stren gthening of the surveillance of AI
human cases through the comprehensive surveillance system of local priority communicablediseases in South Sulawesi Province. At the time of the Mid-term review survey, it was confirmedthat certain results were about to be generated in quality AI surveillance system thanks to strong
commitment from both Indonesian and Japanese sides.
1-2 Project Overview(1) Super Goal
Surveillance system of priority communicable diseases is strengthened in Indonesia.
(2) Overall Goal
1

Cumulative Number of Confirmed Human Cases of Avian Influenza A/(H5N1) Reported to WHOSurveillance of AI human cases is stre ngthened, through the comprehensive surveillance system of
local priority communicable diseases, in other provinces in Indonesia.
(3) Project Purpose
Surveillance of AI human cases is stre ngthened, through the comprehensive surveillance system of
local priority communicable diseases, in South Sulawesi Province.
(4) Outputs
1) Detection, reporting and recording of suspected AI human cases are strengthened, through thecomprehensive surveillance system, at each le vel from public and privat e health facilities to
MOH.
2) Early reporting system of suspected AI human cases, as well as local priority communicablediseases, is developed in the community in some health centers.
3) Capacity of outbreak investigation of provincial and district rapid response teams is enhanced.
4) Sampling and shipment of specimens from suspected AI human cases and their close contacts
and management of their test results are streamlined.
5) Collaboration of surveillance and response among the central level, the province and districts
is enhanced.
(5) Input（as of the evaluation）
Japanese Side
Dispatch of Experts：56.22 M/M
Provided Equipment：PC with modem system for EWARS, Server for SMS Gateway
Local Cost：Approx. Rp. 5, 890,000,000
Training in Japan： 22 personnel
Indonesian SideCounterparts： 22 personnel
Land and Facilities：Project office spaces in the Ministry of health (MOH) a nd the South
Sulawesi Provincial Health Office (PHO)
Operational Cost： Preparation of Training and Seminar, etc.
2. Terminal Evaluation Team
Members

LeaderDr. Mitsuhiro
USHIO

Executive Technical Advisor to the Director General,
Human Development Department, JICA

Dr. Kaname
KANAI

Epidemiology

Director-General, Department of International
Cooperation, National Center for Global Health andMedicine (NCGM)

Dr. Chiaki
MIYOSHISurveillance

Director, 2nd Expert Service Division, Department ofInternational Cooperation, NCGM

Mr. Seiji KATO Cooperation
Planning

Advisor, Health Division 3, Health Group 2, Human
Development Department, JICA

Evaluation andAnalysis

Consulting Division, Japan Development Service
Co., Ltd.

Dr. Yoichi
INOUE

Period of May 17, 2011 – July 1, 2011
Evaluation

Study Type: Terminal Evaluation

3. Summary of Evaluation Results
3-1 Achievements（1） Output 1
The Objectively Verifiable Indicators (OVIs) for Output 1 are ge nerally achieved at the time of
the Terminal Evaluation.
Based on these findings from the base line survey, the Technical Protocol was prepared andmethods for detecting, reporting, recording, assessing surveillance conditions and reporting
surveillance data were standardized. The Technical Protocol has been put into operation via theDistrict Surveillance Officers (DSO) training, etc. and it was officially recognized as a provincial
governor’s decree in May 2010. The Technical Protocol has been made more practical throughconducting revisions based on actual application and the res ults of table-t op exercise. Since
improvement of test results following training has been confirmed and operations according to theTechnical Protocol are taking root, it can be confirmed that the knowledge and skill ofsurveillance human resources have been enhanced. Regarding detection, reporting, recording andfeedback of information too, human resources have been developed via the trainings andhardware systems utilizing SMS gateways and the Internet have been bolstered.
（2） Output 2
It is anticipated that the OVIs for Output 2 will be achieved in general by the end of the project
period.
In the trainings for the Rapid Response Team (RRT), DSO, Early Warning Alerting and Response
System (EWARS) and Health Centers, the trainees learned how to raise public awareness in cases
where suspected AI human infection occurs in the community. Moreover, the Technical Protocol
specifies the roles of community health volunteers (Kader) regarding surveillance. In theCommunity-based Surveillance (CBS) training, guidance was carried out on early reporting of
infectious outbreaks using the training module based on the Technical Protocol. Based on good
cooperative relations with the PHO and health centers, since community health volunteers arestarting to give reports on AI and other important infections (including zero reporting) inaccordance with a m odule for Kader , a great c ontribution to the surveillance system can be
anticipated in the future.
（3） Output 3
It is anticipated that Output 3 will be achieved in general by the end of the project period, though
an OVI hasn’t been fulfilled.
The Technical Protocol was prepared and RRT activities were standardized based on the findingsof the baseline survey. Also, RRT training using the training module prepared based on theTechnical Protocol was implemented, and the improvement in the knowledge and skills of humanresources in charge of ra pid response to AI and o ther infectious outbreaks can be ex plained
through the improvement between pre-training and post-training test results and the fact that rapid
response has been appropriately implemented according to the Technical Protocol.
Concerning the periodic sharing of information with the livestock sector and the health promotion
sector, so far there has not been sufficient preparation of documents, and this point is likely to be
stressed in the RRT refresher training scheduled for the second quarter in fiscal 2011. However,there is strong awareness that join t response by hea lth centers and the livestock se ctor regarding
AI is needed, and information sharing is routinely conducted on the central level, province level,
district level and lower.
（4） Output 4
Considering the target scope (target group) and term of the Project, it is difficult to achieve this
Output 4 during the project period, however, it may be said t hat the environment for en hancing
sampling, shipment of samples and management of test results has been established.
Based on the baseline survey results, the method for sampling, shipment of sa mples and test
results management was standardized in the Technical Protocol. Using the training module thatwas prepared based on the Technical Protocol, training including practical sampling, shipment ofsamples and control of test results was implemented for the p ersonnel who actually handle
samples. In th e table-top training too, procedures have been reconfirmed. When a suspected AI
human case was reported on September 2010, samples were obtained by the RRT, demonstratingthat RRT sampling was possible.
Since improper handling of samples was occasionally found as of the Final Evaluation, it is hardto say that this Output has been fully realized. The officers responsible for actually handlingsamples are laboratory technicians, so there is a limit to how far this issue can be tackled in termsof the project scale including the target group. However, the BBLK is applying the checklist on
handling of samples that was prepared by the Project, and it has promised cooperation in future
training, etc. Since the Technical Protocol has standardized the handling of samples and a system
has been constructed for monitoring appropriateness in the sampling and its shipment, a certain
degree of improvement can be anticipated from now on.
（5） Output 5
It is anticipated that the OVIs for Output 5 will be achieved in general by the end of the project
period.
The baseline survey revealed problems in collaboration between the central level, South Sulawesi
Province and the districts, and the Technical Protocol standardized the method of c ollaboration
between each level regarding surveillance, epidemiological response, information transmission
and supervision and guidance, etc. In the RRT training, DSO training and table-top exercises too,guidance is conducted on the strengthening of collaboration according to the Technical Protocol.
Moreover, introduction of EWARS and SMS gateway has made it possible to synchronously
share information; in particular, concerning outbreaks that need to be res ponded to jointly,
significant improvement has been seen in the timing of collaboration between the district andprovincial RRT.
（6） Project Purpose
Although the numerical targets have no t been attained under OVIs for reporting timeliness andfeedback, sure improvement has been continued and, considering that appropriate outbreak
investigation is implemented, surveillance of suspected AI human cases including important local
infections has been strengthened and the project targets have generally been achieved.
However, in line with the introduction of EWARS and SMS Gate way following the Mid-term
Review, major improvement occurred in the timeliness of weekly reporting and immediate sharing
of information, and if the present monitoring, supervision and guidance are maintained, it isanticipated that these OVIs can be comfortably achieved. Moreover, at the time of the suspected
AI human infection and poultry AI outbreak during the project period, information wascommunicated pertinently within the health sector and with the livestock and other sectors, andrapid response by the RRTs was appropriately implemented according to the Technical Protocol.
3-2 Summary of Evaluation Results（1） Relevance
The MOH emphasize on the importance of AI control in the National Long-term Health
Development Plan 2 005-2025 and its concrete measures of the Health Sector Strategic Pl an
2010-2014. Since it is confirmed that the priority of s urveillance system reinforcement for l ocal
priority communicable diseases including AI is being maintained at the time of the Final
Evaluation, the Project Purpose aiming to str engthen the surveillance for local priority
communicable diseases via AI surveillance is highly consistent with Indonesian health policies atthis time. Moreover, improvement of act ual operation of surveillance based on the National AI
Guidelines through the development and installation of “Technical Protocol”, consequent capacity
building to health personnel in charge of surveillance, and reinforcement of disease surveillance areconsistent with the respective needs of the MOH, PHO and DHOs.
The importance of preparedness and international collaboration against emerging and reemerging
infectious diseases is clearly cited in “the Japan’s Global Health Policy 2011-2015”as well as theJapanese ODA policy. Thus, the Project, aiming to assist AI surveillance in Indonesia with highincidence of AI human cases, also meets the Japan’s aid policies.
（2） Effectiveness
Although Indonesia does have the national guidelines on s urveillance, its contents do notadequately reflect actual operations on the ground. In order to remedy this situation, the TechnicalProtocol has been prepared and actual operation has been reinforced in accordance with thenational guidelines and the International Health Regulations (IHR) and based on ample discussions
with stakeholders. The various training programs of the Project have been implemented according
to the Technical Protocol and human resources have been developed in order to ap propriately
conduct surveillance. Moreover, it has been confirmed that surveillance on the ground is conductedaccording to the Technical Protocol and that appropriate responses based on the Protocol have beentaken in actual cases of suspected AI human infection outbreaks occurring during the Project
period.
However, concerning the degree of achievement of OVIs, the t argets that were indicated in the
Mid-term Review have not been attained. These targets were set ac cording to national criteria andare deemed to be appropriate as eventual marks, however, it cannot be denied that they weredifficult to attain by the end of the Pro ject. Having said that, in line with the introduction of
EWARS and SMS Gateway, major improvement occurred in the timeliness and completeness ofweekly reporting and the targets have generally been achieved since the start of 2011. If the present
monitoring, supervision and guidance are maintained, it is anticipated that these OVIs can be
comfortably achieved, and from the viewpoint of “has the surveillance system been strengthened,”
it is deemed that the Project Purpose are generally being attained. From these reasons, it isconsidered the effectiveness of the Project is generally high.
（3） Efficiency
Several factors, such as delay in budget implementation from Indonesian side attributed to thedifference in fiscal year system between Indonesia and Japan, dispatch of long-term JICA expert,
delay in p ublication of the T echnical Protocol, and insufficient communication amongst
stakeholders, affected progress of activities at the initial phase of the Project, though the project
activities have been implemented in accordance with Plan of Operation (PO).
Though the Project efficiency was i mpeded by these delays from the viewpoint of adv ancing the
project activities according to schedule within the project period specified in the PO, project period
itself was effectively utilized in the Project. Moreover, since revision of the Technical Protocol wascompleted as of the t ime of the Fin al Evaluation and distributed to th e MOH and rele vantorganizations in the South Sulawesi Province, and remaining activities such as RRT refresher
training will be conducted as scheduled. Therefore, it is considered that aforementioned issue arosein the project period did not cause severe influences on ultimate achievement of t he Project
Purpose.
Summing up, in terms of the progress of the Project activities, efficiency was generally maintained.
（4） Impact
The surveillance system including early detection and response, reporting system and other related
procedures were standardized and streamlined, and thus, it is considered that the necessary
foundation has been formed for surveillance system for lo cal priority communicable diseases
including AI in t he South Sulawesi Province. The Technical Protocol, revised with sufficientreflection with actual situation in South Sulawesi, is being distributed to relevant institutions andfacilities. Accordingly, further improvement in surveillance system can be anticipated.
The MOH appreciates the model of surveillance system established in the South Sulawesi
Province, and expressed their vision for deployment of the model to nationwide in a stepwisemanner, beginning by the Central Sulawesi Province. In fact, the MOH held an advocacy meetingon April 2001 at the Central Sulawesi, and is planning to install EWARS around June to July 2011.
Moreover, the MOH is considering utilization of South Sulawesi PHO personnel nurtured by the
Project for lecturers at the time of at the t ime of actual implementation. Thus, it is obvious that the
Project exerts a be neficial impact on Indones ian surveillance system from the perspe ctive of
probability of achievement for the Overall Goal of the Project.

（5） Sustainability
The importance of com prehensive AI c ontrol is c learly cited in t he National Long-term Health
Development Plan 20 05-2025 and the Health Sector Strategic Plan 2010-2014, which prescribes
the concrete measures and policies for achieving this, and the MOH expressed verbally that theimportance and priority would be maintained for many years to come. Indeed, the MOH made apractical effort to step-by-step deployment of the surveillance model for suspected AI human cases
developed in the South Sulawesi Province, starting by the Central Sulawesi Province as a mostprobable candidate. Based on these facts observed, it is considered that political sustainability canbe anticipated in some degree.
The MOH m ade a practical effort to step-by-step deployment of the s urveillance model forsuspected AI human cases deve loped in the Sou th Sulawesi Province, starting by the Ce ntral
Province. However, at the time of practic al introduction, it is considered that the MOH a nd
provincial government should reinforce the commitment of s ecuring human and financial
resources. Having said that the Project nurtured personnel in PHO as supervisors and/or lecturers, it
is desirable to estimate the human resources and contents of activities that can be used as areference material.
As was stated above, a certain degree of self-sustainability from technical perspective was secured
since the practical operation of AI surveillance system based on the Technical Protocol has
established and human resources in charge od surveillance are favorably nurtured. However, it issuggested that further t echnical assistance will be needed for ex pansion and deployment of thesurveillance model to other provinces. Moreover, from the financial and technical aspects, further
commitment from provincial and district governments will be required for maintenance of current
activities in the South Sulawesi Province and for the deployment to other pr ovinces. Greater
sustainability could be secured if continuous technical assistances including some financial support
were obtained by any means in addition to continuous effort from Indonesian side.
3-3 Factors that promoted the attainment of the Project（1） Concerning the project design
Good cooperative relations with other sectors were maintained through the project period.
Activities have been conducted in collaboration with the livestock sector and laboratory sector via
the RRT, and the information sharing setup improved greatly following the start of the Pro ject.
Moreover, since cooperation was obtained from them by dispatching lecturers in the projecttraining courses and in the Technical Protocol revision work, this can be viewed as a contributing
factor to the achievement of Outputs.
（2） Concerning the implementation process of the Project
EWARS, RRT and DSO were respectively introduced to Indonesia by the WHO, USCDC and
USAID, however, these resources had not been adequately introduced on the ground level. In the
Project, these existing systems were utilize d and modified according to conditions in South
Sulawesi in an effort to support concrete introduction. Also, the modules used in the training wereprepared based on those created by other development partners. Accordingly, althoughcollaboration has been indirect, since effective synergy has been obtained with other resources, theresulting impact has in a sense been greater than that arising from ordinary collaboration.
3-4 Factors that impeded the attainment of the Project（1） Concerning the project design
No major obstacles have been observed as far as the project design is concerned.
（2） Concerning the implementation process of the Project
Financial autonomy is im proving and budget steps for the pr oject activities have generally been
appropriately conducted. However, as a result of measures to promote decentralization, some
district health offices have only managed to secure enough budgets for t he minimum necessary
outbreak response activities and it is difficult for them to conduct regular feedback, supervision andguidance in the health centers.
3-5 Conclusions
The OVIs for the Projec t Purpose of final year are under the process of measurement at the timeof Final Evaluation Survey. Nevertheless, it c ould be estimated to be able t o achieve the OVIs
because of in recent 3 weeks those indicators have been more than 80%.
The Technical Protocol, which is expected as essential output of the project activity, has alreadybeen elaborated and revise d based on t he results of the ac tual practice in the project sites.
Furthermore, the Technical Protocol was authorized officially as a provincial governor’s decree sothat that is major outcome of the Project and it is highly evaluated.
Moreover, the MOH i mplemented advocacy at the Central Sulawesi Province on the TechnicalProtocol elaborated by the Project and several provinces expressed their interests to get it.
Through the mentioned above, we can see signs to step into the Overall Goal “Surveillance of AI
human cases is strengthene d, through the comprehensive surveillance system of local priority
communicable diseases, in other provinces in Indonesia” in spite of short time (3 years) of project
activities. It could be said that the Project has achieved brilliant success.
3-6 Recommendations< The MOH>
1.

The MOH should continue to exert a strong commitment to local priority communicablediseases, including AI.

2.

It is strongly suggested that the MOH should exert efforts to introduce and/or establish thesurveillance system for the local priority communicable diseases at provinces where hasn’t
been yet.

3.

The MOH should provide opportunities to enhance sharing of experiences and information
amongst PHO.

< The PHO>
1.

The PHO should continue to exert efforts to sec ure budget necessary for operat ion of t hesurveillance system, and to develop human resources further.

2.

The PHO should provide opportunities to enhance sharing of experiences and information
amongst DHOs regularly.

< DHOs>1.

DHOs should continue to exert efforts to secure budget necessary for operation of thesurveillance system, and to develop human resources further.

2.

DHOs should provide opportunities to enhance sharing of experiences and information
amongst Health Centers regularly.

< The Project / JICA Experts>
1.

The Project should clarify the contents in the Technical Protocol where t o be subje ct tomodification in accordance with regional circumstances, when Indonesian side introduce
the South Sulawesi surveillance model established by the Project to other provinces.

2.

The Project should compile project experiences in the South Sulawesi Province, such as
operational process by component and challenges, so that the experiences can be utilized as
references.

< Common Element>
1.

Counterparts at all levels should exert efforts to maintain and enhance the relationship withrelated partners such as livestock sector and laboratory sector.

3-7 Lessons Learned
1.

This Project almost achieved its purpose to develop the AI surveillance model development
in Indonesia despite that it was in a sh ort term of three y ears with several administrative
levels concerned. The reason was to have obtained a strong commitment of th e both ofcentral and provincial level under the project design that aimed to achieve the national
strategy by making existing surveillance systems as a package adjusting to a local situation.

2.

It might be necessary to design the Project in terms of the dissemination to other provinces
considering the short cooperation period of 3 years.

3.

Local staffs’ technical basis and als o ownership have been developed by the following
project activities; first elaborating "Technical Protocol" adapted for the current situation of
the region, and then revising it through the job and table-top exercise. Consequently, theirself-sustainability has been enhanced, which is proved by acquisition of th e local budget
securing and also by the strong motivation for expanding the project activities to anotherprovinces.

4.

Cost sharing, which the Project had introduced intensively, brought the change of
consciousness on cost and the acquisition of budget at the provincial and district levels,
eventually leading to the an important change regarding to the self-sustainability after thetermination of the Project.

5.

The Project Purpose is to s trengthen the surveillance system of AI, and it is esse ntial todevelop capacity of the concerned staff to achieve its purpose. Furthermore, it is regarded
that strengthening close relation among the implementing organizations through the project
activities such as trainings and supervision etc. enabled to establish the system finally.

6.

At the time when official agreement was issued for the commencement of the Project after
the detailed planning survey, budget for the Pro ject by the In donesian side could not beallocated for the first year of project period, since deadline for budget application had
passed. Accordingly, the activities of the Project planned for the first year was interrupted.
Therefore, Japan side should give attention to budget allocation process in partner countries
at the planning stage of projects.

3-8 State of the follow-up#####