Summary
I. Outline of the ProjectCountry: Afghanistan

Project Title: Tuberculosis Control Project

Issue/Sector: Healthcare

Cooperation Scheme: Technical cooperation

Division in Charge: Health Systems Division, Total
Human Development Department(R/D): Sep 04 – Sep 09 (5 years)
Period

of

Cost

(at

the

time

of

evaluation):

601million yen
Partner Country’s Implementing Organization: National
Tuberculosis Control Program (NTP), Ministry of Public
HealthCooperation

Supporting Organization in Japan: Research Institute ofTuberculosis/Japan Anti-Tuberculosis Association
RelatedCooperation

Project:
1. Background of the Project
After the 23-year civil war that started with the invasion of the former Soviet Union, reconstruction assistanceprocesses are now evolving. As a result of researches conducted by donor countries including Japan andinternational organizations, it has been found that Afghanistan is one of the countries where the healthcare
conditions are the worst. Especially, the most common cause of death in Afghanistan is infectious diseases, among
which tuberculosis (TB) accounts for a large percentage. TB is also an important disease from the perspective ofhuman security as it is creating a huge burden for the poor in the country.
The Afghan government promotes development of human resources engaged in TB control by providing
training to TB control personnel (laboratory technicians and TB control officers) at the national and regional levels,
while they have established the National Tuberculosis Control Program (NTP) within the Ministry of Public Healthto develop and promote guidelines for TB control across the country.
Going back to the history before the civil war, Japan started technical cooperation projects in 1974, and in1979 constructed the National Tuberculosis Institute (NTI) through a grant aid. Having decided in 1994 to promote
Directly Observed Treatment with Short-course Chemotherapy (DOTS), which WHO promotes worldwide as ameans of TB treatment, Afghanistan aims to reduce the prevalence rate and the death rate of TB by widely
spreading DOTS.
This project was launched in September 2004 with the project objective of “making quality TB treatment
services with DOTS available across the country” through the reinforcement of NTP as well as cooperation withother donors and NGOs. The project consultation team dispatched in November 2005 and the mid-term evaluationteam dispatched in January 2008 modified the PDM accordingly, after checking the progress of the project and thechanges in NTP and other local organizations. Newly added project activities were the establishment of aframework and promotion of Community DOTS and Urban DOTS with the aim of enhancing the quality of DOTS,
and the enhancement of the function of National and Regional Reference Laboratories.
In April 2009, when it would be about 4 months before the end of the project, the terminal evaluation team wasdispatched to check the performance of the project and conduct evaluation based on the 5 evaluation items.

2. Project Overview(1) Overall Goal
Mortality and morbidity caused by tuberculosis are reduced nation-wide in Afghanistan.

(2) Project Purpose
Quality tuberculosis control services through the DOTS strategy are available nation-wide in Afghanistan.

(3) Outputs
Ԙ NTP’s organizational, institutional and functional capacities are strengthened to provide quality
tuberculosis control services through the DOTS strategy.
ԙ A model of quality tuberculosis control services through the DOTS strategy is established at selectedarea(s) to extract lessons learned for the NTP.
Ԛ Quality laboratory network of sputum smear microscopy with External Quality Assessment (EQA)
system is established.

(4) Inputs (at the time of evaluation)
Japanese Side:  Long-term Experts

8 in total

Equipment

72,438 thousand yen  Short-term Experts

19 in total

Local cost(*)

146,306 thousand yen  Trainees received

9

OthersFollow-up budget 4,171 thousand yen(*) Local cost includes general administration expenses (for office suppliers and communication) and local activity
expense.

Afghan Side:  Counterparts224  Land and Facilities

Project office (in NTI)

Equipment

Local Cost
OthersII. Evaluation Team
Members of Evaluation Team

(1) Team Leader: WATANABE Kozo (Mr.), Director, Health Systems Division
Human Development Department, JICA
(2) TB Control: Reshad Khaled (Dr.), Director in Chief, Head of Board,
Medical Association KENSHIKAI(3) Evaluation and Planning: OKADA Miku (Ms.), Staff, Health Systems
Division, Human Development Department, JICA
(4) Evaluation and Analysis: KINOSHITA Makiko (Ms.), Researcher, Social
Development Department, Global Link Management

Period of Evaluation

21/04/ 2009 – 08/05/ 2009

Type of Evaluation: Terminal

III. Results of Evaluation
1. Project Performance(1) Project Purpose
Quality tuberculosis control services through the DOTS strategy are available nation-wide in Afghanistan.
As for the 1st indicator, “to achieve and maintain 85% of the success rate of TB treatment”, 85% of the success rate
has been achieved and maintained as of 2007.

As for the 2nd indicator, “to achieve and maintain 70% of the case detection rate in the areas where TB control
measures with DOTS have been introduced”, the case detection rate reached 71% in 2007 and is maintained at 73%
in 2008.
However, the status varies among regions and there is also some doubt about the basis for calculation and datareliability. Therefore, it is difficult to evaluate and judge the performance of the project based on these indicators
alone. On the other hand, considering that all activities have been successfully conducted and the outputs have beenachieved, we can say that the project is showing steady progress toward the realization of the project purpose.
(2) Outputs
Output 1 “NTP’s organizational, institutional and functional capacities are strengthened to provide quality
tuberculosis control services through the DOTS strategy.”
Output 1 has been largely accomplished. The capacities of the whole organization and human resources of NTP
have been strengthened, with the aim of enhancing the capacities of staff members (such as TB control officers,
laboratory technicians and healthcare providers) at different levels like province, district and end-facility. The
national TB control system has also been established, with strategies, guidelines, implementation plans and SOPs
for national-level TB control. Moreover, it is reported that NTP provides various types of training on TB to all of
the many healthcare workers as well as administrative officers and laboratory technicians and well-trained humanresources are deployed even at the facilities of TB control service providers. NTP has also worked toward training
standardization by creating various training manuals and guidelines. It is reported that on-site instructions based onsuch guidelines are also provided and almost all facilities are now able to submit quarterly reports. Although the
DOTS coverage reached 100% (a TB diagnosis facility in each township) in 2007, not all the citizens have easy
access to DOTS. In order to provide quality TB treatment services with DOTS, the quality of the whole services
need to be further enhanced by improving Community DOTS to provide better access to DOTS and the TB control
program according to the characteristics of urban areas where there as many public and private healthcare facilities.
Output 2 “A model of quality tuberculosis control services through the DOTS strategy is established atselected area(s) to extract lessons learned for the NTP.”In the model areas, Balkh Province and Kabul Province, DOTS services are provided by healthcare workers andNGO staff who have received training on TB. Moreover, regular meetings are held and quarterly reports arepresented at the provincial level. While the treatment success rate in Balkh Province is 92% (in 2007) and exceeds
the target, the rate in Kabul Province is 57.2% (in 2007), far below the target rate of 85%. In Kabul Province, notonly the treatment success rate but also the case detection rate stays as low as 38%.
It is reported that in Balkh Province a framework for DOTS service has been established thanks to the cooperation
with a NGO conducting BPHS. The poor performance in Kabul Province is caused by a decreasing number ofNGOs providing DOTS service through BPHS in addition to the population migration into the urban areas andpoverty problems. There is a need for TB control measures appropriate for the specific situation in such urban
areas.
Output 3 “Quality laboratory network of sputum smear microscopy with External Quality Assessment(EQA) system is established.”
We can say that Output 3 has also been almost achieved thanks to the relevant activities conducted toward this
output. All the laboratory technicians working at BPHS/EPHS healthcare facilities have received the standard

training course for sputum smear examination provided by NTP. Examination manuals, posters and standard
operating procedures have been distributed and were revised in 2008. EQA has been introduced and spread acrossthe country, and is now conducted quarterly in NRL and local areas (first-class provinces). Regarding the major
errors detected through EQA, a system to promptly provide on-site instructions has also been established. (In someareas only scheduled on-site instructions are available.)
Considering that the establishment of a laboratory network for sputum smear examinations with EQA requires highlevel expertise and commitments of various persons/organizations involved from end-facilities to upper-level
facilities, it deserves recognition that the implementing organization in Afghanistan, which is still in a recovery
period, successfully established a laboratory network in such a short period of time.
(3) Implementation Process
It is generally considered that the project has been successfully conducted, overcoming many obstacles such as thedelay in fund provision from Afghanistan, frequent replacement of NTP directors, a weak mechanism to coordinate
among donors and deterioration of public security. The contributing factors were that strong faith and partnershipwas established between NTP staff and Japanese experts; that the project was considered as part of NTP’s activities;
and that project activities were adjusted when needed according to the needs and situations of NTP.
Moreover, the project worked toward establishing a nationwide framework for TB control through strong efforts tonurture regional and provincial TB control officers and laboratory staff and staff at local facilities, while nurturing
the central human resources in NTP. It has helped produce benefits. The project also provided support forpolicymaking for TB control at the central level and for institutional enhancement such as development ofguidelines. Then it conducted pilot application in model areas and made necessary adjustment on their activities
while giving NTP feedback about issues and lessons learned there. In this way the project has contributed to theestablishment of a TB control system that matches the situations of the work sites. These processes have beenpromoted under the initiative of NTP and with the support of the project. They also cultivated a sense of ownershipof the activities.
2. Summary of Evaluation Results(1) RelevanceThis project is highly relevant to the needs and development policies of Afghanistan and also to Japanese aid
policies toward Afghanistan. Afghanistan is one of the 22 high-burden TB countries in the world defined by WHO
and TB control is considered to be an important issue. TB control is cited as a priority issue in Afghanistan
National Development Strategy (ANDS) and as Goal 6 in Afghanistan MDGs. The expansion of the healthcare
sector is considered as a priority area in Japanese ODA policies toward Afghanistan, and it was confirmed in the
Conference on Economic Cooperation Policies for Afghanistan that the support in the healthcare sector should be
reinforced. The project design that aimed at developing human resources and systems for TB control at all levels
as well as reinforcing NTP’s organizational capacity with a goal of providing quality TB treatment services acrossthe country is considered appropriate.

(2) EffectivenessThe overall effectiveness of this project is considered high although there are still some activities to be conductedin the future for the provision of quality TB control services.
The indicators for the project purpose, the treatment success rate and the case detection rate, have been achieved.
However, it is difficult at this time to evaluate the project performance based on these target values alone, because
there seems to be some doubt about the basis of the data calculation. On the other hand, with most of the planned
project activities successfully conducted and 3 indicators highly accomplished, it seems that the achievements ofthe project have largely contributed to the realization of the project purpose. The flexible approach of the projectwith a focus on NTP’s needs, good communication between Japanese experts and NTP counterparts, and quality
training provided in Japan by the Research Institute of Tuberculosis/Japan Anti-Tuberculosis Association
generated positive effects. On the other hand, frequent replacement of NTP members and the situation where NTP
cannot play a central role in conducting an effective donor coordination mechanism are considered as obstacles tothe realization of the project purpose.

(3) Efficiency
The efficiency of this project is high, considering that inputs from the Japanese side were appropriate in terms of
quality, quantity and timing and they have been made heavy use of. The amount of funding provided by theAfghan side required for the project implementation is still limited (mostly for labour cost), but they have started
covering utility costs and maintenance cost for the NTI building. The counterpart, NTP, is also trying to recruit
new staff members and regional TB control officers as needed. However, due to the delay in the fund provisionfrom Global Fund R4, some of the planned activities (e.g. monitoring, evaluation, on-site instruction andworkshops) could not be carried out as scheduled and had to be cancelled.

(4) Impact
It is not clear at this point whether the project goal will be achieved because, as stated earlier, there is a problem
the reliability of data related to the indicators.
The expected positive impacts were as follows. If a TB patient does not receive appropriate treatment, 10-13
people will be infected in a year and some may die. Considering the fact that drug-resistant TB, which is 10-15
times more infectious, is becoming a problem, we can easily imagine that the negative impact will be even bigger.
In other words, the provision of appropriate TB treatment will prevent further spread of TB. This will help
improve the labour productivity (enabling people to work will bring about positive economic effects), andtherefore will contribute the development of the country. The project has also unexpectedly caused some positive
impacts to other sectors. For example, the project has fostered awareness of the importance of healthcare
education in the Ministry of Education and other ministries through the educational activities and workshops
provided by the project to school teachers and regional religious leaders and therefore generated active
collaboration among different sectors.

(5) Sustainability
Although the frequent replacement of the NTP’s management members was a hindrance, it is hoped that NTP will
enhance its strength as a team and work toward strengthening the organization under the new leadership. As forthe finance, although the budget of NTP itself is limited and its financial sustainability will likely stay low, thebudget for the TB program activities for the next 5 years has been secured with the funding of GFATM R8. In the
future, NTP will have to improve its proposal writing ability to secure external fund sources on its own as well asits program management ability for effective budget activity planning and appropriate budget execution. As forthe technical aspect of sustainability, a considerable amount of expertise regarding a laboratory network has beenacquired. Improvement of management ability, ability to provide quality training and operational research ability
will still have to be worked on as remaining tasks.
3. Factors Promoting Better Sustainability and Impact(1) Factors concerning Planning࡮ This project, with its office in the NTI building, cannot be separated from NTP. Above all, the high degree ofprofessionalism and flexible approach of the Japanese experts have made significant contribution to theenhancement of the organization and capacity of NTP.
࡮ Training in Japan and a third country was very useful to the counterpart as they obtained new skills that
could immediately be utilized for the actual operations.
࡮ The project worked on institutional improvement by supporting the development of policies and guidelines,
which were then utilized for pilot activities in model areas and modified, while the project made feedback to
the central organization of NTP about the issues and lessons learned. Thus a TB control system that fits thesituation of the work sites has been established.
࡮ The project has contributed to the development of organizational and technical sustainability by taking alogistical support approach with a focus on the framework of the national control program.
࡮ The project has raised motivation of the counterpart by summarizing the activities and achievements at thework sites and internationally publishing such information.

(2) Factors concerning the Implementation Process࡮ The project put weight on the needs of NTP, provided flexible support and modified plans when necessary.
Some activities for private-public DOTS, Community DOTS, cultivation examinations etc. were newly
added to the PDM as important activities to achieve goals although they were not included in the originalplan.
࡮ Respect and trust built between NTP staff and Japanese experts promoted smooth communication andtechnical cooperation/support.
࡮ The quick actions taken against serious shortage of antitubercular drugs that occurred twice in 2007 and2009 are considered to have contributed to realizing the project purpose.
4. Factors Inhibiting Better Sustainability and Impact(1) Factors concerning Planning࡮ Due to the delay in the fund provision from Global Fund R4, some of the planned activities (e.g. monitoring,evaluation, on-site instruction and workshops) could not be carried out as scheduled and had to be cancelled.

(2) Factors concerning the Implementation Process࡮ Frequent replacement of NTP leaders and resignation of trained staff members sometimes hindered thesmooth transfer of techniques.
࡮ Although there is a coordination mechanism to avoid redundant activities of concerned parties of donor
countries, lacking strong leadership, NTP has not been able to product positive results by promoting
activities in line with the national TB strategic plan.

5. Conclusion
The project has successfully conducted all the planned activities and utilized efficient inputs and effective approach
despite the challenges they had to face from time to time. As a result, the project has achieved noticeable results.
Although it is difficult to measure the achievement of the project purpose and goals based on the set indicators
alone, the evaluation team has confirmed remarkable improvement of the organization of NTP and the capacity of
the TB control personnel from the central level to the end-facility level. Therefore, it seems clear that the project
has brought benefits to the national TB control program in Afghanistan. Although benefits are thus being produced,
in order to achieve the ultimate objective of providing quality TB control services, it is necessary to fully utilize theguidelines and SOPs already created, to continue monitoring and on-site instruction, and to continuously andsteadily work on some core activities as remaining tasks such as strengthening of a laboratory network with EQA
under the strong leadership of NTP and in cooperation with donors.
Considering the above, it has been confirmed that this project needs to be continued in order for a quality TB
control program to be actualized and expanded to the whole nation of Afghanistan.
6. Recommendations (concrete actions, suggestions and advices about this project)
Considering the implementation status of the project activities and the degree of achievement of the outputs, it isimperative to establish an effective mechanism to coordinate donors. It is also essential to fully utilize theguidelines and SOPs in order to provide and settle standard and quality TB control services across the country.
Moreover, we recommend that the project should address some concrete issues as described below.
(1) Issues to be addressed during Phase I
Output 1 and Output 2
࡮ To strengthen the leadership and management ability of NTP so that effective operation will be realized
࡮ To improve the quality of data relating to TB control by improving accuracy of records and reports (or bycollecting such data as the number of potential patients and the TB positive rate)
Output 3
࡮ To create SOP for the smear transportation system where sputum samples collected at BHC are sent to the
CHC level for examination so that the laboratory network will be strengthened and access will be expanded࡮ To infuse more human resources and nurture their capacity to perform cultivation examinations throughon-the-job training in NRI
࡮ To infuse more human resources and realize the introduction of drug susceptibility test (to find drug-resistentpatients)(2) Issues to be addressed continuously in Phase I and Phase II
Output 1 and Output 2
࡮ To enhance NTP’s program management ability (planning, budget management and execution,
implementation, M&E and improvement)࡮ As for the provision of quality TB control services, to work on the issues of nationwide expansion ofCommunity DOTS, Urban DOTS in Kabul, support to the groups of vulnerable people and drug-resistant TB

Output 3
࡮ To enhance the capacity of laboratories and NRI across the country¾ To enhance and strengthen the quality of the EQA system¾ To work toward improving the capacity to perform cultivation test and drug-resistant test
7. Lessons Learned (through this project that will be useful for the identification, formulation,
implementation, operation and management of similar projects)
8. Follow-up Situation
It has been decided that this project will be continued as Phase II (5 years). When the Phase I terminal evaluationteam was visiting the project site, the Phase II ex-ante evaluation was also carried out and the outline of Phase II
was confirmed.