Summary of the Evaluation
I. Outline of the Project
Project Title:
Country:

The Project on Support for Expansion and

Afghanistan

Improvementof

Literacy

EducationinAfghanistan
Sector:Cooperation Scheme:

Basic Education

Technical Cooperation Project
Total cost (as of the time of evaluation):

DivisioninCharge:Basic Education Team I, Group I 324.777 million Japanese yen(BasicEducation),

Human Implementing

Development Department

organizationinAfghanistan:

Literacy Department (the LD), Ministry of
Education, Afghanistan (the MOE)

(R/D) June 1, 2005
Cooperation
PeriodOrganization in Japan: JICA
RelatedCooperation:

Two years and four months Non-Formal(March 1, 2006 - July 31, 2008)

Education

Strengthening

of

(NFE)

in

Project

Afghanistan (March 25, 2004 - March 31, 2007)

1-1 Background of the Project
During the long civil wars which caused disruption to education at all levels in Afghanistan, many ofthe people there have lost chances of receiving education. It is said that this fact left the estimation of
11milion illiterate population. It is an urgent issue to promote the literacy education for those who
could not receive educational opportunities to solve their own problems, to improve their life, which
eventually leads to the establishment of peace, the construction of democratic nation and thedevelopment of its economy and society.
Under such a situation, the Ministry of Education (MoE) sees the promotion of literacy education as
one of the important sectors in the educational policy and the Literacy Department (LD) mainly
operates activities of literacy education. In addition, international and local NGOs also provide not
only literacy class to adults but also various classes integrated into the local and individual needs.
The LD does not grasp, however, the achievement of provincial LD’s activities or the situation ofNGO’s implementation of literacy classes well and the capability of planning, implementation andmanagement is still not enough to promote literacy education effectively. Furthermore, as the needsof literacy education is large and diverse, flexible literacy programs are required.
With such a condition, it is desirable to promote literacy education nation-wide including various
activities with collaboration of NGOs rather than to implement literacy classes only under the LD.
With the request by the government of Afghanistan for Japan to support the administrative capacity ofLD as a center of promotion of literacy education as well as response huge needs of literacy
education, the Project on Support for Expansion and Improvement of Literacy Education in
Afghanistan starts in March 2006 for 2 years and 4 months.
1-2 Project Overview(1) Overall Goal
The literacy rate in the project provinces have increased.
(2) Project Purpose
The provision and the quality of literacy education are enhanced in the project provinces through the
improvement of the overall management capacity of the Literacy Department (LD).
(3) Outputs
1) Capacity of the LD is improved in planning and management of literacy classes in the projectprovinces in the following aspects: Data management, Teaching material  management, andCapacity Improvement of Supervisors
2) Provision of literacy classes in the project provinces is expanded(4) Inputs (As of the time of evaluation, including some fixed plans)
Japanese side:
Short-term experts
Accepted trainees(Training in a third

6 people Total 49 M/M
6 people Total 3 M/M

Operation cost

75.197 million yen

country)
Afghan Side:C/P staff

41 (Including Working

Local costProvision of Project officeGroup members)
II㧚Evaluation Team
Leader:MembersofMr. Katsuhiko KamiyaGroup Director, Basic Education
Human Development Dep.,, JICA

Education Planning:

Senior Advisor, Institute for International

the Ms. Tomoko Masuda

Cooperation, JICA

evaluation team
Cooperation Planning:
Ms. Yumi Tokuda

Evaluation Analysis:
Ms. Rie Koarai

Program Officer, Basic Education Team I,
Basic
Education
Group,
Human
Development Dep., JICA

Save the Children Japan

Evaluation Period

January 18, 2008 - February 8, 2008

Evaluation type: Final evaluation

III. Overview of Evaluation Results
1. Achievements of the Project(1) Outputs
1) Capacity of the LD is improved in planning and management of literacy classes in the projectprovinces in the following aspects: Data management, Teaching material  management, andCapacity Improvement of Supervisors
Regarding the three functions of data management, management of teaching materials, and capacity
improvement of supervisors, the following outputs were observed through the activities of theWorking Group (WG).
i) Data management[Outcome]
By the development of 4 types of data collection forms and the implementation of training, theclarification of existing data collection processes and high-quality data collection have become
possible. So far, data collection of 4,356 courses has been completed.
[Challenging issue]
The training on data input and analysis has been implemented already, but full monitoring of data
input, analysis, reporting, and the ability to use data for planning will be necessary by the end of the
Project.
ii) Teaching material management[Outcome]
Three forms of the distribution and management of teaching materials have been developed. After the
training on the forms, the process of distributing teaching materials including the new literacy primer(LAND Afghan), to the levels of province, district, class, and facilitator/learner, improved.
[Challenging issue]
The budget for printing and distributing teaching materials in the future is insufficient, and there is noprospect of any budget increase at present.

Further dialogues for securing the budget are needed.

iii) Capacity improvement of Supervisor[Outcome]
Examples have been observed, where supervisors' methods of instructing facilitators have improved,since the supervisors came to understand their roles (monitoring of literacy classes, appropriate
instructions to facilitators, etc.) through training programs and reading the supervisor's manual, which
WG members developed,[Challenging issue]
There is room for further improvement in the abilities necessary to improve the manual and training.
In addition, full monitoring on the performance of supervisors needs to be continued in the future.
2) Provision of literacy classes in the project provinces is expanded[Outcome]

In cooperation with NGOs, literacy education was provided to 10,239 people (378 classes). Throughthe training sessions implemented by NGOs for the facilitators of the LD, the Project contributed toenhancing the capability of the LD.
[Challenging issue]
Regarding the regions for NGO activities, the selection criteria included the regions where it isdifficult for the LD to open literacy classes under its direct management. Some examples, however,were observed where there was not enough coordination between the provincial LD and the NGO.
(2) Project Purpose (The provision and the quality of literacy education are enhanced in the projectprovinces through the improvement of the overall management capacity of the Literacy Department(LD).)
1) Quantitative improvement in literacy education[Outcome]
Through the cooperation with NGOs, the opportunity to attend literacy classes was provided to
10,239 people (target number was 10,000) in regions where it is difficult for the LD to open literacy
classes under its direct management.
2) Qualitative improvement in literacy education[Outcome]
It is considered that the Project was able to contribute, to some extent, to the improvement of thequality of literacy education, through the provision of a new literacy primer and the enhancement of
the LD's capability, such as the improvement of the supervisors' ability of instructions to facilitators.
[Challenging issue]
The Project’s implementation period was too short and it did not receive enough inputs to achieve an
improvement in the quality of literacy education. In addition, more thorough guidance by supervisorsand support for facilitators are necessary.
(3) Overall Goal (The literacy rate in the project provinces has increased )[Outcome and challenging issue]
As stated above, literacy education opportunities were provided to 10,239 people through the Project.
Even so, in view of the enormous need for literacy in the target areas and the whole area of
Afghanistan, the target achievements of the Project did not contribute enough to literacy rate
improvement. In addition, to increase the literacy rate, the establishment of an environment forcontinuous learning needs to be considered.
2. Evaluation by Five Criteria(1) Relevance: High- The importance of literacy and non-formal education has been addressed in the subsequent policy
documents of both the Afghan government and the Japanese government.
- With the estimation of 8 million of adults who did not receive basic education in Afghanistan, theneed for improving capacity of the LD and increasing access to literacy education has been enormousand the Project approach to tackle both the challenges is of high relevance.
(2) Effectiveness: High- The expansion of the provision of literacy education will likely be achieved by the end of the
Project period, reaching at the target population 10,000.
- The capacity of LD in the administration of literacy education, concerning management of data andteaching materials as well as strengthening of supervisors' abilities, has improved. There is, however,room for further improvement.
- Some positive practices have been found at the field level. Yet, it cannot be said that these outputs
have achieved the quality of literacy education delivered in each literacy class enough, which requires
more direct inputs from longer-term perspectives. Considering the fact that the Project purpose set is
too ambitious for its inputs and period, it can be evaluated as high.
(3) Efficiency: Moderate- The quality and timing of the training provided were relatively satisfactory for WG members of LD,
professional staff members and supervisors.
- To improve capacity of the LD as well as to increase the provision of literacy education, inputs suchas human resources were not sufficient enough and the Project period was too short.
- Delays of printing post literacy books hindered the efficient and timely distribution of teaching andlearning materials.
(4) Impact: High- The positive impact observed is that the LD has come to be aware of the importance of theintegration of literacy education and life skills through the association with the implementing NGOs.
- There is a possibility that various forms and the supervisors’ manual developed in the Project will
be utilized in other donors’ project in near future.
(5) Sustainability: Low- The promotion of literacy and non-formal education in Afghanistan is one of the priority issues of
the National Education Strategic Plan (NESP) for Afghanistan approved in 2007 and correspondswith its policy and it is likely to be kept prioritized as it is now.
- On the other hand, while the policy of LD is not clear and its structural reform is also expected, thetechnical, systematic and financial capabilities are not yet ensured. Therefore, it is still insufficient toeffectively continue the activities carried-out in this Project. It is necessary for the LD to develop
action plans and integrate these activities into their annual plans, so that the effective system can be inplace including appropriate financial and human resources.
3. Factors Contributing in the Production of Effect(1) Factors Concerning the Planning- Some examples of a cooperation system between the LD and NGOs were demonstrated and theimportance of the NGO’s role was recognised by the LD to some extent, although the synergetic
effect of both approaches of the LD’s capacity improvement and expansion of literacy education
based on the NGO’s cooperation was not particularly large.

(2) Factors Concerning the Implementation Process- Placing the Project office in the LD and cooperating with the LD enabled smooth communication
between Japanese experts and C/P of the LD.
- The style of technical cooperation that respected Afghan ownership was highly appreciated by theAfghan side.
4. Problems and Factors that Raised Problems(1) Planning- The quantity and contents of input were not appropriate for achieving the part of the Project Purpose
regarding improving the quality of literacy education. In addition, the Project period of two years andfour months was too short to allow a contribution to the improvement of quality. The Project resultsof the LD's capability improvement and expansion of literacy classes have not necessarily lead to an
improvement in the quality of education, and the Project Purpose set at the beginning was too high.
(2) Implementation Process- The deterioration of the security situation limited the regions where the Project could be
implemented and those where Japanese experts were available to conduct monitoring.
- Appropriate qualitative and quantitative indicators were not set in PDM over one and a half years
after the commencement of the Project, which was because the mutual understanding of indicators in
the PDM among the Project members and staff members of the LD, and PDM was not used forappropriate monitoring of the improvement in the LD's capability or that of the quality of literacy
education.
- The three WGs and Japanese experts in charge were concentrating solely on their own activities,
and coordination and management among WGs and for the entire Project were not sufficient.
5. Conclusion- The capacity of the LD in the administration of literacy education, concerning management of data
and teaching materials as well as strengthening supervisors’ abilities has improved.
- Through the cooperation with NGOs where literacy classes were provided, the Project contributed
the expansion of literacy education.
- There are several stages between capacity improvement of the LD and quality improvement ofliteracy education at a field level. Therefore it was too short to contribute to the improvement ofliteracy education through capacity improvement of the LD in the period of the Project. The Project
Purpose was also set too high in the beginning.
- Other approaches or activities could have been added and used to achieve the both purposes ofcapacity improvement of the LD and the quality improvement of literacy education itself.
-This Project will be finished as it plans. To improve the quality of literacy education, however, alonger term and direct assistance is needed. It is suggested to consider a succeeding project utilizing

and spreading of the outcomes of the Project.
IV. Recommendations and Lessons Learned
1. Lessons Learned<Purpose and approach of the Project>(1) The original PDM aimed to improve the quality of literacy education (purpose) through the
improvement in capability of the LD (Project Outputs), but it was difficult to achieve both aims
during the Project period of only two years and four months under an insufficient education system ina post conflict country. The setting of the Project Outputs and Purpose in the first place is considered
to be too ambitious in comparison with the Project Inputs (period and human resources). ProjectOutputs and Project Purposes need to be feasible and tangible, so that the people of both Afghanistan
and Japan who implement the Project can conduct the activities with a clearer vision.
(2) Regarding the Project Output that was the capability improvement of the LD, the number of fields
was limited to three due to considerations of the size of the Project input, and the focus was on the
improvement of administrative capability at the central government level. For this reason, the Project
could not sufficiently become involved in the distribution and establishment of the developedmaterials and mechanism at a field level. In addition, under the current conditions where theadministration is not working sufficiently and where there is a large disparity among regions,
top-down type administration on literacy is limited, and the LD's improvement in capability does notnecessarily lead to the improvement of literacy education in classes. Bearing capability improvementin mind, while looking to the medium- and long-term, it can be seen that activities that directly
contribute to the improvement of quality in classes are necessary, although there can be limitation ofactivities under the security condition.
(3) The linkage between component 1 (the LD's capability improvement) and component 2(development of literacy classes by subcontracting it to NGOs) was not fully expected from the
beginning, and it was a design that caused some difficulty in mutually utilizing the lessons that werelearned respectively. It might have been possible to take the approach of setting a vision of the LD,bearing in mind the cooperation with NGOs from the very beginning, and trying to improve thecapability towards that status. More prudent examination was necessary at the time of forming andimplementing the Project.
(4) The fact that the Project team had an office in the LD and worked with the C/P every day helped
to promote the communication between Japanese experts and the C/P. It was effective for building therelationship of mutual trust and the Afghan ownership. It was highly evaluated to have a good
relationship among them especially under the circumstance where other donors did not establish an
office at the site.
<Direction and possibility of literacy education>

(5) Although the LD and NGOs were in a complementary relationship in the development of literacy
classes, the LD did not have a clear vision of cooperation with NGOs, and there was insufficient
cooperation between both sides. With this Project, the way of collaboration with how the LD has
become clear through the development of literacy classes subcontracted to NGOs, and the LD'sawareness of NGOs has increased. However, the direction for developing literacy education incooperation with external resources, including NGOs, has not yet been found. More active
discussions based on the experiences gained by this Project are expected to be carried-out in the
future.
(6) The Project coordinated with other JICA-assisted project of tuberculosis (T.B.) prevention andtreatment though the literacy classes subcontracted to NGOs. Posters describing how to prevent andtreat T.B. were distributed to literacy classes and T.B. health workers were invited to give facilitators
training for basic information of TB. In these literacy classes, accurate information regarding T.B.
was delivered to the people. In this way, literacy classes can be a place for offering various
information useful to learners’ daily life through coordination with other sectors’ activities.
<Formation and implementation of projects in the future>(7) Considering the characteristics of Afghanistan where the situation changes drastically in a shortperiod of time, and that obtaining enough information before starting a project is difficult, it is clearlynecessary to make a flexible system that can cope with changes at the site. When a project has a task
to perform, a change of activity results in a change of contract, and flexible handling is difficult.
However, mutual communication and flexible handling in a contractual relationship, at the very least,
are essential. The LD highly appreciates the CLC model proposed in the Strengthening of
Non-Formal Education Project which had been implemented ahead of this Project, and an
introduction to the CLC is provided in the literacy education policy. Based on such conditions, it isconsidered that a larger contribution to the LD's policy could have been possible if the developmentof CLC type literacy classes (effective combination of literacy education and life skills, cooperation
between the LD and NGOs) had also been promoted in this Project. It is necessary for the people
concerned to be very aware that a project implemented in Afghanistan needs to be particularly
flexible, and that various lessons that JICA has learned from its projects need to be shared.
2. Recommendations<The team recommended the Project to take the following actions by the end of the Project>(1) It is necessary to practice activities (periodic joint meetings, etc.) for sharing information among
the WGs and the components (capacity improvement and provision of literacy classes subcontracted
to NGOs).
(2) Each WG has produced outputs such as improved formats, training programs, manuals, and so on.
These improvement need to be integrated in the LD’s daily operation and utilized in their policy
making. In order to support this process, the Project will help the WGs to consider what they can doafter the Project and develop action plans based on their experience.
(3) The target areas of this Project were three provinces and one city. The continuity of the activities
would be enhanced by openly sharing the Project outputs with the people in charge of literacy inother provinces and increasing their knowledge. There are other donors in the field of literacy, andinformation also needs to be provided to them to gain their cooperation. In addition, it is necessary to
set-up opportunities to share the suggestions on the cooperation with NGOs, which were made basedon experience gained during the Project, and discuss for further cooperation.
<Team recommended the following points for the medium- and long-term prospect>(4) It is obvious that the LD alone cannot cover the vast number of illiterate population and resources
need to be sought every possible institutions such as NGOs, other ministries and agencies to operate
literacy classes. Through this Project, the LD has come to appreciate NGO activities to a certain
degree by gaining an understanding of such activities, although the LD's understanding on the use ofNGOs has not improved to a great extent. Earnest discussions should be held within the LD on thecollaboration with NGOs in the future and the role the LD should play as their supervisor.
(5) With the introduction of the new literacy primer and clarification of the lesson schedule of 6 daysa week, the facilitators meetings that had been held until then (Thursday training) were abolished. As
a result, the opportunity for facilitators to continuously improve their skills was lost. To expand
literacy classes in the future, there is no other way than to use human resources with limited basic
abilities or low education levels. To use them, in addition to the training given at the time of
employment, more technical support to help them conduct daily work properly will be necessary.
(6) At present, the assessment criteria for the learners who completed literacy classes are not clear,and the number of those people also has not been correctly recorded. It is necessary to clarify theminimum criteria and make a mechanism for facilitators to easily evaluate the learners' ability.
(7) The people involved in the improvement of literacy administration and manual development wereprimarily the members of the LD in Kabul, and provincial needs are not necessarily reflected. In therevision of manuals in the future, needs of various local areas should be included.
(8) In future cooperation, while the strong initiative to increase quantity (to cover all provinces) istaken a place, support for enhancing capability of the LD needs to be continued as well. Furthermore,
including a mechanism that benefits literacy classes at the site is also necessary.
End