Summary of Terminal Evaluation
1. Outline of the ProjectCountry:Lao

People’s

Democratic Project Title : Improving In-service Teacher Training for

Republic

Science and Mathematics Education (ITSME)

Issue/Sector : Basic Education

Cooperation Scheme : Technical Cooperation Project

Division in Charge : JICA Laos OfficeTotal Cost (At the time of evaluation) : 302 Million Yen(R/D): 19 Nov. 2009

PartnerCountry’s

Implementing

Organization:

Department of Teacher Education (DTE), Ministry of
Period of

22 Feb. 2010- 31 Oct. 2013 Education and Sports (MOES)

Cooperation
Supporting Organization in Japan : NA
Related Cooperation : The Project for Improvement of
School Environments in Champasack and Savannakhet
Provinces(Japan’s Grant Aid for Community Empowerment)
1-1

Background of the Project

As one of the top prioritized sectors in contributing to poverty reduction in Lao PDR, various efforts
have been made in the education sector under the following three pillars: (1) improvement of equitable
access; (2) improvement of quality and relevance; and (3) improvement of administration andmanagement. Although the enrollment rate in the primary education has been improved, other indicatorsthat are related to the quality of education, for example, repetition, dropout, and completion rates havenot been improved as expected.
With these backgrounds, the Education Sector Development Framework (ESDF) places an emphasis on
the improvement of teachers’ quality by strengthening the capacity of in-service training.
The Japan International Cooperation Agency (JICA) has supported the improvement of teacher
education through the Project for Improving Science and Mathematics Teacher Training (SMATT) from2004 to 2008, which aimed at improving quality of science and mathematics lecturers at teacher
education institute (TEI). Training contents and learner-centered approaches of SMATT have beenrecognized as relevant and useful for persons and organizations concerned. Recognizing theseachievements and impacts of SMATT project, the Government of Lao PDR requested the continued
support to enhance and expand the outputs of SMATT project to the school level.
1-2

Project Overview(1) Overall Goal
Quality of lessons in target provinces is improved.
(2) Project Purpose
Quality of science and mathematics lessons in target schools is improved.

xi

(3) Output
Output 1: Mechanism for improving lessons is strengthened.
Output 2: Human recourses to promote improvement of lessons are strengthened.
Output 3: Materials for improving lessons are developed.
(4) Inputs (At the time of evaluation)【Japanese Side】
Expert: 6 person
Training in Japan: 30 participants
Equipment: A total of US$273,950
Local activity cost: A total of US$196,866【Lao Side】
Counterparts (C/P): A total of 40 persons
Local activity cost: A total of 746,623,000 KIP
Provision of office space for the Project and utilities
2.

Evaluation Team
Ms. Machiko KAMIYA, Team Leader, Senior Representative, JICA Laos OfficeMemberDr. Keiko MIZUNO, Teacher Education, Education Policy Advisor to MOES
Mr. Kazuyuki KAKUDA, Cooperation Planning, Representative, JICA Laos Office
Ms. Tomoe TAIRA, Evaluation Analysis, Consultant, IC Net Limited

Period of
Evaluation

1 July – 19 July. 2013

3.

Results of Evaluation

3-1

Achievement of the ProjectType of Evaluation: Terminal Evaluation(1) Outputs
Output 1:

The Output 1 is mostly achieved. The extent of achievement will be higher if the indicator

1-4 is met by the end of the Project.
The Indicator 1, i.e., “An official document agreed between TTC and PES to establish a professional
network for supporting IS activities in target provinces” is achieved. The “ITSME Future Plan and
Implementation Strategy,” which clarified the roles and functions of the DTE, PESSs, DESBs, andTTCs in supporting IS activities that could also serve as a professional network, was signed by the
director of the DTE on April 1, 2013 and agreed among the DTE, target PESSs, DESBs, and TTCs. The
Indicator 2, i.e., “MOES budget approved to operationalize the mechanism in target provinces” is alsoachieved. At the central level, the DTE budget of LAK 746,623,000 in total was allocated for the
Project activities throughout the Project years except the period from June 2013 onward. At the local
level, the PESSs and DESBs bore necessary expenses for school monitoring, and all schools bore
necessary expenses for IS. The Indicator 3, i.e., “Utilization of developed model lesson plan in target
schools,” is fully achieved. Compared to the status at the baseline, all the target schools came to utilize
the model lesson plans developed by the Project in the IS activities. The Indicator 4, i.e., “Feedback on
the utilization of the model lesson plan to PES through DEB,” is expected to be achieved. Verbal
reporting has been conducted fully by all the target eight DESBs. Periodic reporting using thexii

monitoring form is also conducted by four DESBs. Since the Project has instructed the remaining four
DESBs to submit the form and received positive reactions from them, it is expected to be conducted by
the end of the Project.
Output 2:

Overall, the Output 2 is mostly achieved.

The Indicator 1, i.e., “Quality of developed model lesson plans is improved,” is fully achieved.
According to the analysis by the Project experts, on a scale of 0 to 9, the score on the quality of 25
model lesson plans in total developed by ITSME trainers by group improved from 7.0 in the first year to
7.3 in the second year, and 8.8 in the third year. The analysis shows that the developed lesson plans met
the target quality in which i) objectives, ii) learning process /student activities, and iii) evaluationquestions of a lesson are clearly set and appropriately formed. The Indicator 2, i.e., “ITSME trainers’
subject knowledge” is achieved at a satisfactory level. Compared to the pre-test, the average scores of
22 ITSME trainers in the post-test on their subject knowledge has increased in all the seven targetedunits of Mathematics and Science. Although the ideal level of achievement would have been 100%, it isfair to say that the level of improvement is satisfactory given the constraints such as a few inappropriate
textbook contents and the limited learning environment of the ITSME trainers. Enhancing knowledge on
the other units remains a future challenge.
Output 3:

The Output 3 is expected to be achieved by the end of the Project.

The Indicator 1, i.e., “The number of model lesson plans developed against the target in the plan statedin activities” (First and second year: no target; third and fourth year: 10 units each) is expected to be
fully achieved by the end of the Project. The target number of model lesson plans was alreadydeveloped in the first to the third year (First year: three units; second year: nine units; and third year: 10
units). The number of model lesson plans exceeding the target has been mostly developed in the fourthyear (14 units) as of June 2013 and is expected to be fully developed by the end of the Project. The
Indicator 2, i.e., “Compilation of materials for improving lessons developed by activities in 3-7,” isexpected to be achieved by the end of the Project. The Project has started compiling the developedmaterials such as model lesson plans and manuals. The Indicator 3, i.e., “A document prepared andsubmitted by the project to propose to MOES for their consideration to adopt the developed materials as
national standard,” is expected to be achieved by the end of the Project. The Project has started
preparations for a set of concrete recommendations on how to utilize the Project materials in theexisting MOES training materials and activities.
(2) Project Purpose
The Project Purpose has been achieved. The level of achievement is expected to be higher if theabove-mentioned outputs are fully produced by the end of the Project.
The Indicator 1, i.e., “Improvement of the quality of lesson performed by target school teachers,” isachieved. According to the analysis of lesson observation in the 32 target schools of the end-line survey
and interview surveys at the eight schools visited in the Terminal Evaluation, in general, the target
school teachers became able to i) set clear objectives, ii) lead students’ activities in an appropriate order,
and iii) summarize and evaluate the lessons. Therefore, the quality of lessons, especially the skills in
forming lessons, by the target school teachers has improved. The teachers are expected to become ablexiii

to teach accurate subject knowledge to students by utilizing the gained skills of forming lessons. The
Indicator 2, i.e., “Improvement of the quality of lesson plans made by target school teachers,” is fully
achieved. According to the analysis by the Project experts and ITSME trainers of 26 lesson plansdeveloped by the target teachers, the quality of lesson plans are improved in the above-mentioned threepoints, especially the point i) mentioned above. A more appropriate arrangement of the points ii) and iii)
based on accurate subject knowledge remains a challenge. In addition, the questionnaire survey in the
Terminal Evaluation shows that vitalization of IS through the utilization of model lesson plans iseffective for improving the quality of lessons (100 % of the respondents or 36 trainers in total strongly
agree or agree on the effectiveness.)
3-2

Summary of Evaluation Results(1) Relevance:【Very High】
Relevance is evaluated as very high. The Project has been highly consistent with the national andeducational policies of Laos as well as the Japan’s assistance policy for Laos, which prioritize the Basic
Education and improvement of education quality for achievement of MDGs. According to several target
DESBs, the improvement of quality of lessons through IS activities has contributed to reducing thestudent dropout rate. Given this, the Project is consistent with one of the important agendas of reducing
the dropout rate mentioned in the mid-term review of ESDP. Although the MOES has encouraged
schools to use IS to improve teachers’ skills, effective activities have not been available to teachers. The
Project has been providing effective materials and their usage to boost IS activities for quality lessons,
therefore has responded to on-the-ground needs. Japan has enriched knowledge and experience of IS.
The Project’s approach to improve the quality of lessons through IS with the usage of Japan’s
experience was effective for helping overcome a major educational challenge.
(2) Effectiveness:【High】
Effectiveness is evaluated as high for the following reasons: 1) two indicators of the Project Purpose
have been achieved; 2) the approach to improve the quality of lesson plans as well as lessons through IS
activities have functioned well and contributed to the achievement of the Project Purpose; and 3) all thethree Outputs have contributed to the achievement of the Project Purpose.
(3) Efficiency:【High】
Efficiency is evaluated as high. The level of achievement of all the three outputs is mostly high. In
general, the inputs from both the Japanese and Lao sides are also utilized efficiently. Most activities atthe local level, such as TOTs, monitoring of schools by DESBs, and IS activities at schools are
conducted as regular work therefore within the regular budget framework (with budget assistance by the
Project for TOTs). These activities have contributed to the achievement of the Outputs which in turnhave helped achieve the Project Purpose. The preconditions in the PDM ver. 1 for the commencement of
the Project were not fully met before the implementation of the Project. However, the Project has
achieved the Outputs with the existing budget and has utilized the training in Japan to maximize theProject outputs.

xiv(4) Impact:【High】
It has not yet come the time for the two indicators of Overall Goal, i.e., “Improvement of the quality oflesson performed by teachers in the target provinces” and “Improvement of the quality of lesson plans
made by teachers in the target provinces” are achieved. However, according to the interview surveys in
the Terminal Evaluation, some non-target schools have started utilizing the model lesson plans,
therefore it is predicted that the quality of their lesson plans are being improved to some extent.
Impact towards the direction of Overall Goal has also seen as followings: 1) target PESSs and DESBs
have developed plans for expanding the Project activities and started distributing the Project materials
to non-target areas; and 2) several DESBs have shared the model lesson plans in their organization andprepared the implementation structure for continuing and expanding the Project activities. In addition,
the following noteworthy impacts are seen: 1) ITSME materials have been utilized in the Mathematics
and Science lessons of the two target TTC; and 2) in all the eight schools visited in the Terminal
Evaluation, teachers have developed lesson plans for the subject of the Lao language, which is not atarget subject.
The Project showed that the utilization of IS has contributed to the improvement of the quality oflessons. Therefore, the Overall Goal is expected to be achieved if the necessary measures are taken by
the MOES for securing the Important Assumption, i.e., “IS is conducted in all the schools in the target
provinces by the Lao side.” Negative impacts are not found.
(5) Sustainability: 【High】(Central Level: Satisfactory, Local Level: High】
Sustainability is totally evaluated as high for the following reasons:1) although sustainability at thecentral level is evaluated as satisfactory, there is clear common understanding among people concernedon future direction and measures to be taken for that after the completion of the Project, and 2)
sustainability at the local level is evaluated as high.
Central (DTE) Level:
Considering the political/institutional, organizational/financial, and technical aspects, central-levelsustainability is evaluated as satisfactory. IS continues to be the activities to be conducted at schools andmonitored by PA as stated in the MOES’s guideline on Pedagogical Advisor. It is hoped that the DTE
will develop concrete action plans along with the signed “ITSME Future Plan and Implementation
Strategy” and link the Project activities and materials with the existing MOES systems, such as PA
guideline, EQS training, in-service teacher upgrading training, and TTC pre-service training curriculum,
to ensure the continuation and expansion of the Project activities. It is also required for the DTE toassign the necessary number of human resources and secure the budget in the future, by considering
incorporating the Project activities into the budgeted ESDP and the existing training courses. The DTE
is required to consider measures to ensure quality dissemination to non-target areas.
Local (TTC/PESS/DESB/School) Level:
Local-level sustainability is evaluated as high. The ITSME trainer team among TTCs, PESSs, and
DESBs in the Project activities for improving the quality of lessons performed well and is expected tocontinue performing well. The decision makers of the target TTCs, PESSs, and DESBs have committed
to assign ITSME trainers to continue and expand the Project activities. Since PESSs and DESBs haveconducted school monitoring and schools have conducted IS with their own budget, the Project
activities are expected to be financially sustained in the future as well. Capacities of ITSME trainers

xv

have been enhanced to the level that they can develop model lesson plans by group with instruction by
the Project experts. It is expected that PESS/DESB/TTC keep learning among themselves to upgrade
their capacity and ensure the high quality of lesson plans in the future. All target schools are now
conducting IS activities through utilization of model lesson plans. Therefore, it is expected that thequality of lesson plans and lessons will be continuously improved at the target schools.
3-3Contributing Factors(1) Factors regarding the contents of plan
1) Utilizing the ITSME trainer team composed of those from different organizations as the core
implementation unit helped each of the organizations to play a distinct role. The ITSME trainer teamplayed the main role in achieving all Outputs thus contributed to the achievement of the Project
Purpose.
2) The Project has taken the approach to enhance capacity of the teachers through utilizing the existing
system of IS. This approach has contributed to achievement of the Project Purpose, improving thequality of lessons and lesson plans.
(2) Factors regarding the implementation process
1) The Project has made it clear that TOT, monitoring, and IS are routine tasks in the regular budget
framework. This contributed to securing financial sustainability.
2) The Project showed enjoyable lessons by demonstrating lessons based on model lesson plans. This
undertaking boosted the interest of the ITSME trainers and teachers in developing such lesson plans,
therefore contributed to emergence of the positive impacts. In fact, many target schools have started
developing at their own initiative the lesson plans of units and subjects not covered by the Project.
3-4

Constraining Factors(1) Factors regarding the contents of plan
The initial PDM did not take into consideration such factors as a few inappropriate textbook contents
and the poor learning environment of the ITSME trainers and teachers. Consequently, they caused
delays in the activities.
(2) Factors regarding the implementation process
It took time to modify the PDM ver.1, causing difficulties in conducting the Project activities asplanned, especially at the beginning. However, after the PDM ver.1 was modified as needed (ver. 2), theremaining activities since then have been mostly implemented as planned.
3-5Conclusions

The Project has achieved the Project Purpose as planned with very high relevance together with higheffectiveness and efficiency. Impact is also high. The Overall Goal is expected to be achieved if
necessary measures are taken, especially those by the MOES. The Project has given some positive
impacts on non-target areas. Sustainability is expected to be satisfactory at the central level and high at
the local level. Thus sustainability at the central level is one of the key issues in ensuring thecontinuation and expansion of the Project activities in the future.
Given the conclusion above, it is appropriate to terminate the Project in October 2013 as planned, andxvi

consider conducting some follow-up activities a few years after the termination in order to further
facilitate the achievement of the Overall Goal and maximize the Project outcome.
3-6

Recommendations

The Team concludes that the Project has achieved the remarkable results.
In order to strengthen the continuous efforts on the ground and further expand its achievement to
non-target areas, the following are recommended for future (and continuous) tasks to be pursued byrelevant stakeholders.
(1) Sharing ITSME approach, its impact and outcomes at national level (Tasks to be led by the Experts)
A national dissemination seminar should be organized to share ITSME experiences andachievements, and to discuss the way forward for future dissemination with relevant central
authorities of MOES as well as Development Partners. A comprehensive dissemination package
should be consolidated including all the developed materials accompanied with recommendations on
how to utilize or integrate such materials in different programs, standards, or guidelines.
(2) Maximizing the existing channels and systems for dissemination and monitoring (Tasks to be facilitatedby DTE with support of the Experts, in collaboration with concerned MOES departments and TTCs.)<Trainings and Guidelines>
The efforts to incorporate ITSME approach into relevant training programs, guidelines andnationally set standards for teachers’ evaluation and certification have been initiated and should beconcretized. EQS trainings for principal and teachers, PA guidelines and IS manuals, monitoring
tools for teachers, are some examples. Since different departments are involved in such trainings anddevelopment of guidelines targeting at the quality improvement, a workable coordination mechanism
should be discussed, agreed and supported by relevant authorities of different departments
concerned. It is expected that the Experts facilitate this process to the extent possible.in theremaining cooperation period.
<TTC pre and in-service trainings>
It is recommended that discussions among TTCs should be facilitated by DTE to maximize TTC preand in-service trainings for nationwide dissemination. The experience, expertise and initiatives bytarget TTCs (Champasak and Savannakhet) should be widely shared and utilized by other TTCs aspart of “ITSME Future Plan and Dissemination Strategy”, with appropriate guidance and orientation
to be provided by DTE.
(3) Strengthening and expanding key human resources (MOES, TTC, PESS, DESB)
Bringing up the professional capacity of the core human resources beyond the cooperation period
shall be a key challenge to quality dissemination of ITSME approach. Opportunities to shareexpertise and experiences among trainers of different programs should be ensured. A joint working
mechanism based on the professional network among TTC, PESS and DESB should be continued
and strengthened. . In this regard, it is suggested that issuance of a ministerial guidance or decreeshould be sought to support and facilitate such a joint working mechanism.
The following are additional suggestions that might be considered by relevant authorities for the effectivexviidissemination.
(4) Utilizing district or school “clusters” for dissemination and monitoring
According to the results of questionnaire survey to ITSME trainers, it is confirmed that the school
cluster mechanism is well functioning in the target areas, and recognized as an efficient means ofdissemination and monitoring. It is encouraged that this mechanism should be further strengthened
so that it can provide meaningful opportunities for principals and teachers to share experiences in
the classroom and to discuss common issues and challenges relating to teaching learning practicefor finding appropriate solutions.
(5) Identifying and promoting “reference site” for good practice of conducting IS and ES activities applying
ITSME approach at school and district levels. (DTE in collaboration with DPPE, PESS, DESB, andschools)
In the evaluation, the Team was impressed to learn many good practices and initiatives made at thefield level for making the best use of ITSME materials and approach.

In this respect, DESB and

PESS with "good practice" should be identified and recognized by MOES as “reference sites” forthe effective external supervision.
As for the school, since many ITSME target schools have been identified as “model schools”, theyshould be further strengthened to be utilized as good “references sites” to disseminate ITSME
approach on the ground.
(6)

Expanding collaboration with trainers of TTC in Lao language
Since the target schools have started to apply ITSME lesson planning approach to Lao language subject,appropriate technical guidance in Lao language might be useful, involving TTC lecturers who are Laolanguage specialists.

3-7

Lessons-learned(1) Effective utilization of the existing framework
As basic policy of the project, the project activities have been planned and implemented in alignment
with the existing framework in Lao PDR, which MOES had developed in order to introduce in-service
training at the school level, namely Internal Supervision. Therefore, these activities were accepted
relatively smoothly by the C/P, and now are becoming as a part of their regular work. This contributed
to the establishment and encouragement of their ownership during the whole period of the Project, andwill contribute to secure the sustainability after the termination of the Project.
(2) Establishment and reinforcement of the partnership at the field level
In implementation of the Project, some of the lecturer from TTC and some of the PAs from
PESS/DESB in the target provinces were assigned as C/P at the filed level, namely ITSME trainer, andformed ITSME trainers’ team. Under the technical support from the Japanese experts, ITSME trainers
have worked collaboratively while sharing their specific knowledge and experience each other, andhave strengthened the unity as a team steadily. The mechanism of this team not only contributed to thesustainable as well as effective implementation of the project activities during the cooperation period
and after the termination of the Project.

xviii

(3) Dissemination and sharing of information on the Project
While the C/P department in MOES is DTE, other departments and organization in MOES such as
DPPE, RIES and ESQAC also have important roles in order to secure the smooth and effective
implementation of the Project, especially in terms of impact and sustainability. However, informationon the progress, outcomes and challenges of the Project have not been shared enough among thedepartments and organization concerned, and also have not been disseminated enough to development
partners concerned for further collaboration. It should be considered that projects disseminate andshare information effectively and efficiently with person, departments and organizations concerned
from an early stage.
(4) Contents of the Baseline/End-line survey
In order to improve teaching and learning at the school level, the Project has been implemented withfocus on activation of IS and ES. Therefore, it should be clarified how IS and ES have been improvedby the intervention of the Project as an evidence on relevance and effectiveness regarding theapproach of the Project. However, data related to IS and ES has not been collected nor analyzed by the
Project to show the evidence because necessary items were not included in the contents of theBaseline/End-line survey. It should be considered that projects specify data/ indicators appropriate tomeasure an effect of intervention at the beginning of its cooperation period, and monitor them
regularly during its cooperation period.

xix