Summery of Final Evaluation
1. Outline of the ProjectCountry: Cambodia

Project Title: Project for Educational Resource
Development in Science and Mathematics at the Lower
Secondary Level (STEPSAM3)

Issue/Sector: Basic Education

Cooperation Scheme: Technical Cooperation

Division in charge:

Total Cost: 520 million JPY

JICA Human Development Department
Period of(R/D): March 27th, 2013

Partner Country’s Implementing Organization:

Cooperation

3 years

Ministry of Education, Youth and Sport (MoEYS)

June 2013- May 2016
RelatedJapanese Cooperation Organization᧶-

The Japanese Grant Aid for Human Resource Development Scholarship (JDS)(Grant Aid)

Cooperation:


JICA Partnership Program in science and mathematics education (Grassroots TC)



Volunteers in science and mathematics education (JOCV/SV)

Project for Expansion of Lower Secondary Schools in Phnom Penh (Grant Aid)

Grant Assistance in education (Grant Aid)

1-1 Background of the Project In Cambodia, normal schools were abolished and intellectuals were killed during the Pol Pot regime
in the late1970s, which destroyed the entire education system. Although the education system hasrecovered to some extent during the rehabilitation period, lack of qualified teachers are still serious
problem in Cambodia. Among all the subjects, low quality of science and mathematics are heavy dragon industrial advances.
JICA implemented its first technical cooperation project, “Secondary School Teacher Training
Project in Science and Mathematics (STEPSAM1)” from 2000 to 2005, aiming at improvement of
pre-service teacher training (PRESET) and in-service teacher training (INSET) for upper secondary
education in science and mathematics. Following that, JICA implemented Science Teacher Education
Project (STEPSAM 2) from 2008 to 2012, focusing on the improvement of science in PRESET inprimary and secondary education. STEPSAM 2 also executed pilot INSET at lower secondary level, andwhich reconfirmed its strong needs. For that purpose, capacity development was required for the
trainers of six Regional Teacher Training Centers (RTTCs).
Thus “the Project for Educational Resource Development in Science and Mathematics at the Lower
Secondary Level” (STEPSAM 3) started in June 2013, aiming to develop Teacher’s Guide and to
improve capacity of trainers at RTTCs who conduct INSET for school teachers to introduce the guides.
1-2 Project Outline(1) Overall Goal
 The educational resources developed by the Project are disseminated to other areas through trainingxi

programmes conducted by MoEYS.
(2) Project Purpose Foundation for MoEYS to support teachers for science and mathematics lesson improvement at thelower secondary level is strengthened.
(3) Outputs
Output 1: Teacher’s Guide for science and mathematics lesson improvement at the lower secondary
level is developed.
Output 2: The capacity of RTTC trainers for science and mathematics lesson improvement at the lower
secondary level is enhanced.
(4) Inputs (at the time of the Final Evaluation)
Japanese Side-

Experts: 12-

Provision of equipment: 1.5 million JPY-

Local costs: 214 million JPY

Cambodian Side-Counterpart personnel assigned to the Project: 115-

Facilities for training sessions and seminars-

Project office and its maintenance fees

2. Final Evaluation Team
Members of

MIZUNO Keiko

LeaderSenior Advisor in education, JICA

the team

TAGUCHI Shimpei

Cooperation Planning

Deputy Director, Basic Education
Team I, Human Development

YAMAGUCHI Yutaka

Evaluation and analysis

Department. JICA
General Manager, Cranberry, Inc.

Period ofApril 18, 2016 – May 9, 2016

Type of Evaluation: Final Evaluation

Evaluation
3. Overview of Evaluation Results
3-1. Project Performance(1) Outputs
Output 1: Teacher’s Guide for science and mathematics lesson improvement at the lower secondary
level is developed.
Indicator 1-1: Developed Teacher’s Guide
Indicator 1-2: Improvement in teaching at the cooperative schools for chapters where Teacher’s
Guide covers.
Result 1 was achieved, since Teacher’s Guide was duly developed and improvement was confirmedby the End-line survey in teaching in science and mathematics at lower secondary schools in thetarget provinces. The Project produced Teacher’s Guide covering 123 lessons in total, which
accounted for nearly 40 % of all the lessons in Grade 7, 8 and 9 mathematics textbooks and nearly
70 % of all the lessons in Grade 7, 8 and 9 science textbooks. Teacher’s Guide was produced byWorking Group (WG) in each subject through 5 “activity cycles of the development” as planned.

xii

RTTC trainers in science and mathematics participated in the revision of Teacher’s Guide,
conducting WG Seminar Meetings. Improvement was noted in teaching at both cooperative schools
for RTTCs and general schools through the End-line survey. The questionnaire survey reported
positive behavior changes of teachers after receiving training for the introduction of Teacher’s
Guide. The survey asked teachers whether they had adopted the ideas and activities introduced in
Teacher’s Guide at their lessons and confirmed whether that behavioral change had occurred after
the training. As a result, 73.2% of teachers reported positive changes in more than 60% of
viewpoints.
Output 2: The capacity of RTTC trainers for science and mathematics lesson improvement at thelower secondary level is enhanced.
Indicator 2-1: Evaluation of RTTC trainers on in-service training
Indicator 2-2: Changes in participants’ attitude toward lesson improvement through in-service
training programmes
Output 2 is regarded to have been achieved, since the surveys conducted in the 5th Teacher’s
Guide Introduction Training confirmed RTTC trainers’ capacity enhancement and positive changes
in participants’ attitude toward lesson improvement. Improvement in the capacity of RTTC trainers
was confirmed in the questionnaire survey, using a self-evaluation check-list to assess 5 aspects; 1)
the level of Subject knowledge, 2) Skills on subject, 3) Motivation, 4) Proficiency in Teacher’s
Guide, and 5) Participant’s knowledge on subject contents, comparing “before the Project” and“after the 5th training sessions”. Improvement was noted in all of the aspects. Through Working
Group Seminar Meeting and continuous training sessions for the introduction of Teacher’s Guide,
RTTC trainers could improve their capacity in the content knowledge, scientific skills and teaching
methods. Clearly positive changes in teachers’ attitude were also noted in a questionnaire survey
implemented by the Project, which recognized the positive changes produced by comparing“before” and “after” the training program in two aspects: 1) motivation to give students additional
knowledge and/or practice on mathematics/science, and 2) motivation to correct errors and mistakes
in the mathematics and science textbooks.
(2) Project Purpose
Foundation for MoEYS to support teachers for science and mathematics lesson improvement at thelower secondary level is strengthened.
Indicator 1: Teacher’s Guide for science and mathematics lesson improvement at the lower
secondary level is approved by MoEYS.
Indicator 2: In-service training contents to introduce Teacher’s Guide to lower secondary scienceand mathematics teachers is approved by MoEYS.
The Project Purpose was achieved, since Teacher’s Guide and the in-service training contents tointroduce Teacher’s Guide are approved by MoEYS. The Consultation Meeting held on 12th
January, 2016 approved the validity of Teacher’s Guide. The Consultation Meeting was led by the
Secretary of State, MoEYS, in which participated the representatives from the Departments relatedxiii

to the Project. Further, on February 1st 2016, MoEYS officially approved the use of Teacher’s
Guide in PRESET training at RTTCs, INSET and teaching at lower secondary schools to improve
the quality of teaching and learning in mathematics and sciences. Having the official approval, in
total 49,862 copies of Teacher Guide are printed and distributed to all of science and mathematics
teachers in the 6 targeted provinces, 6 RTTCs, Teacher Training Department (TTD), Provincial
Office of Education (POE), and WG members. From September 2013 to October 2015, RTTC
trainers conducted 5 cycles of in-service training for lower secondary school teachers in science andmathematics in the 6 target provinces to introduce Teacher’s Guide. In each of the Working Group
meetings held as preparatory sessions for the INSET mentioned above, the Project proposed andgot approval for a training implementation manual and a set of handouts developed by TTD and the
Project Team.
3-2 Summary of Evaluation Results(1) Relevance: High-The Project is in line with the national policies in the field of education, the National Strategic
Development Plan (NSDP) 2014-2018, Education Strategic Plan (ESP) 2014-2018 and Teacher
Policy Action Plan (TPAP).

-

The Project is consistent with the needs of the target groups. Since MoEYS places emphasis on
science and mathematics education in its development plans, RTTC trainers needed to strengthen
further their knowledge and training skills in those subjects. Teachers of lower secondary schools
also needed to improve their subject knowledge and teaching skills. To respond to their needs, there
was a shortage in training and educational resources including Teacher’s Guide.

-

Project’s approaches have proved to be effective and efficient in order to support teachers forscience and mathematics lesson improvement. The Project aimed to develop Teacher’s Guide inconformity with school textbooks and to provide training with all the 6 RTTC trainers to conduct
INSET in the provinces where RTTCs are located.

-

The Project is in line with Japan’s cooperation policy, constituting one of the most important
projects in the “Program for Improving Science and Mathematics Education” in “Rolling Plan for
the Royal Government of Cambodia of April 2015”.

(2) Effectiveness: High-The Project achieved its project purpose, as MoEYS approved the use of Teacher’s Guide,
confirming the validity of the contents and the INSET programmes. Teacher’s Guidebook was
produced through the collaboration among Working Group members, which enhanced the quality.
WG successfully produced Teacher’ Guide and conducted INSET for the introduction.

-

Teachers highly appreciate detailed and clear explanation in Teacher’s Guide, which was noted bymany teachers interviewed during the Final Evaluation.

-

Through the process of development Teacher’s Guide and INSET for the introduction, RTTC
trainers improved their capacity of conducting INSET in sciences and mathematics using Teacher’
Guide. Meanwhile, their level of understanding on the contents of Teacher’s Guide still needs to bexiv

further improved.
-

Teachers recognized the importance of “teaching based on conducting activities”, rather than“letting students memorize” with simple experiments and activities related to daily life. They
became better in using materials around to produce simple experimental materials through theparticipation in INSET and through the use of Teacher’s Guide, according to remarks made byschool teachers interviewed.

-

Experiments introduced in Teacher’s Guide are very simple and require only materials locallyavailable. Teacher’s Guide provides teachers with improvised, more applicable and more
implementable experiments with clear explanation and pictures. However, depending on thesituation of schools, there are some experiments which are difficult to prepare and conduct becauseof time and funding constraints at school level.

(3) Efficiency: High-The Project conducted efficiently all the 5 cycles of training sessions planned for the introduction ofTeacher’s Guide, in which participated almost all the trainers of 6 RTTCs , and the participation
rate of school teachers in science and mathematics in the target provinces were very high.

-

The training contents were concentrated in limited time allocated to training sessions. According to
RTTC trainers interviewed, sometimes they had difficulty in completing all the activities planned to
be implemented in training sessions. This shortage of time was caused partly due to the increased
number of lessons covered by Teacher’s Guide.

-

The Project functioned well by utilizing the achievement of past projects, which highly enhanced
efficiency in the implementation of the Project. The Project employed efficiently humane resources
strengthened and educational concepts introduced by STEPSAM 1 and 2 projects.

-

Coordination among development partners has been beneficial in the project implementation. The
Project has been coordinating with EDSP3, VSO and VVOB, which have been active in supporting
teacher training in science or mathematics.

(4) Impact: Relatively High-The Overall Goal will be achieved. MoEYs will disseminate Teacher’s Guide all the non-target 19
provinces in addition to the 6 provinces covered by the Project. MoEYS will print and distribute
Teacher’s Guide and conduct training in those provinces.

-

Teacher’s Guide will be used in other training programs together with INSET. MoEYS will
integrate Teacher’s Guide into the Upgrade Training for primary teachers to be able to teach atlower secondary schools.

-

Teacher’s Guide were used as one of the reference documents for teachers to prepare lesson plans tostrengthen mathematics and science education through piloting INSET by ESDP 3.

-

Teacher’s Guide can be used as one of the resource books for teachers to implement lesson study inscience and mathematics. TTD plans to train secondary teachers on lesson study.

-

Teacher’s understanding on subject contents was significantly improved, according to theself-evaluation surveys conducted by the Project. However, the results of end-line survey showedxvthat teachers’ understanding yet reached at satisfactory level and students’ understanding was also
still limited.
-

Simple experiments introduced in Teacher’s Guide stimulate students’ interests during lesson,according to remarks made by school teachers interviewed and in the questionnaire surveys by the
Project. Efforts to build this kind of motivation promote students’ learning in science andmathematics, which may attract them to further study in these subjects.

(5) Sustainability: Relatively High-Even after the end of the Project, the policies, ESP 2014-2018 and TPAP 2015 are expected tocontinue. MoEYS recently places more emphasis on STEM (science, technology, engineering andmathematics), which is needed to transform Cambodian industrial structure from a labor-intensive
to a skill-based industry envisaged by “Cambodia Industrial Development Policy 2015-2025”.

-

In the system of PRESET and INSET, substantial institutional changes are being planned according
to TPAP. Teacher’s Guide will be useful and is expected to be used continuously, as MoEYS
approved the use of it for both PRESET and INSET.

-

MoEYS will provide significant funding to distribute Teacher’s Guide and to implement its
introduction training in the 19 provinces not covered by the Project. MoEYS intends to use thebudget in the strategic budget plan 2017 – 2019 for that purpose. This will be a great contribution toteachers in science and mathematics in the provinces, who will benefit from Teacher’s Guide.
Financial constraints may continue with regard to implementation of regular INSET.

-

Teacher’s Guide will be a very useful complementary teaching material to help teachers in making
better use of the current text books in science and mathematics for grade 7 to 9, until MoEYS
produces new textbooks under a reformed curriculum. In addition, teacher’s Guide will bebeneficial to teachers, even after new textbooks are introduced, as it contains useful additional
knowledge to enrich teacher’s interest and ability both in science and mathematics, in addition to
attractive simple experiments in science.

3-3. Factors that promoted realization of effects(1) Factors concerning to Planning-

The Project held School Director Meeting 4 times since December 2014, in addition to ordinary
School Director Meeting organized periodically by POE and Municipality Department of Education(MDE). Since school directors play key roles at school level, School Director Meetings was aneffective measure to promote participation in training sessions and application of Teacher’s Guide
by school teachers.

(2) Factors concerning to the Implementation Process-

Teacher’s Guidebook was produced through the collaboration among Working Group members, inwhich participated representatives from related departments, National Institute for Education (NIE)
trainers, RTTC trainers and Japanese experts. In this process, drafts were shared and reviewed by
the Working Group members, which further enhanced the quality of the Teacher’s Guide byxvi

localizing well the contents.
-

Implementation of seminars and training sessions with high participation rates was made possible,
by the collaborative efforts with representatives from TTD, POE and MDE. They made a lot ofcontribution to facilitating participation of trainers and teachers, who are sometimes busy and notmuch motivated in participating in seminars nor training, sacrificing their time.

3-4. Factors that inhibited realization of effects(1) Factors concerning to Planning  N/A(2) Factors concerning to the Implementation Process-

In addition to inspection system by POE and DOE, MoEYS is in the process of strengthening thesystem of inspection and monitoring at district level including District Training and Monitoring
Team (DTMT). Still it may take time for MoEYS to establish an effective national inspection andmonitoring system to support the implementation of training to increase the capacity of teachers.

3-5. Conclusion
Based on the results of the evaluation, the Team concludes that the Project successfully achieved its
project purpose.

MoEYS approved the validity of Teacher’s Guide, approving the use of it in PRESET

training at RTTCs, INSET and teaching at lower secondary level. Teacher’s Guidebook was produced
through a good collaborative process among Working Group members and the Japanese experts,
enhancing its quality and localization, which improved capacity of RTTC trainers and teaching at
schools. Teacher’s Guide was produced efficiently covering lessons more than originally planned,
nearly 40 % of all the lessons of the textbooks in mathematics and nearly 70 % of all the lessons inscience subjects. The Teacher’s Guide were printed and distributed to all of the teachers in science andmathematics in the 6 targeted provinces.
Further, Teacher’s Guide will be disseminated by MoEYS to all the 19 provinces in addition to the 6
provinces already covered by the Project. Teacher’s Guide can be also used in different types of training
such as upgrading program, PRESET, and promotion of lesson study and lesson plan. Althoughteachers’ and students’ understanding is still limited on science and mathematics knowledge according
to the Project’s surveys, its continuous use and reinforced monitoring and inspection are expected tocontribute to improve their understanding. Teacher’s Guide are be very useful as complementaryteaching material for the current textbooks. Even after new textbooks are introduced, Teacher’s Guide
will still be useful, as it contains additional knowledge to enrich teacher’s interest and ability, in
addition to attractive simple experiments in science.
3-6. Recommendations(1) One of the two outputs of the Project is the enhancement of the capacity of RTTC trainers, and it
was expected that RTTC trainers strengthen their capacity through participating in Working Group
Meetings for Teacher’s Guide development, and actually conducting INSET on Teacher’s Guide.
However, the results of assessment conducted by the Project indicate that the level of RTTC trainers’
understanding on the contents of Teacher’s Guide is significantly low across all mathematics andxvii

science subjects. In this respect, RTTC trainers should closely review their own test results, and identify
and reflect their weaknesses in content knowledge, skills and teaching method.

It is also

recommended that trainers should overcome their weaknesses through actual practice and utilization ofTeacher’s Guide. It is encouraged that the trainers should support and learn from each other byorganizing Lesson Study on a regular basis.
(2) Teacher’s Guide has been officially approved by MoEYS for its utilization in PRESET and INSET,
and teaching at lower secondary schools. It is commendable that MoEYS has allocated the budget for its
dissemination to non-target provinces accompanied with orientation training. In this respect, adissemination Plan should be prepared by TTD including how the orientation WS should be provided.
Adequate guidance and instruction for Directors on the utilization of Teacher’ Guide in school-based
INSET should be part of the plan. It is also recommended that TTD prepare implementation guidelines
for RTTCs and also lower secondary schools, which briefly explain how Teacher’s Guide should be
utilized in INSET and PRESET, and school based training in a concrete manner.

Distributing

Teacher’ Guide accompanied by such guideline will facilitate effective and systematic application of
the Teacher’ Guide for quality improvement. Since the allocated budget should be limited, alreadyavailable mechanisms or opportunities should be maximized. Such mechanisms or opportunities include
Directors’ Meetings, Technical Group Meetings, Internal and External Supervisions, and Inspection.
(3) It is expected that the more a teacher utilizes Teacher’s Guide for lesson improvement, the easierand enjoyable the lesson can be, and thus the students’ learning is expected to be promoted.

Teacher’s

Guide can be effectively utilized in the process of lesson study as part of school-based INSET. The
Team suggests that TTD should plan with relevant stakeholders including at district and provincial
levels to revitalize lesson study for school based INSET where Teacher’s Guide can be actively utilizedby participants. This should be part of dissemination strategy, and again, principals play a key role inleading and supporting its implementation at school level.
(4) It is recommended that the contents of Teacher’s Guide should be taken into account when preparing
national/semester assessment so as to strengthen coherence between assessment and utilization ofTeacher’s Guide.

By doing so, teachers should become more motivated to utilize Teacher’s Guide in

their classroom teaching and to change their emphasis in the classroom from “memorization” to“problem solving”.
(5) It is important that teachers and principals should be continuously encouraged to utilize Teacher’s
Guide to further enhance their teaching quality.

In this respect, it is recommended that utilization ofTeacher’s Guide in their regular teaching activities and school based INSET should be included as acheck item for regular school monitoring.
(6) It is advisable to maximally utilize the human resources developed through the Project, particularly
RTTC trainers, in the ministry’s priority task relating to science and mathematics education.

xviii

3-7. Lessons Learned(1) Without ensuring quality learning at primary level, quality improvement interventions at higher
level such as STEPSAM3 could not bring much impact on learning outcomes as demonstrated by the
results of End-line survey. Students should be adequately equipped with foundational skills, knowledge
and attitude to learn science and mathematics from primary education in order to advance their learning
at higher levels leading to STEM.
(2) Skills, knowledge and resources accumulated by a series of technical cooperation projects (Lesson
Study, Inquiry Based Learning, Teachers’ Guide, etc.) from the past to the present are appropriately
mobilized for improving mathematics and science education in Cambodia in a coordinated manneralong with a clear policy direction and commitment. The continuity of core human resources of
MOEYS over different projects (STEPSAM 1 through STEPSAM3) may have facilitated theconsolidation of accumulated resources and experiences in an efficient and effective manner.
(3) Combining international (Japanese) and local subject experts in technical assistance for Teachers’
Guide development was effective so that the Japanese standards be appropriately adapted to local
context. Local experts translated the draft materials into Khumer and facilitated the discussions on thematerials in a working group meeting where stakeholders of different departments and institutes of
MOEYS, POE and RTTC have participated. This process has also contributed to nurturing ownership ofCambodian counterparts as well as deepening their understanding on the contents of the materials.
(4) Appropriate orientation for school principal is indispensable to introduce and disseminate newteaching references such as Teachers’ Guide for improving lessons. Adequate understanding of school
principals on the usefulness of Teachers’ Guide was essential to promote an enabling environment for
the introduction and effective utilization of Teachers’ Guide at school level.

xix