Summary of the Evaluation
I. Outline of the ProjectCountry:

MongolSector: Basic Education

Project Title: Project for Strengthening Systems for
Improving and Disseminating Child-Centered Teaching
Methods
Cooperation Scheme: Technical Cooperation Project
Total cost (as of the time of evaluation):
280 million yen

Divisionin Charge:

Cooperation
PeriodJICA Mongol Office(R/D) December 8, 2009

Implementing organization in Mongol:
Ministry of Education and Science (MES)*Ministry of Education, Culture and Science (MECS)
renamed as MES since Sep. 2012.
Organization in Japan: Koei Research Institute (KRI),Tokyo Gakugei University

Three years (March 2010-February 2013) and extended for six months (March to August,2013)

II. Evaluation Team

Members of
the evaluation teamEvaluation Period<Japanese Side>
1.	Team Leader:Mr.Satoru Takahashi, JICA Senior Advisor
2.	Education Planning: Mr. Takeshi Matsuyama, JICA Human Development Dept.
3.	Evaluation Planning 1: Ms. Moeko Imayoshi, JICA Mongolia Office
4.	Evaluation Planning 2: Ms. Ch. Oyuntsetseg, JICA Mongolia Office
5.	Evaluation and Analysis: Ms. Chie Tsubone, Global Link Management, Inc.
<Mongolian Side>
6.	Leader: A. Tuya, Head, Primary and Secondary Education Division, MES
7.	Member: J. Narantuya, Officer, Strategy and Policy Planning Department, MES
8.	Member: N. Oyuntsetseg, Professor, Chemistry Department, National Universityof Mongolia
2012/10/1 - 2012/10/18

Terminal Evaluation

III. Overview of Evaluation Results
3-1 Achievements of the Project(1) Outputs
Output 1:	Capacities of “the District/Aimag Teams”* in all District/Aimags to disseminate “the Teaching
Methods” are enhanced.
Indicator 1

Training participants increase their satisfaction and understanding.

Indicator 2

Training participants’ behavior is positively changed.

Indicator 3“District/Aimag Team” members attend the training based on the training package (at least
8 participants) .

xi

Output 1 has been mostly achieved. Indicator 1 shows that the average satisfaction level of training
participants in 2011 was high, which was more than 4 out of 5-sclae evaluation while understanding levelon the Teaching Methods was relatively low; 39.6% of teachers and 14.9% of school managers answered
correctly to questions regarding the Teaching Methods. Because recent data on the levels of satisfaction andunderstanding is not available, the level of improvement could not be analyzed quantitatively. However,teachers and specialists of DECs interviewed by the evaluation mission team acknowledged that theirunderstanding on the Teaching Methods was improved through the Project. Regarding Indicator 2, Lesson
Study was implemented by 64% of all the schools in the country after training, which indicates relatively
high level of behavioral changes because implementation of Lesson Study nationwide is the indicator used
for the Overall Goal. Indicator 3 has been fully achieved.
Output 2: Models of “Lesson Study” are developed in model District/Aimags.
Indicator 1

Model schools develop “Lesson Study Implementation Plan”.

Indicator 2

Model schools conduct “Lesson Study” at least twice a year in the eight subjects.

Indicator 3

Quality of “Lesson Study” at model schools is improved.

Indicator 4

Satisfaction level of participants in “Lesson Study” at model schools is increased.

Output 2 has been mostly achieved. Indicator 1 has been achieved. Regarding Indicator 2, compared to thedata of 2010/2011, the frequency of implementation of Lesson Study in 2011/2012 was increased though it
was slightly below the objective. In the academic year 2011/2012, 57% of model schools conducted Lesson
Study more than twice in 8 subjects, and 93% conducted it more than twice in 6 subjects. About Indicator
3, though quantitative data was not available, alternative qualitative information indicating improvement
was obtained. Therefore, it was mostly achieved although there is still room for improvement. Indicator 4shows that the satisfaction level of participants in Lesson Study in year 2012 was increased compared tothat of year 2010.
Output 3:	Capacities of schools in model District/Aimags to practice “the Teaching Methods” are enhanced.
Indicator 1

70% of teachers and management posts of all schools in model District/Aimags’ complete
the training on “the Teaching Methods” based on the training package.

Indicator 2

80% of all schools in model District/Aimags formulate “Lesson Study implementation plan”.

Indicator 3

Lessons learnt and recommendation on teaching methods improvement from model District/
Aimags are collected.

Output 3 has been mostly achieved. Regarding Indicator 1, more than 70% of teachers and school
managers completed training in each model District/Aimag. About Indicator 2, more than 80% of model
schools developed a Lesson Study implementation plan. Regarding Indicator 3, a management handbook todisseminate the Teaching Methods, which includes lessons learnt and recommendations collected throughthe implementation of the Project, is being developed and will be finalized in December 2012.
Output 4:	The environment to disseminate and establish “the Teaching Methods” in PRESET is improved.
Indicator 1” The Teaching Methods” in the training package is introduced to PRESET.

Indicator 2

Policy recommendation is adopted by Ministry of Education, Culture and Science (MECS)
in its annual policy objectives.

xii

Output 4 has been mostly achieved. Lesson Study as a tool to disseminate the Teaching Methods wasincorporated into the curricula of Faculty of Chemistry and Faculty of Physics at National University ofMongolia, and it is also being introduced in Mongolia State University of Education. On the other hand,
Indicator 2 was not appropriate to show the achievement of Output4 though the indicator was achieved.
(2) Project Purpose
Project Purpose: Systems to disseminate “the Teaching Methods” nationwide are strengthened.
Indicator 1

The quality lessons utilizing the Teaching Methods in the 8 subjects are practiced in model
schools.

Indicator 2“Lesson Study” is conducted at least twice in at least the 70% of schools in model District/
Aimag every year.

Indicator 3

All Districts/Aimags formulates the training plan on “the Teaching Methods”.

Indicator 4

Political, financial and human resources commitment are made by MECS.

The Project Purpose has been mostly achieved. Regarding Indicator 1, according to the results ofmonitoring by the project team, though most of the subjects scored more than three out of five-scaleevaluation, there is room for improvements in some subjects in certain District/Aimags. Indicator 2 and 3
have been achieved. Indicator 4 has been mostly achieved except for development of an action plan by MES
to disseminate the Teaching Methods.
3-2 Evaluation by Five Criteria
Criteria

Relevance

Evaluation Results▪	Relevance of the Project is moderately high.
▪	"Comprehensive National Development Strategy" (2007-2012), Mongolian national
development plan, and "Educational Master Plan (2006-2015)" place emphasis onteacher development as a key strategy for educational development. According toJapan’s country assistance policy for Mongolia, the Project corresponds to one of theDevelopment Issues, “Improvement of Basic Social Services”, targeted by the Priority
Area “Assisting Inclusive Growth”. Thus, the Project is consistent with the policies ofboth countries.
▪	DECs and schools had been struggling to implement the Teaching Methods, andneeded technical supports. Therefore, the Project, which aims to strengthen systems todisseminate the methods, caters to the Mongolian needs.
▪	Capacities of teachers were strengthened through Lesson Study, which has contributed
to enhanced teamwork among teachers, effective use of blackboard, development oflesson plan focusing more on children, and particularly creation of a culture of criticalcollaborative inquiry in school. Capacities of Professional Team and DECs were also
strengthened to observe lessons and provide appropriate advice and comments toteachers. Such evidence and commitments show that the approach is appropriate thatthe foundation for disseminating the Teaching Methods has been consolidated.

xiii▪	After the mid-term review, circumstance of the project changed that INSET is under
the responsibility of IUTP, which is a newly established organization resulted frompolitical reformation.

Effectiveness▪	Effectiveness of the Project is medium.
▪	The Project Purpose has been mostly achieved with remaining issues regarding qualityof lessons and development of action plan on dissemination of the Teaching Methods
by IUTP.
▪	The outputs are mostly achieved that teachers, school managers, DEC staff andProfessional Team members acknowledge that their capacities were developed throughthe Project.
▪	The approach to disseminate the Teaching Methods through Lesson Study waseffective to strengthen capacities of teachers in short period of time. Moreover, as it
is an approach to promote teamwork, capacities were enhanced not only at individuallevel but also at school level.
▪	Coordination with other interventions such as Social Sector Support Program andJapan Overseas Cooperation Volunteer raised the efficiency of the Project.
▪	Although qualitative data gathered sufficiently but quantitative data to show
improvements at teacher and student levels was not collected by the Project sufficiently.

Efficiency▪	Efficiency of the Project is high.
▪	Lesson Study is a practical and cost-efficient training that it does not require special
venue and traveling fees, and produces results in short period of time.
▪	MES speared certain amount of budget for the Project and DECs and schools also
conducted related activities utilizing their own resources.
▪	Outputs of the Phase I of the Project, which are teachers’ guides and Professional Team
members, were optimized.

Impact▪	Impact of the Project is relatively high.
▪	Overall goal is presumed to be achieved that one of the two indicators has alreadyachieved. Besides, though the other indicator is not fully achieved, yet 476 schoolsout of all the 747 schools (64%) in the country conducted Lesson Study at least once
during the project period.
▪	Positive impacts at student level are observed by teachers such as more active
participation in class, and higher motivation to study. In addition, related training
and other activities were conducted in model and non-model areas utilizing their own
resources; many model and non-model schools conduct Lesson Study not only in eightsubject targeted by the Project, but also other subjects such as Mongolian Language
and History; one of the Professional Team members will participate in an international
conference and make a poster presentation on the case study of Lesson Study.

xivSustainability▪	Sustainability of the Project is medium.
▪	It is likely that dissemination of the Teaching Methods will be incorporated in the
education reform, which will be conducted from 2012 to 2016. However, because
IUTP is a new organization, recruitment of staff is still ongoing, and the details of theplan have not been determined yet.
▪	Specific budget has not been allocated to disseminate the Teaching Methods after theend of the Project. However, MES stated that it would secure necessary budget.
▪	The current teacher evaluation system does not incorporate teacher’s practice on the
Teaching Methods.
▪	Capacities of MES, Districts/Aimags and schools were developed through the Project.
However, there is room for improvement in quality of lessons and Lesson Study, andcontinuous technical supports by MES and PT will be necessary.

3-3 Factors Contributing in the Production of Effect(1) Planning
Relatively longer duration of supports since the first phase
The foundation to disseminate the Teaching Methods was laid by the first phase. Especially, theProfessional Team became able to make proper comments and advice in Lesson Studies enable the Project
to carry out activities of Phase II without significant difficulties. Moreover, good relationship with the
Mongolian side developed during the first phase enabled the Project to operate smoothly.
Introduction of practical Lesson Study approachLesson Study functioned as a practical tool to practice the Teaching Methods for teachers. Capacities
of teachers were strengthened through Lesson Study, which has contributed to enhanced teamwork
among teachers, effective use of blackboard, development of lesson plan focusing more on children, andparticularly creation of a culture of critical collaborative inquiry in school. Capacities of Professional Teamand DECs were also strengthened to observe lessons and provide appropriate advice and comments toteachers. Lesson Study can also involve other schools easily by inviting them as observers. This nature of
Lesson Study allows quick dissemination of the approach. .
(2) Implementation Process
High level of ownership at the Mongolian side
MES secured budget for the Project. DEC and schools also conducted activities utilizing their own
resources. In addition, MES stated that they will work out a concrete plan for continuous professional
development of teachers in Mongolia, utilizing IUTP as a core implementing agency. Such evidence andcommitments show that the foundation for disseminating the Teaching Methods has been consolidated.
Availability of teachers’ time to engage in Lesson Study
Teachers in Mongol are required to stay at school to engage in school affairs for certain hours even after
they finished teaching all the classes of the day. Therefore, they had enough time to engage in activities of
the Project, otherwise they would have engaged in second job outside the school.

xv

3-4 Problems and Factors that Raised Problems(1) Planning
Insufficient teachers’ understanding regarding contents of the Teaching Methods
Although teachers learned how to conduct Lesson Study, their understanding level on child-centeredteaching methods is still not sufficient. Therefore, the Project tried to accompany Professional Team member
every time they conduct Lesson Study, in order to show what the appropriate advice for the teachers topractice child-centered teaching is.
Insufficient teachers’ knowledge regarding contents of subjects
It was indicated by JICA experts that teachers’ knowledge regarding subjects is not sufficient, and it is
one of the constraining factors to improve quality of lessons. Thus, the project requested every school toprepare Lesson Study by team including school managers. Besides, the Project carried out study session onsome subjects （math and natural science) how these subjects are taught in Japan, in order for teachers tounderstand more about the objectives teach the subjects.
(2) Implementation Process
Load on Professional Team members and model schools to disseminate Lesson Study
Professional Team members mentioned that allowance for monitoring visit is not sufficient, and conditionof accommodation is difficult. Also, schools need to adjust schedule to provide training for non-model
schools. Some schools mentioned that they do not have sufficient budget to conduct training for non-model
schools. After the Project requested MES, travel allowance has been allocated to participate in the training
or monitoring in non-model Aimags/Districts and non-model schools.
3-5 Conclusion
Through the careful examination of outputs and indexes and other impacts attained, the Project Purpose
has been mostly achieved. The Project has high relevance that is aligns with the policy of both Mongolian
and Japanese side. Although the Project effectiveness is medium due to the INSET system renovation anduncertainness of the Mongolian side’s implementation body, yet without these external factors, the methodof Lesson Study is effective to achieve the Project purpose. It could be confirmed through the unexpected
positive impacts generated from the Project. For example, the Lesson Study activity has expanded to nontargeted areas and subjects. Currently the model and non-model schools conduct Lesson Study not only on
the eight target subjects, but also other subjects. There are also positive impacts on students such as active
participation in class, higher motivation to study, and so on. Some of DECs in non-model Districts/Aimags
conduct training and Lesson Study with their own initiatives and resources. It clearly shows that a part of theoverall goal has achieved.
On the other hand, understanding about the Teaching Methods and Lesson Study in non-model Districts/
Aimags is not as high as that of the model District/Aimags. Even model District/Aimags need to deepen
understanding about the Teaching Methods through Lesson Study. And the concrete action plan fordissemination of the Teaching Methods needs to be developed by IUTP in close collaboration with MES. Thisimplies that sustainability of the project from financial and institutional aspects should be ensured.
In this regard, the Mongolian side requested that JICA extend the Project for a certain period of time tostrengthen the capacity of disseminating the Teaching Methods. MES stated that Professional Team will, basedon the mutual contract, cooperate with IUTP in developing capacities of specialists in IUTP, implementing the
Lesson Study, and developing the training module for dissemination of the Teaching Methods. MES considers
that the dissemination of the Teaching Methods will be included as one of the key activities of IUTP andxvi

JICA’s technical support to develop the dissemination plan will be necessary. With the thorough considerationon the results of the evaluation, a short period of extension will be reasonably necessary that it leads to asustainability of the Project and further positive impacts by expanding to non-model areas.
IV. Recommendations(1) Utilizing Professional Team and Specialists in DECs
In order to continuously enhance the quality of lesson and Lesson Study, it is essential to have external
observers （Professional Team and specialists in DECs) other than school teachers to provide advice andcomments. The following measures are expected to be taken.
(a) Professional Team
It is recommended that the mechanism be established to enable PT to visit schools in Districts/Aimags
and attend teleconferences connected to those sites. In addition, in order to encourage PT members（mostly university professors) , it is recommended to strengthen linkage their research work with
Lesson Study activities. For example, it would be effective to encourage professors to attend international
conferences on Lesson Study, or to establish academic society on Lesson Study in Mongolia.
(b) Specialists in DECs
Specialists in DECs have an important role to support schools to enhance the quality of Lesson Study
because they are supposed to closely monitor schools. It is recommended that: 1) including monitoring
work for Lesson Study in their professional responsibilities; 2) appointment of capable persons andtraining on the Teaching Methods; and 3) allocation of budgets for monitoring schools.
(2) Considering Teacher’s Engagement in Lesson Study in their Career Path
MES stated that the evaluation system for teachers will be improved for focusing more on the performanceof teachers and students. As one of items of personnel assessment, teachers who are engaged in Lesson
Study should be positively evaluated in their career development.
V. Lesson Learned(1) Involving Faculty Members of Universities
Not only DECs and schools, university professors are involved as Professional Team in the project
activities. Those human resources with academic expertise are valuable to enhance the quality of Lesson
Study. Also, such involvement contributes to introducing the Teaching Methods to PRESET in universities.
(2) Realizing Self-efficacy of Teachers at the Early Stage of the ProjectThrough implementing Lesson Study, teachers have recognized various impacts on students andteachers themselves. Those successful experiences led to their self-efficacy, encouraging them to be more
actively involved in Lesson Study. So it is vital for the target group to feel the self-efficacy and confirm theeffectiveness of the Project at the early stage.
(3) Reaching Out to Non-model Areas for Nationwide Dissemination
In addition to intervention to model areas, some extent of intervention such as training and monitoringon Lesson Study has been implemented for non-model areas. It prompted some non-model DECs and nonmodel schools to implement and disseminate Lesson Study with their own initiatives and resources during
the project period. With the achievement of the Overall Goal in perspective, it would be effective to includexviisome extent of intervention in non-model areas as well in the original project design.
(4) Setting Indicators to Measure Attitude Changes of Teachers and Students
In order to show clear results of education projects, it would be effective and persuasive to set indicators
on how attitudes of teachers and students are changed before and after intervention.

xviii