Summary of Evaluation Results
I.

Outline of the ProjectCountry: Mongolia

Project Title: Teaching Methods Improvement Project
towards Children’s Development in Mongolia

Issue/Sector: Education (Teaching Methods

Cooperation Scheme: Technical cooperation

Improvement)
Division in Charge: Basic Education Division I,
Basic

Education

Group,

Human

Total cost: about 290,000,000(YEN)

Development

Department, JICA

Period ofApril 2006 – July 2009

Partner Country’s Implementing Organization:(3 years 4 months)

Ministry of Education, Culture and Science (MECS),

Cooperation

Elementary Education Improvement Center, Mathematics
Education Improvement Center, IT Education
Improvement Center, and Science Education
Improvement Center (the four Teaching Methods
Improvement Centers), and Institute of Education

1.

Background of the Project

The Government of Mongolia has introduced the new education standards in September 2005 within theframework of education sector reform. The new education standards focused on shifting from a 10-year basic
education system to a 12-year education system, lowering the school entry age from eight to six, and introducing
new subjects such as integrated learning, natural science (integrated science), and others. In accordance with the
new education standards, the teaching methods have been also expected to change from conventional
teacher-centered to student-centered ones. However, at the school level, it has been hard to implement the new
education standards because their contents are too academic for teachers to put them into practice in theclassroom. To address these challenges, the Government of Mongolia requested the Government of Japan for atechnical assistance project to improve teaching methods.

2.

Project Overview

The Project has been implemented by MECS in cooperation with the Institute of Education, the four Teaching
Methods Improvement Centers, the Department of Education Culture (DEC) of Ulaanbaatar (UB) City, Selenge
Aimag and Dornod Aimag and nine model schools to shift from conventional rote learning methods to the newteaching methods that allow children to construct knowledge by themselves. More specifically, the Project hasdeveloped the teacher’s guidebooks in eight subjects, namely elementary science, general science, chemistry,
physics, arithmetic, mathematics, IT education, and integrated learning every year for the three-year project
period. The first-year draft teacher’s guidebooks were employed and examined in trial lessons in three model
schools of UB City. The second- and the third-year draft teacher’s guidebooks were employed and revised
through trial lessons and lessons analyses in all nine model schools in UB City, Selenge Aimag and Dornod
Aimag. The expected outcomes of the Project are shown below.

(1) Super Goal: Teaching methods which support children’s developmenťthe Teaching Methods” aredisseminated in the country.
(2)Overall Goal: Teaching methods which support children's development (“the Teaching Methods”) aredisseminated in model Aimags and City.
(3)Project Purpose: The Teaching Methods are developed in accordance with the new educational standards
introduced in primary and lower secondary education.
(4) Outputs
Output 1: The Teaching Methods are studied and developed in Mongolian context.
Output 2: Developed Teaching Methods are examined by Aimag/City supervisors and teachers, so that theteaching methods are applicable in local school conditions.
Output 3: The examined Teaching Methods are improved through trial lessons in model schools, so that theteaching methods are more applicable in local school conditions.
Output 4: Monitoring model is developed and practiced to see introduction and continuous practice of theteaching methods.
(5) Inputs
Japanese Side:
Expert: 12 people in the following six professional fields: 1) Project Manager/Education Planning; 2) Science
Education; 3) Arithmetic/Mathematics; 4) Integrated Learning; 5) IT Education; and 6) ProjectManagement and Monitoring.
Equipment: 5.46 million yen for such items as digital cameras, digital video cameras, and computers
Operational Cost: 7.54 million yen for training in Mongolia

Number of Trainees Received: 18

Mongolian Side:
Main Counterparts: 19 people

Members of Working Groups: 57 people

Operational Cost: 11.63 million yen, i.e., 176 million Tg, was allocated for the budget for developing andprinting the Teacher’s Guidebooks, and operational costs for working groups. Due to the lack of therelevant information and data, it was not confirmed how much of the operational costs that the Mongolianside actually bore.
Land and Facilities: Office space provided
II.

Evaluation Team

Members

Mr. Atsushi MATACHITeam Leader

Senior Advisor (Education) JICA

ofMs. Sumiko ASANO

Cooperation Planning 1

Basic Education Division I, Basic

Evaluation

Education Group, HumanTeam

Development Department, JICA
Mr. Kiyotaka MIYAZAKI

Cooperation Planning 2

Assistant Resident Representative,
JICA Mongolia Office

Ms. Purevsuren ENKHZAYA

Cooperation Planning 3

Program Officer of Education, JICA
Mongolia Office

Ms. Toshiko SHIMADA

Evaluation Analysis

Consultant, IC Net Limited

Eval. Period March 2, 2009 – March 20, 2009

Type of Evaluation: Final Evaluation

III. Results of Evaluation
3-1 Confirmation of Results
Overall, most of the four Outputs have been achieved or almost achieved, which will contribute to the attainmentof the Project Purpose. The university teachers took the lead in developing the draft Teacher's Guidebooks foreight subjects (Output1). These draft Teacher's Guidebooks were revised by working groups, Model Aimag/City
supervisors and model school teachers through various workshops and training (Output 2). In model schools, trial
lessons were conducted and monitored (Outputs 3 and 4). After trial lessons, lesson analyses were undertaken inwhich project stakeholders discussed and analyzed the Teaching Methods and the draft Teacher's Guidebooks.
The comments of the model schools were finally incorporated into the Guidebooks to be applicable in all schools.
In this way, the achievement of the Outputs 1 to 4 has contributed to the achievement of the Project Purpose, i.e.,the development of the Teacher's Guidebooks in accordance with the new education standards and the needs ofteachers and schools.

3-2 Summary of Evaluation Results(1) Relevance: Very high
The Project aims to break away from the conventional rote learning methods and develop the new teaching
methods that allow children to construct knowledge by themselves. Thus it matches the needs and priorities of
MECS in accordance with the new education standards of 2005. Since the Project is expected to contribute toenhancement of the quality of education through the development of the new teaching methods, it is also
consistent with the 2006-2015 Master Plan to Develop Education of Mongolia that aims to improve the access to
educational service and the quality of education. Furthermore, the Project has addressed the needs of teachers inmodel Aimags/City by developing the Teacher’s Guidebooks that describe the details of the Teaching Methods tobe employed in lessons. According to the Japanese Government’s Country Assistance Program for Mongolia of
2004 and JICA’s Plan for Country-specific Implementation Program of 2006, human resource development thatmatches the market-oriented economic reform is one of the four priority areas for assistance. The programs above
highlight the necessity of assistance for basic education. The Project, therefore, is consistent with the Japanese aid
policies.
In its approaches, the Project put an emphasis on the process of development of the Teacher’s Guidebooks inwhich three different actors, namely university teachers, Aimag/City supervisors, and model schools worked
collaboratively. This approach seems appropriate since it helped them acquire practical skills and knowledge anddevelop the Teaching Methods that would meet the needs of schools. Since the nine model schools were selectedfrom urban and rural areas in two Aimags and UB City, and also from well-equipped complex schools andnon-well-equipped ones in urban areas to develop the Teaching Methods, the Teacher’s Guidebooks are expected
to be applicable to all schools in Mongolia. Thus the Project as a whole has a high degree of relevance fortechnical cooperation.

(2) Effectiveness: HighAll the four Outputs have been almost achieved, which is expected to contribute to the attainment of the ProjectPurpose by the end of the Project period. The cooperation of the three actors who had learned from one another topursue their common goals of the development of the Teacher’s Guidebooks contributed to enhancing theeffectiveness of the Project. Another contributing factor is that the Japanese experts with extensive experience inthe field of education imparted the new skills and knowledge of the Teaching Methods to the counterparts. Thishas greatly raised the morale of the counterparts to be involved in the Project activities and make the maximum
use of learning opportunities in the Project. Therefore, it is fair to say that the effectiveness of the Project is high.

(3) Efficiency: High
Although some of the inputs such as transportation costs to attend trial lessons were not provided by the
Mongolian side as agreed, other inputs from both sides have been provided as planned. Thus the majority ofactivities has been carried out smoothly and has been all contributing to producing the Outputs. The training inJapan in particular, which is directly linked with the Project activities, contributed to smoothly producing the
Output 1. This design enabled the participants to effectively and efficiently learn the practical skills andknowledge of the Teaching Methods employed in Japan from local schools, teachers and the Japanese experts.
Thus the Project has a high degree of efficiency as a whole.

(4) Impact: Moderately high
The availability of the ADB loan from MECS allowed the Project to distribute one set of the Teacher’s
Guidebooks developed in the first and second years to all schools in the country, although this was not included in
the original plan of the Project. The dissemination of the Teaching Methods to non-model schools has been
gradually proceeding in three model Aimags/City through the existing in-service teacher training, open lessons,meetings, and other training. Among others, the following unexpected impacts have been observed: thedissemination of the Teaching Methods in pre-service teacher training courses in the teacher’s colleges; the issueof several Orders of the Minister that are expected to provide teachers with incentives to improve their teachingskills and to more actively interact with children; and the active interaction between the children in model schools
and their parents at home.
DECs are expected to play a key role in disseminating the Teaching Methods. As the Teaching Methods differ
from conventional ones, the existing in-service training alone seems insufficient for further dissemination of the
Teaching Methods to non-model schools. Moreover, if the budget for DECs is greatly reduced due to the financial
crisis, it will be difficult for DECs to launch new programs related to the dissemination of the Teaching Methods.
At the final evaluation, it is too early to say that the Overall Goal and the Super Goal will be achieved, thoughsome positive impacts have been confirmed. Therefore, the degree of the impact of the Project is assessed asmoderately high.

(5) Sustainability: Moderately highNo major changes are expected in the direction of the new education standards, but they were being evaluated andrevised by MECS at the time of the final evaluation. During the Project period, the following Orders of the
Minister that are expected to sustain the effects of the Project were issued by MECS: 1) a new system for
performance evaluation of teachers in accordance with the new education standards; 2) a new framework forin-service training of teachers; and 3) a new system for obtaining professional qualifications of teachers. Thus, thesustainability in the policy aspect is likely to be high. In the technical aspect, most of the Project stakeholders
have acquired the skills and knowledge of the Teaching Methods and are likely to apply them at work. The
Manual on Teacher's Guidebook Development to be developed and published by the end of the Project will help
sustain the know-how for developing Teacher's Guidebooks. Therefore, the knowledge and skills transferred by
the Project are highly likely to be sustained after the completion of the Project. Under the Project, the working
groups have played an important role in developing the Teacher’s Guidebooks. The Project stakeholders so far
have been highly motivated to sustain the effects of the Project. Thus it is possible that the Project stakeholders
will be involved in the development, dissemination and practice of the Teaching Methods in their own
organizations. Accordingly, the sustainability in the organizational aspect is relatively high.
For further dissemination of the Teaching Methods developed by the Project, individual efforts of a few
motivated pilot teachers and Aimag/City supervisors alone are not enough. An effective mechanism should be
developed and strengthened to enable top officials of DECs, Aimag/City supervisors who were not involved in
the Project, and principals and vice principals of non-model schools to understand the importance and necessityof the Teaching Methods and assist teachers in employing these methods in lessons. Since such a mechanism has
yet to be established at the time of the final evaluation, the sustainability in the institutional aspect is moderately
low. As the budgets of the Four Centers, the DECs, and the model schools are not secured, these actors areunlikely to accelerate the dissemination of the effects of the Project without any external assistance. In addition, if
the financial crisis adversely affects MECS, it may be difficult to allocate an additional budget for promotion of
the Teaching Methods. More internal efforts and closer cooperation with development partners are necessary tosustain and extend the effects and impacts of the Project. Thus, at the time of the final evaluation, thesustainability in the financial aspect is medium. Given the assessments above, it is fair to say that thesustainability of the Project is moderately high.
In addition, after this final evaluation period, JICA has decided to conduct the social sector support program incollaboration with ADB in May 2009. The objective of the Program is to protect the poor during the financial
crisis and to improve the targeting of social assistance and living conditions of the poor by ensuring andstrengthening basic social services and protecting essential social sector expenditures, thereby contributing to
improvement of public expenditure management, and enhancement of social development in Mongolia.

3-3 Factors promoting sustainability and impact(1) Factors concerning planning
The institutional arrangement of the Project was well designed as university teachers, model Aimag/City
supervisors, and model schools had been directly involved in developing the Teacher’s Guidebooks and the new
Teaching Methods through a variety of training, trial lessons, monitoring of trial lessons, and lesson analyses.
This approach made it possible to develop the Teacher’s Guidebooks that differ from the conventional andacademic ones developed by university teachers alone, but are expected to be more applicable to schools andteachers. The approach is also the most positive contributing factor that facilitated the achievement of the Outputs
and the Project Purpose and enhanced the effectiveness of the Project.
The training in Japan also helped the counterparts acquire the practical expertise on the Teaching Methods

employed in the classroom by teachers in Japan. The practical knowledge and skills that Mongolian counterparts
have learned in Japan were also shared with other counterparts who did not participate in this training, which also
deepened the former group’s understanding of the Teaching Methods. Thus the training in Japan was very
effective in enabling counterparts to develop the Teaching Methods that suit the actual conditions of Mongolia
and in creating a ripple effect such as introducing the new Teaching Methods in the pre-service teacher training
courses of the teachers colleges.
The development cycle of the Teaching methods in the Project including trial lessons, monitoring of trial lessons,and lesson analyses was undertaken for three years in UB City and for two years in Selenge Aimag and Dornod
Aimag. This allowed most of the counterparts to acquire the practical knowledge and skills step by step, which
can contribute to the attainment of Outputs, several positive impacts, and the sustainability of the Project in thetechnical aspect.

(2) Factors concerning the implementation process
Prior to the Project, some of the core counterparts had, to some extent, acquired the knowledge about the teaching
methods being employed in Japan through the technical transfer from one JICA expert who had been dispatchedto MECS or from the other technical training in Japan conducted by the “Strengthening the Planning Capacity for
In-Service Teacher Training Project.” The assignment of these counterparts with a high sense of ownership andresponsibility to the Project has led to smooth implementation of the Project. In addition, the transfer ofknowledge and skills from the Japanese experts to the counterparts has been successfully undertaken. The
Japanese experts provided their Mongolian counterparts with appropriate knowledge and skills every year, i.e.,
lesson study (jugyou kenkyu), analyses of teaching materials (kyozai kenkyu), and child development to stimulate
the counterparts' motivation for learning.

3-4 Factors inhibiting sustainability and impact(1) Factors concerning planning
The indicators were not well identified in the PDM to measure the degree of the achievement of the Project
Purpose and the Output 1. Although the content of the Project Purpose and the Output 1 differed from each other,
the same indicator was set to measure the degree of the achievement of these two items. Thus, the indicators need
to be adjusted at the time of the final evaluation according to what has been really happening on site.
Occasionally the logframe was not clearly formulated and well documented, although the PDM was revised by
the Project based on the recommendations of the mid-term evaluation. It seems hard for even the projectstakeholders to explain the difference between the summaries of the Project Purpose and the Outputs in the PDM.
The PDM thus needs to be theoretically formulated, and well documented and shared among the Projectstakeholders in the planning stage of the Project.

(2) Factors concerning the implementation process
Indicators in the PDM can be set clearly or changed based on results of baseline surveys or discussions of projectstakeholders after the start of a project. In this Project, it was found at the time of the final evaluation that several
indicators were not objectively measured and had no target values. Thus the final evaluation study team had toadjust indicators and newly set the criteria of judgment according to what has been actually happening. Also, thealternative data and information such as the results of the Questionnaire Survey conducted by the local
consultants of JICA prior to the final evaluation were used to measure the achievement of the Outputs and the
Project Purpose since some of the indicators in the PDM were not available. In hindsight, the results of thebaseline data should have been well utilized to set the objective and measurable indicators in the PDM.

3-5 Conclusion
Most project activities have been smoothly carried out through the good cooperation between the counterparts
and the Japanese experts. The Project has properly addressed the priorities and needs of MECS, schools, andteachers. It is also consistent with the Mongolian education policies and the Japanese aid policies. Thus, therelevance of the Project is very high. Since most of the Outputs and the Project Purpose have been almost
achieved, the Project as a whole has a high degree of effectiveness. The working groups for eight subjects havecommitted themselves to developing the new Teaching Methods and the Teacher’s Guidebooks. The training inJapan was particularly effective in imparting practical knowledge and skills of the teaching methods to thecounterparts, which has contributed to smooth implementation of the Project. As the relatively small size of
project inputs have been producing effects as expected, it is fair to say that the efficiency of the Project is high.
The ripple effects of the Project have been confirmed to some extent. Overall, the sustainability of the Project ismoderately high at the time of the final evaluation. If the institutional and financial aspects are strengthened byestablishing an effective mechanism and securing a sufficient budget for the dissemination of the new Teaching
Methods, the prospects for sustainability will improve.

3-6 Recommendations

3-6-1

Collaboration of university teachers with classroom teaches

The approach adopted by the Project where university teachers and classroom teachers on the ground work
together, seemed to be highly effective because the university teachers were able to understand the actual needs ofclassroom teachers and children on the ground. The knowledge they have acquired in the process has been
improving the quality of pre-service in the universities.
In addition, as some of the university teachers are involved in the curriculum reform and textbook
writing, the knowledge about the Teaching Methods that they have acquired through the project arealso expected to be reflected in the curricula and textbooks in Mongolia in the long run.

3-6-2

Inter-linkage of the project activities in Mongolia and Japan

Since the training in Japan was planned and conducted by the Japanese experts who were involved in the Project
activities in Mongolia, the training was well coordinated with the Project activities in Mongolia. This contributed
to enhancing the effectiveness of the Project.

3-6-3
(1)

Necessity of making use of the Base-line and End-line surveys for project monitoring
The date collected for the Base-line survey and the End-line survey should have been better utilized tomonitor the progress of the Project. Although the Project conducted a comprehensive Base-line survey
in the three Aimags/City, it seems that those date were not well utilized to monitor the progress and toevaluate the effectiveness of the Project.

(2)Indicators of the PDM should have target values. If the decision was made not to set the target values
for the indicators, the decisions and reasons should have been recorded in the official documents suchas JCC minutes.

END