Summary of the Terminal Evaluation
I.

Outline of the ProjectCountry: Mongolia

Project: Development of Human Capacity for
Weather Forecasting and Data Analysis

Sector: Natural environment

Cooperation scheme: Technical cooperation(turnkey contract)

Division in charge: Disaster Management
Division 1, Water Resources and Disaster
Management Group, Global Environment
Department

Cooperation amount (at the time of evaluation):
501,275,000 yen

Period ofFeb. 2005 to Oct. 2008
cooperation

Partner country’s implementing organizations:
National Agency for Meteorology, Hydrology
and Environment Monitoring, Ministry of Natureand Environment
Supporting organizations in Japan: Japan
Meteorological Agency, Japan Weather
Association

1.

Background to and outline of the cooperation

Mongolia’s key industries are agriculture and livestock farming, which account for about 20%of the GDP and employ about 42% of the total working population. In this situation, the recent
large-scale droughts and zud (snow disasters) caused serious damage to Mongolia’s society
and economy. In 2001 and 2002, about 2.69 million domestic animals died, resulting in total
damage amounting to 1,008 billion tugrik (about 96 billion yen). The annual damage since1999 has totaled about 2.31 million domestic animals amounting to about 316.8 billion tugrik
in losses (about 30.2 billion yen). Weather disasters, not only from droughts and zud, but also
from flooding, hailstone disasters, and strong winds, have occurred throughout the country
and have become a factor hindering the social and economic development of Mongolia, which
depends on agriculture and livestock farming. In addition, there is concern that not only
short-term weather changes, but also climate change due to global warming and long-termchanges in the natural environment and terrestrial ecosystems due to desertification and otherforms of climate change may have a serious impact on the life of the Mongolian people, who
greatly depend on nature, through the impact of weather conditions on agriculture andlivestock farming and on water resources.
To cope with these problems through the National Agency for Meteorology, Hydrology andEnvironment Monitoring (NAMHEM), which is the national meteorological agency, theMongolian Government has carried out activities to improve the meteorological service
according to a master plan drawn up with the support of the Japanese experts. Japan has so far
dispatched experts including those for the support of establishing the master plan four times
and twice given grant aid for the installation of observation, forecasting and network
equipment. Concretely, the introduction of equipment for weather observation, forecasting andcommunication, such as weather radar and an automatic weather observation system, resulted
in the establishment of the required physical facilities to some extent. As for the non-physicalfacilities, JICA’s unique training activities, such as the overseas dispatch of engineers, resulted
in a certain level of achievement in raising technical skills and improving the weather
information system. However, the technical level of NAMHEM as a whole could not be raised
1/8

through the acquisition of the excellent skills of other countries in such fields as numerical
forecasting and meteorological analysis, which has became a substantial factor hindering thedevelopment of the meteorological sector. In addition, weather information has not been usedappropriately due to a lack of understanding on the part of the users, such as administrative
agencies and livestock farmers, with the result that disaster prevention activities have not been
carried out appropriately.
In response to this situation, the Mongolian Government requested Japan to carry out atechnical cooperation project for improving the weather forecasting and warning systemthrough the introduction of new weather analysis and forecasting technologies such as
numerical forecasting, preparing predictive information on weather changes in the Mongolian
region due to global warming, creating an early warning system for droughts and zud (snowdisasters), preparing information on the observation of the yellow sand phenomenon andincreasing the understanding of administrative agencies and livestock farmers concerning
weather information through the development of technical human resources engaged in themeteorological service in order to further expand the foundations of the weather observation
service cultivated through the dispatch of experts and the provision of grant aid, aiming tocontribute to the appropriate management for natural disasters and the assessing the impact ofclimate change. As a result, the “Project for the Development of Human Capacity for Weather
Forecasting and Data Analysis in Mongolia” has been carried out since February 2005. The
mid-term evaluation of this Project was carried out in September 2006, and this Project isscheduled to end in October 2008.
Contents of the cooperation(1)

Overall goal
Weather information will be used for the management of natural disasters and the impact
assessment of climate change.

(2)

Project goal
More reliable and useful weather information (including information on the yellow sandphenomenon) will be provided in a timely manner through the development of thecapacity of meteorological service officials and environmental experts.

(3)

Outputs
1) Realization of numerical forecasting through the use of a regional model for
Mongolia2) Prediction of climate change in the Mongolian region due to global warming
through the use of a climate model
3) Preparation of weather forecasts (short-term, medium-term, long-term) based on
numerical forecasting data
4) Creation of an early warning system for droughts and zud5) Increase in the understanding of weather information on the part of the national andlocal governments, related organizations and end users (including livestock farmers
and residents).
6) Stable management of the weather observing and forecasting system (weather
radars and computer networks)
2/87) Preparation of the observation information on the yellow sand phenomenon(4)

Inputs (at the time of evaluation): Total amount of inputs: 501,275,000 yen
Japanese side (JICA):
Dispatch of 14 experts; provision of equipment totaling about 236,125,000 yen;
acceptance of 11 training participants; payment of local costs totaling about
19,719,000 yen
Mongolian side:
Placement of 7 counterparts; provision of land and facilities: offices for theexperts; payment of local costs totaling 235.1 million tugrik

II.

Outline of the evaluation teamTeam
members(1) Team Leader/Supervisor: Mr. Kazuo SUDO, Advisor, Global Environment
Department, JICA
(2) Weather Analysis Evaluation: Mr. Masashi KUNITSUGU, Numerical
Prediction Division, Forecast Department, Japan Meteorological Agency(3) Evaluation Analysis: Ms. Yukiko SUEYOSHI, Global Link Management Inc.
(4) Cooperation Evaluation: Mr. Shohei MATSUURA, Disaster Management
Division 1, Water Resources and Disaster Management Group, Global
Environment Department, JICA

Period of September 5 to 25, 2008
theevaluation

Type of evaluation: Terminal evaluation

III. Outline of the evaluation results (A: very good; B: good; C: partially good but has
problems; D: bad)
3-1

Confirmation of the achievements(Degree of achievement of the project goal)
Evaluation result (B): Based on the results of this evaluation study, we made the following
judgments: (1) The skills transferred by the Japanese experts have been used effectively for
NAMHEM’s daily work; (2) Because the skills acquired by the counterparts (CPs) weredisseminated among young engineers, the project goal to improve the meteorological service
through the development of human resources of NAMHEM is being realized.
(Degree of achievement of Output 1) Numerical forecasting
Evaluation result (A): Output 1 aims to develop human resources to provide more accurate
forecasting services based on numerical forecasting, using the computer system introduced
according to the grant aid project called the “Project for Improvement of the Meteorological
Information Network,” which Japan carried out in 2003. As a result of this evaluation study, it
was confirmed that the CPs have acquired the knowledge and skills necessary for thedevelopment of a regional numerical forecasting model through the lectures and practiceconducted by the Japanese experts and reached a level that is sufficient for them to be appliedin everyday operations. The achievements of this Project can be seen from some factual
results: for example, the implementation of this Project has resulted in an improvement in theaccuracy of forecasting in the Mongolian region and extension of the forecasting period fromthree days to five days.
3/8

(Degree of achievement of Output 2) Climate change prediction
Evaluation result (B): For the purpose of supporting the implementation of NAMHEM’s“Meteorological Environment Sector Development Program until 2015,” Output 2 aims to
improve the accuracy of the climate change prediction model and disclose information
through the development of human resources engaged in the climate change prediction service(Output 2 is planned to be realized by the end of this Project). Because a CP went abroad tostudy during the project period, but no new CP was assigned, the target number of humanresources to be developed was not achieved. However, it is confirmed that the official
studying abroad will be engaged in the climate change prediction service after returning to
Mongolia. Moreover, because the officials whose capability in the sector was improved
through this Project are positive about the training of young officials, the achievement offurther outputs can be expected in the future.
(Degree of achievement of Output 3) Weather forecasting
Evaluation result (B): Although NAMHEM has improved the information processing system
and accumulated weather data through the development of weather forecasting equipment
provided by Japan’s grant aid and other sources, it is necessary to train engineers who caneffectively use the equipment and data. This output aims to establish a guidance working
group centering on the CPs and to develop human resources for the preparation of guidance
based on the results of numerical forecasting. Because no new CP was added in spite of a CP
studying abroad or retirement during the project period, the target number of human resources
was not achieved. On the other hand, because the officials whose capability in the sector was
improved through this Project have begun to train young engineers on their own, theachievement of further outputs can be expected in the future.
(Degree of achievement of Output 4) Early warning system for droughts and zud
Evaluation result (B): This output aims to improve the existing weather observation system for
agriculture and livestock farming and develop human resources engaged in the creation of anearly warning system for droughts and zud based on the observation data. The implementationof this Project resulted in the subdivision of the scale of the map for grass volume and its levelfrom the county (sum) to the village (bag) level and the preparation of guidelines that specify
the alert point necessary for the issuance of a warning for droughts or zud. However, the useof the guidelines is limited.
(Degree of achievement of Output 5) Popularization of the weather service
Evaluation result (A): This output aims to assess the needs of weather information users andincrease their understanding of weather information through seminars and workshops forgovernment offices related to disaster prevention and the pilot provincial agencies related todisaster prevention and residents. The implementation of this activity resulted in the
deepening of the knowledge of weather information users and an increase in the motivation of
information providers of the weather forecasting services through their dialogue with theusers. Moreover, this activity also contributed to the promotion of cooperative relationships
with NAMHEM and related government offices, especially NEMA.
(Degree of achievement of Output 6) Management and maintenance of the weather
observing and forecasting system
Evaluation result (A): This output aims to promote the sustainable use and maintenance of the
Doppler weather radar and the computer network that Japan gave through grant aid and todevelop the human resources necessary for the management and maintenance of the
4/8

equipment and the operation of the systems. As a result of interviews with the Japaneseexperts and CPs, it was confirmed that skills in the management and maintenance of theweather observing and forecasting system had reached a sufficient level at the time of theevaluation. On the other hand, although the equipment procured with grant aid is in operation,it has become decrepit and the manufacturers have already discontinued the manufacture ofspare parts.
(Degree of achievement of Output 7) Monitoring of the yellow sand phenomenon
Evaluation result (A): This output was added in March 2005 after the beginning of this Project
and aims to provide the equipment necessary for the creation of a network for monitoring theyellow sand phenomenon and to develop the human resources necessary for this monitoring.
The data sent regularly from the four observatories in which equipment for observing theyellow sand phenomenon was installed were effectively used by the CPs and Japanese experts
to provide the information necessary for analyzing the situation of the yellow sandphenomenon and the conditions related to air pollution.
3-2

Summary of the evaluation results (A: very good; B: good; C: partially good but
has problems; D: bad)(1)

Relevance

Evaluation result (A): NAMHEM’s “Meteorological Environment Sector Development
Program until 2015” specifies that NAMHEM will aim to improve its meteorological service
through the development of human resources, the modernization of meteorological equipment,
the creation of a meteorological network, etc. by 2015. Moreover, in relation to environmental
conservation, one of the priority sectors, Japan’s ODA policy called “2004 Country Assistance
Program for Mongolia” specifies comprehensive support for meteorological andenvironmental monitoring and the establishment of a natural environment information system.
There is therefore consistency with this Project. Because NAMHEM is a government agency
that has unified control of the provision of weather information, improvement in theknowledge and skills of NAMHEM officials is essential for improving the meteorologicalservice. Therefore, it can be considered that the selection of the target group and the executing
agency was appropriate.
(2)

EffectivenessEvaluation result (B): It has been confirmed that the implementation of this Project resulted in
an improvement in the meteorological knowledge and skills of the CP and that thedevelopment of human resources resulted in an improvement in the accuracy of weather
information. However, the indicator for measuring the degree of achievement (the number oftrained persons) has still not been fulfilled completely. Because the knowledge and skills
essential for improving NAMHEM’s meteorological service have already been transferred, it
can be considered that the outputs planned for this Project have directly contributed to theachievement of the project goals. Output 5 aims to carry out awareness raising activities
among the administrative agencies and the nomads that use weather information and topromote the improvement of the quality of a meteorological service that reflects the needs ofthe users.
(3)

Efficiency

5/8

Evaluation result (B): It was confirmed that the inputs from the Japanese side were carried out
almost according to plan in terms of quantity, quality and timing and that the inputs were fully
used in achieving the project goal. These inputs have already been effectively used for the
daily work of the CPs without causing any limitations on project activities. The equipment
provided by the Japanese side was also used appropriately and engineers required for its
maintenance have also been assigned appropriately. It can be considered that, to improve theefficiency of this Project, it was necessary to confirm the status of progress based on the PDM
more frequently. The Mongolian side also appropriately provided the personnel and facilities
required for the implementation of this Project. Although some activities were delayed due to
one CP studying abroad and other personnel changes, all the activities are expected toeventually be implemented.
(4)

ImpactEvaluation result (B): To achieve the overall goal, it is essential to strengthen the cooperative
relationship between NAMHEM and the related organizations and use weather information
for the preparation of policies related to the management of natural disasters. At present,
however, this seems to be still at an early stage where NAMHEM shares with relatedorganizations weather information whose accuracy level has been increased than before. As
positive impacts, the following cases were reported: the CPs used the monitoring data of theyellow sand phenomenon, one of the achievements of this Project, for improving research andstudies and expanding the joint research network with other organizations and local workshop
participants have changed their behavior in accordance with the weather information.
(5)

Sustainability

Evaluation result (A): With regard to the political aspects, because NAMHEM is planning todevelop human resources in the weather sector, improve weather information and create anationwide weather network as specified in the “Meteorological Environment Sector
Development Program until 2015,” political support has been secured to continue theachievements of this Project. With regard to the organizational and financial aspects, giventhat 1) NAMHEM has a high sense of responsibility for and ownership of the weather sector
in Mongolia as the government agency that controls the Ministry of Nature and Environment’s
provision of weather information and that 2) NAMHEM’s annual budget has been on an
upward trend because of its importance in the national strategy for the weather sector, thesustainability can be said to be high. Moreover, because the CPs have already begun to use
new the skills and knowledge acquired through this Project for their daily work and have been
training engineers in their department, the sustainability was also judged to be high from theviewpoint of skills and knowledge.
3-3

Factors for the emergence of the effects

The first factor for the emergence of the effects is that the foundations for weather observation
service on the part of the CPs were established by the dispatch of experts and the provision of
grant aid by Japan. Moreover, according to the results of interviews with the CPs, it waspointed out that the knowledge and skills transferred from the Japanese experts were directly
connected to their daily work and fulfilled the Mongolian side’s technical needs. This
increased the ownership by NAMHEM’s CPs and engineers of this Project and theirmotivation to participate in the project activities, resulting in smooth progress in various
activities. The other factor regarding the increase in ownership by the CPs was the direct
6/8

dialogue with the users in local workshops. It was reported that the participation in theworkshops served as an opportunity to promote the provision of an easy-to-use meteorologicalservice.
3-4Problems and related factors

One of the problems with this Project is that, because some of the CPs at the beginning of thisProject went abroad to study or left their jobs, but were not replaced, the target number of theCPs was not attained. On the other hand, because the officials trained under this Project havebeen eager to train young officials, it can be expected that the necessary number of trainedengineers will be achieved.
3-5Conclusions

To realize the project goal of the “provision of more accurate weather information through thedevelopment of human resources,” JICA developed human resources for NAMHEM tocontribute to the improvement of the meteorological service. At the time of this evaluation,some activities are in progress concerning the seven outputs for the achievement of the project
goal. The CPs have acquired the capabilities to continue the activities independently, use themfor their daily work and disseminate the achievements. Because of this, the equipment
provided under this Project has also been used effectively and a system for maintaining theequipment has been established. Therefore, the sustainability of this Project was judged to bevery high. To expand the achievements of this Project in the future, it is desirable for
NAMHEM to strengthen its cooperation with related government offices and agencies andclarify the role of the weather sector in the management of natural disasters.
3-6

Recommendations

The recommendations to NAMHEM in the terminal evaluation can be summarized as follows:
1) NAMHEM should continue to verify the numerical forecasting model and use the modelfor the service.
2) NAMHEM should widely popularize the fruits of this Project related to droughts, zud,climate change, etc., among the general public.
3) NAMHEM should establish an early warning system for droughts and zud, create an
in-house system for disseminating information among the general public without failand provide information to the Ministry of Nature, Environment and Tourism (MNET),
the Ministry of Food and Agriculture (MFA), the National Emergency Management
Agency (NEMA) and other government offices.
4) NAMHEM should continue to hold the training seminars that were given in the threepilot provinces and expand the target areas.
5) NAMHEM should secure the budget for the above-described activities and thecontinuous maintenance of the provided equipment even after the end of this Project.
3-7

Lessons

1) Lessons concerning the technical transfer: If, as with the CPs in this Project,personnel have substantial experience in the meteorological service, the transfer ofadvanced skills and knowledge by the Japanese experts can be carried out more
effectively. Moreover, if leading persons are appointed as the CPs, the transfer of
7/8

technical skills to young engineers and the dissemination of knowledge among them will
be further promoted. As a result, this Project is highly likely to become a factor in
facilitating independent development.
2) Lessons concerning the monitoring system: Technical transfer, which centers on the
dispatch of short-term experts, may face difficulties in continuously carrying out
monitoring activities to check the progress of the project. Therefore, it is important for
the CPs and Japanese experts to share information on the progress of the project andpositively promote consultations with each other about problems during theimplementation of the project in order to improve the efficiency of the project.

8/8