Evaluation Summary
1. Outline of the ProjectCountry: Malaysia
Project Title: Project on Networked Multimedia Education System
Issue/Sector: Telecommunications
Cooperation Scheme: Technical Cooperation Project
Division in Charge:
ICT Team, Group II, Social Development Department
Total Cost (at the time of evaluation): Approx. 910 million yen
Period of Cooperation(R/D): 1 July 2001 - 30 June 2005
(Extended period):(F/U):(E/N) (Grant aid)
Partner Country's Implementing Organization(s):Ministry of Energy, Water and Communications (MEWC); Multimedia University (MMU)
Supporting Organization(s) in Japan: Ministry of Internal Affairs and Communications (MIC)
Related Cooperation: Nippon Telegraph and Telephone East Corporation; Matsushita ElectricIndustrial Co., Ltd.

1-1 Background to the Project
The Government of Malaysia has been forging ahead with the Multimedia Super Corridor(MSC) program since it launched the Seventh Malaysia Plan 1996-2000 (7MP) as part of its
efforts to promote information technology (IT) at the national level. The goal of the programis to join the club of developed countries by 2020 with this huge infrastructure for anadvanced information society. To achieve this goal, the Malaysian government establishedMultimedia University, Malaysia (MMU) in 1999 to train engineers in the IT and multimedia
fields.
The Malaysian government requested Project-type Technical Cooperation (now “a technicalcooperation project) from Japan with the aim of establishing the Networked Multimedia
Education System (NMES) that would build on MMU as the hub site and five regional
educational institutions as remote sites.

11-2 Project Overview(1) Overall Goal
To expand the NMES and involve more institutions at home and abroad in the fields ofengineering, IT and multimedia.
(2) Project Purpose
To establish the NMES at MMU and the remote sites.
(3) Outputs
1) A system will be in place for satellite-based tele-education at MMU and the remote sites.
2) Tele-education courses will be provided according to the curricula of MMU and the remotesites.
3) Effective multimedia teaching/learning materials will be used in the tele-education courses.
(4) Inputs (until the time of evaluation)
Japanese side:
Long-term Experts: 8 experts
Short-term Experts: 24 experts
Trainees received: 15 persons

Equipment: 468,805,000 yen
Local cost: 16,564,000 yen

Malaysian side:
Counterparts: 35 persons
Land and facilities
Local cost: 1,070,058 ringgit

2. Evaluation Team
Members of the Evaluation Team
(Role/responsibility: NamePosition)
Team Leader: Nozomu GODA Senior Advisor, Institute for International Cooperation,JICA
Technology Transfer Planning: Akira IWATA Professor, Graduate School of Engineering,
Nagoya Institute of Technology
Technical Cooperation Planning: Yuji AOKI Assistant Director, International Cooperation
Division, International Affairs Department, Telecommunications Bureau, Ministry of Internal
Affairs and Communications
Evaluation Analysis: Takako HARAGUCHI Permanent Expert, International Department
Associates Ltd.
Cooperation Planning: Tomoyuki YASUDA, ICT Team, Group II, Social Development
Department, JICA
Period of Evaluation: 15 May 2005 - 28 May 2005
Type of Evaluation: Terminal Evaluation
2

3. Results of Evaluation
3-1 Achievement Level
The project has largely achieved the outputs. The NMES, a satellite-based system to support
real-time, interactive multimedia tele-education has been in place at the hub site (MMUCyberjaya Campus) and the five remote sites. Following the technology transfer, it is now
operated and maintained chiefly by the counterparts. The Initial problems such as instabilityof satellite communication and malfunctions of some instruments and applications have beensolved with adjustments to and replacement of the instruments. The introduction of MPEG4
in 2005 has improved the audiovisual quality and resulted in the use of less satellite
bandwidth. [Output 1]
The NMES-based courses are now provided for students who wish to acquire formal
qualifications and degrees approved by the Malaysian government (diplomas, and bachelor’sand master’s degrees). The total operating hours of NMES to date stand at 2,246 hours forclasses and 836 hours for meetings, short courses and seminars. A total of 856 students haveenrolled in tele-education courses and at least 5,800 people have attended short courses /seminars using NMES. [Output 2]
Because the text-based and slide-based reference materials used in face-to-face classes are
considered both effective and feasible for tele-classes as well, the lecturers prepare them fortele-classes as well and update them as necessary. They have yet to prepare video-based
multimedia teaching/learning materials for tele-classes. The performance of the first group ofstudents at the remote sites compares favorably with that of those at face-to-face classes.
Students in the master’s courses are not so satisfied with the tele-classes; they call for higher
audiovisual quality and more interactive discussion. [Output 3]
The Evaluation Team concludes that the Project Purpose has been achieved. By April 2005,
16 students, including 13 at the remote sites, completed the first NMES-based diploma course.
The implementation process of the Project was largely successful, except for the equipment
installation process, in which responses to the initial system failure left much to be desired.

3-2 Summary of Evaluation Results(1) Relevance
The Evaluation Team Considers the Project relevant. The purpose of the Project was in linewith both the development policy of Malaysia (including developing skilled workers,
reducing regional disparities and meeting the need for human resources for MSC and otherpolicies under the national development plan) and Japan’s assistance policy for Malaysia (ashighlighted by the government-to-government agreement on human resources development
for MSC, and the use of information and communication technology in education). Demandfor tele-classes and higher education at the remote sites generally remains high, but initial
demand for tele-education at one remote site has decreased after it hired new lecturers forface-to-face education.

3(2) Effectiveness
The Evaluation Team considers the Project effective. The Project Purpose of establishing
NMES has been achieved. For one, the first group of students in the NMES-based diploma
course compared favorably with those in the face-to-face counterpart. For another, MMU
now does not need to send lecturers to its remote sites due to the establishment of NMES andan accumulation of experiences in operating the system (the outputs).
(3) Efficiency
The Evaluation Team concludes that the Project was largely efficient except for one major
aspect. The project activities were sufficient for achieving the outputs. All the inputs wereused for implementing the activities. The problem rested with the untimely assignment ofexperts from JICA. For example, the dispatch of engineers to cope with the frequent cases of
initial system failure was delayed until the middle of the project period. Experts indeveloping teaching/learning materials were sent to Malaysia during the period of systeminstability, which made it difficult for them to concentrate on their original duties. Still, theintegrated assignment of JICA experts in the second half of the project period helped produce
sufficient outputs in the end.
(4) Impact
The overall goal has not been achieved and there were no specific plans to involve otherinstitutions in NMES at least at the time of the terminal evaluation. Although the Project has
demonstrated that NMES provides an effective tool for tele-education, additional investment
to increase its beneficiaries is essential for ensuring that the Project has an impact on thedevelopment plan of Malaysia (by contributing to human resources development). The
impact of the Project on technological development in Malaysia is obvious because theProject realized the practical application of real-time, interactive multimedia tele-education
for the first time in Malaysia, where asynchronous media (mail or the World Wide Web) was
the norm for distant learning. Policy and budgetary commitment to increasing theparticipating institutions is necessary for attaining the overall goal.
(5) Sustainability
The impact of the Project will likely be sustained at the current level at least. At the policy
level, the Malaysian government (or MEWC to be precise) has committed itself to supportingthe continuation of NMES and bear the repair costs of the equipment. It has alreadydeveloped a plan to expand NMES. At the organizational level, MMU has shown a high levelof organizational capacity. The structure and staffing for operating NMES will be maintainedeven after the completion of the Project. Malaysia will continue to fund the operation andmaintenance of the current system. (It is unclear whether Malaysia will invest in the plan toexpand the NMES as mentioned above.) On the technical front, the additional technology
transfer in 2005 have allowed the counterparts to acquire system maintenance skills that are
well above the levels of the system users and adequate to cope with equipment failures
promptly.

43-3 Contributing Factors(1) Concerning the project plan
Institutions with high levels of organizational and implementation capacity have been selected
as the counterparts.
(2) Concerning the implementation process
Continued efforts toward stabilizing the system and improving tele-classes; and high levels ofownership of these efforts by the Malaysian side.

3-4 Inhibiting Factors(1) Concerning the project planUnclear system specifications; the lack of arrangements for holding the contractor responsible
for the initial test operations; the adoption of an application that had not been used widely
even in Japan; inadequately defined measures for system maintenance; and other factors.
(2) Concerning the implementation process
Untimely responses of JICA Headquarters to the requests for prompt action to the initialsystem failures and for technology transfers for system maintenance; and a high proportion of
the counterparts who left their jobs.

3-5 Conclusion
The Project is successful. It has achieved its purpose and demonstrated that the NMES
provides an effective tool for tele-education. It is expected that the maintenance andexpansion of the NMES will further contribute to the development of human resources withhigh levels of skills in Malaysia.
It was agreed that the Project will be completed on June 30, 2005 as scheduled.

3-6 Recommendations (specific solutions, suggestions and advice for the Project)
To help maintain and build on the positive effects of the Project, the Evaluation Team
recommends that the Malaysian government and MMU take the following measures:(i) To monitor progress and achievements in tele-education courses, as the application ofMPEG4 to NMES starts in June 2005 when the new semester begins;(ii) To continue and redouble their efforts to raise the levels of satisfaction of the users,
especially the students in the master’s courses, regarding the tele-education courses;(iii) To continue and redouble their efforts to increase the number of beneficiaries of theNMES-based tele-education;

5

(iv) To study measures to stem the hemorrhage of counterpart engineers in system operationand management;(v) To further improve the techniques for tele-classes, including teaching/learning materials
and teaching methods, as this is an aspect that began to be addressed recently after the initialemphasis on system stabilization; and(vi) To make policy and budgetary arrangements necessary for increasing the number ofinstitutions participating in NMES.

3-7 Lessons Learned (especially those that provide information that is useful for
identifying/formulating, implementing, and administering similar projects)
Major difficulties in the Project, including a long-lasting period of low quality and reliability
after the provided equipment and system were put in place, and the inability to conclude amaintenance contract for the system were attributed to the inadequate process of project
formulation. The Evaluation Team has therefore analyzed major problems in the project
formulation process and identified opportunities for improvement as shown below for future
projects in the similar sector:(i) Assessing the real needs of the partner country and fully understanding why it asked Japanfor assistance and what kind of assistance it expected from Japan;(ii) Meticulously selecting equipment for the project and securing a comprehensive contract
covering everything from written specifications and procurement methods to maintenance;(iii) Placing a premium on the track record, stability and reliability of the equipment in thespecification preparation process; and(iv) Holding substantial consultations and then reaching an agreement with the partnercountry over necessary budget allocations in an equipment maintenance contract.

6