Source document: https://docs-lawep.s3.us-east-2.amazonaws.com/1698746622576.pdf (Malaysia)
ENHANCING MONITORING AND EVALUATION FOR BETTER RESULTS Abstract This paper provides an overview of the general principles and rationale behind the development of a results-based management approach for public sector management.

Starting this year, the public sector management will implement the Outcome-Based Budgeting (OBB) system.

This system is being developed to improve policy and program execution and manage results for good governance and expand accountability.
Prepared by Parliamentary Research Officer, Amy Tam January 2013 amytam@parlimen.gov.my Contents 1.0 2.0 3.0 4.0 5.0 Introduction General Principles of Results-based Management Approach Malaysia’s Results-based Management Approach 3 4 5 3.1 3.2 3.3 3.4 Program Performance Budgeting System and Modified Budgeting System .................................... 5 Outcome-Based Budgeting ............................................................................................................... 7 Internalised Self-Evaluation (ISE) .................................................................................................... 13 MyResults ....................................................................................................................................... 15 Monitoring and Evaluation – A Critical Tool Summary and Conclusions 16 18 January 2013 1.0 Introduction The world is experiencing a growing demand for greater accountability and transparency, not only on policies and programmes but of, entire governments’ performance.
Perhaps, the most significant shift in the last decade, is the growing emphasis on expanding accountability and transparency in the use of public funds, often referred as “managing results”.

It is, indeed, an emergent global trend and it indicates that the world is progressing towards a more responsible society.

This trend brings about innovative approach on how governments should respond to the demand – how should results or performance be measured?
“Results1” is simply defined as sustainable development outcome and “managing results” as a management strategy which focus on performance and the achievement of output and outcome.

The development of public sector management is fluid and subject to multiple theories and suggestions on how to get more value from the limited resources to promote an effective government.

Global economy outlook provides another challenge.


Public sector planners, policy makers and development partners must ensure that their economy have sufficient fiscal space to absorb global shocks.

It has become a major topic of discussion among developed and developing countries, international organisations and civil societies to focus global discussions on the need to improve the utilisation of resources and its impact on developments.
The Paris Declaration on Aid Effectiveness 2005 emphasised the principle of focusing on “development results and results get measured”.

The Accra Agenda for Action (AAA, 2008) emphasised on “real and measurable impact on development”.

Malaysia is a signatory to both the Paris Declaration and Accra Agenda Action.
1 For the purpose of this paper, “results” is used interchangeable with “outcome” ENHANCING MONITORING AND EVALUATION FOR BETTER RESULTS Page 3 ENHANCING MONITORING AND EVALUATION APPROACH TO ADDRESS DEMAND FOR GOOD GOVERNANCE AND ACCOUNTABILITY January 2013 In line with the principle of measuring results and impact on developments, the public sector is implementing Outcome-Based Budgeting (OBB) system this year.
The introduction of the OBB in the 10th Malaysia Plan (10MP) (2011 – 2015) will address the weaknesses of the Modified Budgeting System (MBS) through strategic integration of planning, budgeting, implementation, monitoring, and evaluation at various levels.

Monitoring and evaluation are critical tools in the implementaion of OBB.
This is, in recognition that data and information gathered on results, when effectively linked to key elements of decision making, particularly resource allocation, contributes to improved organisational effectiveness and governance.

2.0 General Principles of Results-based Management Approach a) Focus on results that matters to the people Decision makers will be guided by measurable results, whether the intended purpose is achieved.
As such, it closes the gap between the supply of services or goods and the demand for such services or goods.

Decision will be made by assessing the defined changes in the behaviour/lifestyle of individuals, households, community and/or firms – i.e.

the intended target beneficiary.


b) Provide clear and transparent linkages with National Development Framework A results-based approach seeks clear and transparent linkages between explicit definitions and measures of performance in national and ministries/agency specific plans, encourages links between budget allocation and performance, and management of human resources.

It presents a positive and practical set of tools to address demand for good governance and accountability.
c) Focus on wider participation in decision making A results-based approach provides a tool for community to voice their definition of goods and services which they really need and not a need conjectured up by politicians or individuals with vested interests.

Measurable results take account of feedbacks from community.
ENHANCING MONITORING AND EVALUATION FOR BETTER RESULTS Page 4 ENHANCING MONITORING AND EVALUATION APPROACH TO ADDRESS DEMAND FOR GOOD GOVERNANCE AND ACCOUNTABILITY January 2013 3.0 Malaysia’s Results-based Management Approach 3.1 Program Performance Budgeting System and Modified Budgeting System Malaysia has introduced budgetary reform initiatives since the late 60’s.

The Program Performance Budgeting System (PPBS) was introduced in 1969 to replace the traditional line item budgeting system.
In the early 90’s, the PPBS was then reviewed and strengthen through the introduction of the Modified Budgeting System (MBS).

Due to fiscal challenges arising from global uncertainties since the late 1990s, the government decided on a number of strategic reforms to improve program and service delivery.

Among the reforms was the focus on outcome in aspects of planning and budgeting.

As early as 2009, guidelines were drawn up on the implementation of an outcome- based approach.
Saddled with demand for greater accountability by the public and a challenging fiscal position in the country, the government decided to replace MBS with OBB on October 2011.

A special OBB team was formed to expedite the implementation of OBB.

OBB is a comprehensive mechanism to translate planning into results through effective policy and program implementation of public funds.
By focusing on outcomes, planning, budgeting, implementation, monitoring and evaluation, resource allocation are optimised and integrated properly into an efficient system.
ENHANCING MONITORING AND EVALUATION FOR BETTER RESULTS Page 5 ENHANCING MONITORING AND EVALUATION APPROACH TO ADDRESS DEMAND FOR GOOD GOVERNANCE AND ACCOUNTABILITY January 2013 Weaknesses of PPSB and MBS: PPSB was based on items of expenditure and as such, it was a mere tool for funds appropriation and disbursement rather than strategic management tool.
There was no integration between the operating and development expenditures; and the personnel performance system and it created only limited linkages among budget performance, resource usage and policy implementation.

As a result, clear relationship between resource utilisation and performance could not be clearly established.

MBS was based on Program - Activities approach within a long term macro planning strategies.
It focused on systematic and structured performance measurement and involved linkages with policy making, resources management, program performance improvement, and other crucial success factors in performance management.

The implementation of development programs and activities was undertaken by ministries and agencies in accordance with the planned strategies.

The implementation of MBS in the early years had positive impact.

However, over time it reverted to budgetary compliance.
ENHANCING MONITORING AND EVALUATION FOR BETTER RESULTS Page 6 ENHANCING MONITORING AND EVALUATION APPROACH TO ADDRESS DEMAND FOR GOOD GOVERNANCE AND ACCOUNTABILITY January 2013 Program managers seldom use MBS as a strategic management tool.

Therefore, attempts to link input efficiency with output and outcome performance were met with challenges.

Implementation and performance were measured by the number of outputs generated, inputs used (expenditure), and physical progress of the development projects.
This has resulted in wasteful expenditure in an attempt to justify greater performance.

“the use of expenditure as one of the performance measures resulted in attempts by agencies to commit wasteful expenditure at year end to justify greater performance.

In addition, the lack of structured monitoring framework, the lack of performance information and insufficient internet capacity hindered managers from undertaking evaluation as mandated under the MBS.
Data for evaluation was only sought after when evaluations were done and not planned from the beginning.

Program managers had difficulty drawing constructive conclusions and to make decisions based on the evaluation findings2.

Deputy Secretary General Dato’ Mat Noor bin Nawi Ministry of Finance, Malaysia 10 September 2012 3.2 Outcome-Based Budgeting Outcome-based Budgeting (OBB) is a strategic management tool designed under the Strategic Reform Initiative (SRI).
It focuses on detailed planning with the requisite horizontal and vertical linkages that contribute to shared or common outcomes.

It advocates a Whole-of-Government approach covering processes of complete cycle from planning to results, at both the national and ministry levels.

The Budget is a central tool through which development results and priorities are realised.
OBB measures results achieved at almost every stage of the project from input application activity completion, outputs delivery and impact achievement.
2 Ministry of Finance, Malaysia, Outcome-based Budgeting and Evaluation: An integrated and holistic approach for improving public sector performance, 2012 ENHANCING MONITORING AND EVALUATION FOR BETTER RESULTS Page 7 ENHANCING MONITORING AND EVALUATION APPROACH TO ADDRESS DEMAND FOR GOOD GOVERNANCE AND ACCOUNTABILITY January 2013 ENHANCING MONITORING AND EVALUATION FOR BETTER RESULTS Page 8 ENHANCING MONITORING AND EVALUATION APPROACH TO ADDRESS DEMAND FOR GOOD GOVERNANCE AND ACCOUNTABILITY January 2013 There are three supporting components: a) The Performance Monitoring and Evaluation (M&E) System is used for systematic and focused program planning, performance monitoring and evaluation, reporting and information utilization for program improvements and policy decision- making; b) The Management Information System (MIS) is used to provide the basis for an effective decision support system at different levels of an organisation; c) An Integrated e-System for Performance Management System - MyResults The dynamics of OBB is the linkage among the three components.
Five ministries were selected for the pilot implementation in 2012: a) Ministry of Finance b) Ministry of Human Resource c) Ministry of Public Works d) Ministry of Education e) Ministry of Transport Government has The identified six transformation levers that for formed developing the OBB model: the basis  A need to shift focus from input utilisation and outputs to outcomes;  A need to addresss the lack of strategic alignment integration and vertical from national priorities down to implementation levels; to  A need identify contributions to common outcomes and to reduce overlapping programs and redundancies; standing  A need to address the long issue of budgeting, dichotomous where the implementation of activities determined along of the and development operating expenditure are reviewed separately and often without proper consultation; lines to  A need improve accountability for results at all levels; of  A need to overcome the lack structured a monitoring and evaluation system, as this represents the a performance based manangement system.
bedrock of Source: MOF ENHANCING MONITORING AND EVALUATION FOR BETTER RESULTS Page 9 ENHANCING MONITORING AND EVALUATION APPROACH TO ADDRESS DEMAND FOR GOOD GOVERNANCE AND ACCOUNTABILITY January 2013 National planning will align with the ministry-level program implementation through the establishment of the Ministry Results Framework (MRF).

The MRF is an integrated results framework comprising planning, budgeting and the basis for developing the monitoring plan which will be used for reporting performance.
The integrated nature of the framework will logically allow the monitoring mechanism to link up with the targets set.

Programs and activities will be carried out according to the strategies identified in the MRF.
ENHANCING MONITORING AND EVALUATION FOR BETTER RESULTS Page 10 ENHANCING MONITORING AND EVALUATION APPROACH TO ADDRESS DEMAND FOR GOOD GOVERNANCE AND ACCOUNTABILITY January 2013 ENHANCING MONITORING AND EVALUATION FOR BETTER RESULTS Page 11 ENHANCING MONITORING AND EVALUATION APPROACH TO ADDRESS DEMAND FOR GOOD GOVERNANCE AND ACCOUNTABILITY January 2013 Source: Ministry of Finance Accountability will be established at the national programs, ministries and activities level through the MRF.
The MRF requires top-down planning approach where each ministry will need to align its programs and activities to the National Development Framework.
ENHANCING MONITORING AND EVALUATION FOR BETTER RESULTS Page 12 ENHANCING MONITORING AND EVALUATION APPROACH TO ADDRESS DEMAND FOR GOOD GOVERNANCE AND ACCOUNTABILITY January 2013 Source: Ministry of Finance 3.3 Internalised Self-Evaluation (ISE) An Internalised Self-Evaluation (ISE) model was developed to enable managers use real-time performance data and respond to performance fluctuations.
The ISE evaluates programs and activities in terms of their appropriateness, effectiveness, efficiency and economic values.

A pilot ISE project involving 20 activities in several ministries was carried out between 1999 and 2004.

Analysis carried out after the pilot implementation found that the ISE brought many benefits to an organisation.

Among the major benefits identified are rich information for decision-making and capacity building.

Hence, ISE can offer options for program enhancement.
ENHANCING MONITORING AND EVALUATION FOR BETTER RESULTS Page 13 ENHANCING MONITORING AND EVALUATION APPROACH TO ADDRESS DEMAND FOR GOOD GOVERNANCE AND ACCOUNTABILITY January 2013 In general, the Ministry of Finance found that ISE are useful 3 because: a) ISE is an on-going development process – it provides real-time data performance; b) ISE promotes organisational learning and contributes to wider knowledge acquisition; c) ISE provides hands-on experience and promotes self-realisation for continuous improvements.
ISE uses four-stage process: a) b) Preliminary planning and assessment stage Evaluation design stage c) Data collection and analysis stage d) Reporting and information utilisation stage ISE promotes three categories of accountability: a) Financial accountability b) Management accountability c) Program accountability 3 Ministry of Finance, OBB Project Team, Sakeri Kadir ENHANCING MONITORING AND EVALUATION FOR BETTER RESULTS Page 14 ENHANCING MONITORING AND EVALUATION APPROACH TO ADDRESS DEMAND FOR GOOD GOVERNANCE AND ACCOUNTABILITY January 2013 3.4 MyResults A comprehensive online integrated system called MyResults, provides users with framework structures for monitoring and evaluation.
MyResults provides for budget submission, budget review and verification, performance monitoring and reporting.

Each ministry will have an access to information about its program objectives, resources utilisation, activity completion, output generation, outcomes and impact achievement from the system, for evaluation purposes.

MyResults can also be used by planners and managers as a basis to conduct the ISE on program or activity.
The Push for Results delivery and Performance information such as completion of activities, of outputs the achievement of outcome will be to various levels of decision makers on a timely basis.

reported OBB is relevant to give decision makers a more accurate and relevant information, where quick and prompt findings are needed for budgetary decisions.

OBB can support the budgetary process by helping either to identify programs or components of programs which can potentially be modified, expanded or cut.
can assist the OBB process budgetary identify by savings improving the efficiency of specific service.

The early warning system or for dashboard management has been to generate developed information on shortfalls in resource utilisation, over expenditure, variances in achievement and the identification of weaknesses.

This enabled management to pinpoint the problem area and its root causes so that the neccessary and remedial actions can be taken fast.
ENHANCING MONITORING AND EVALUATION FOR BETTER RESULTS Page 15 ENHANCING MONITORING AND EVALUATION APPROACH TO ADDRESS DEMAND FOR GOOD GOVERNANCE AND ACCOUNTABILITY January 2013 4.0 Monitoring and Evaluation – A Critical Tool Monitoring and Evaluation (M&E) are not standalone activities.

It is a critical tool of a results-based management system.
M&E is basically a performance management tool that integrates all key components of an integrated results-based management system - development planning, budgeting and resources allocation within an evidence orientated environment.

Monitoring tracks key performance indicators at different program levels so that intervention can be done to keep projects and programs on track.

There is currently no centralised and integrated monitoring system.
Projects implementation are monitored by the Implementation and Coordination Unit (ICU).

The ministries and agencies developed and maintained their own internal monitoring system.

Monitoring and data collection activities are carried out manually.

Hence, data are outdated and inaccurate to respond to performance fluctuations.

Evaluation demonstrates accountability and assesses the achievement or failure of a program.
Evaluation of program performance and results is a mandatory exercise undertaken at least once every five years.

However, program evaluation is often not carried out as mandated under the MBS.

It is carried out on an ad-hoc basis.

This is due to factors, including, lack of structured monitoring mechanism resulting in a weak management information system, lack of enforcement by the central agencies, and also possibly lack of expertise in monitoring and evaluation.
As a result of insufficient linkage between the planning and budgeting processes, it posed a major challenge to evaluate and determine the outcome.

Countries, such as, Chile, Uganda and Vietnam, have undertaken serious efforts to implement results-based approaches to their development policies.

Studies have shown that these efforts are producing tangible benefits and the quality of living has improved.

Take for instance, the Uganda Tracking Public Expenditures project in the Education Sector.
ENHANCING MONITORING AND EVALUATION FOR BETTER RESULTS Page 16 ENHANCING MONITORING AND EVALUATION APPROACH TO ADDRESS DEMAND FOR GOOD GOVERNANCE AND ACCOUNTABILITY January 2013 When the Government of Uganda decided to publish the monthly inter- governmental transfers of public funds through newspapers and radio and also required primary schools to post information on inflows of funds for all to see, follow- up expenditure surveys showed that the flow of non-wage funds improved from 13% reaching schools in 1991 – 1995 to about 80% - 90% reaching schools in 1999 and 2000.
Japan’s Transparent, Accountable, People-Orientated, Paricipatory, Efficient, Effective, Equitable Sustainable Budgeting System (or known as TAP2E3BS) has been increasingly adopted in a number of cities all over Japan.
Under the budgeting system, city residents have required the city mayor to formulate the city budgets in accordance with the needs and requirements identified through rounds of discussion among individual citizens and civil society groups including NGOs and community-based organisations in the city.
It has resulted a city’s long term development plan prepared every 10 years with annual and mid-term reviews conducted regularly by the municipal office in consultation with the municipal planning committee which is represented by various civil society groups voluntarily coming together for the purpose.
In addition, it was found that TP2E3BS brought the following changes: a) Citizens have a stronger sense of ownership – to keep the city safe and clean; b) Formation of watch groups to ensure no wastage on administration expenditures and making sure proper allocation of budget; c) Citizens pay greater attention to people in need by providing public assistance and skill improvement programmes for the physically and mentally disadvantaged and financially poor; d) Citizens are more politically conscious of their responsibilty to ensure the right candidates for assemblyman; ENHANCING MONITORING AND EVALUATION FOR BETTER RESULTS Page 17 ENHANCING MONITORING AND EVALUATION APPROACH TO ADDRESS DEMAND FOR GOOD GOVERNANCE AND ACCOUNTABILITY January 2013 5.0 Summary and Conclusions The public sector’s role is no longer confine to delivering essential services.
Instead, it is a prime mover and catalyst for change.

In its move to implement a results-based management approach, there is a need to take a holistic and integrated approach to manage the budgeting system.

The core of the public service performance now lies on the ability of the public sector to manage public funds diligently and deliver results.

The introduction of the OBB in the 10MP is a significant milestone in the history of Malaysia’s development plan.
The 10MP marks the second last phase of a 5-year development plan for Malaysia as we forge ahead towards Vision 2020.

Special attention is needed on the detailed planning to drive the country forward.

The ISE model and MyResults application will be widely deploy under OBB as they provide program managers with updated information through systematic monitoring and dashboarding.
ENHANCING MONITORING AND EVALUATION FOR BETTER RESULTS Page 18 ENHANCING MONITORING AND EVALUATION APPROACH TO ADDRESS DEMAND FOR GOOD GOVERNANCE AND ACCOUNTABILITY January 2013 Challenges on the implementing OBB: a) The need for stakeholders and top management’s support and commitment in planning, monitoring and evaluation; b) The need for better understanding of OBB to internalise evaluation function into an organisational culture; c) The need to support continuous learning and critical program review to improve decision-making; d) The need for continuous capacity building to keep the evaluation momentum going; e) The need to have a sufficient resources allocated for evaluation; and f) Greater need for well structured monitoring and evaluation framework.
The implementation of OBB will be complemented with three key initiatives.

The initiatives are: a) Introduce a two-year rolling cycle within a 5-year planning horizon It acknowledges the dynamic nature of development planning and enables the public sector commits within its financial capacity and at the same time, allows flexibility in the reprioritisation of programs and projects, and also is open for new opportunities.
Programs and projects are allocated budget based on a two-year rolling cycle, to ensure seamless planning and implementation.

b) Embedding an integrated approach to planning Better coordination across agencies at an earlier stage will ensure effective use of resources.

An integrated approach will require all stakeholders to examine economics, social and environmental costs and benefits prior to project selection.
This approach takes a holistic assessment of existing facilities, the surrounding projects, National Physical Plan, State Structure Plans and Local Plans as a guide in planning and sharing of resources, particularly land use, infrastructure, utilities and services; ENHANCING MONITORING AND EVALUATION FOR BETTER RESULTS Page 19 ENHANCING MONITORING AND EVALUATION APPROACH TO ADDRESS DEMAND FOR GOOD GOVERNANCE AND ACCOUNTABILITY January 2013 c) Regular monitoring and evaluation of outcome Managing information systems will be strengthen to enable systematic and regular performance monitoring, evaluation and reporting information on progress.
This will ensure remedial actions are taken timely and when necessary.

Funding of future programs and projects will depend on the delivery of program milestones and outcomes as measured by KPIs.

ENHANCING MONITORING AND EVALUATION FOR BETTER RESULTS Page 20 ENHANCING MONITORING AND EVALUATION APPROACH TO ADDRESS DEMAND FOR GOOD GOVERNANCE AND ACCOUNTABILITY January 2013 REFERENCES 1.
Mohd Sakeri Abdul Kadir, (2012), Institutionalizing Evaluation in Organisations: Lessons from an Internalised Self-Evaluation Approach.

Kuala Lumpur: Ministry of Finance.

2.

Jabatan Perdana Menteri, (2011, 18 April 2011), Teks ucapan Majilis Perasmian Bengkel Programming Outcome-based Budgeting.

Kuala Lumpur: Jabatan Perdana Menteri.

3.

Asia Pacific CoP-MfDR, (2011), Framework for Results-Based Public Sector Management and Country Cases.

4.
4.

Perbendaharaan Malaysia Pasukan Projek OBB, (2012, June 2012).

Pelaksanaan Outcome-Based Budgeting, Persidangan Penolong Akauntan Sektor Awam Kali Ke-10.

5.

Kementerian Kewangan, (2012), Bajet Berasaskan Outcome, Pekeliling Perbendaharan Bil.

2 Tahun 2012.

6.

Ryokichi Hirono, (2012), Reflections and Moving Forward in Results-Based Policy Review and Budgeting: Case of Japan.

Kuala Lumpur.

7.
Kuala Lumpur.

7.

Stout, Susan, (2012), Approach to Policy and Program Performance Improvement under an Evaluation-Orientated Environment.

Kuala Lumpur.

8.

Richard Barahim, (2012), Integrated e-System for Whole-of-Government Performance & Budget Management, Kuala Lumpur: Ministry of Finance.

9.

MyResults Malaysia at website: www.myresults.treasury.gov.my ENHANCING MONITORING AND EVALUATION FOR BETTER RESULTS Page 21