Source document: https://docs-lawep.s3.us-east-2.amazonaws.com/1698927899287.pdf (United Kingdom)
Multilateral Aid Review Update Driving reform to achieve multilateral effectiveness December 2013 Organisations/Funds included in the Multilateral Aid Review Update ■■ The African Development Fund ■■ International Fund for Agricultural Development (IFAD) (AfDF) ■■ The Asian Development Fund (AsDF) ■■ The Caribbean Development Bank (CDB) ■■ Central Emergency Response Fund (CERF) ■■ The Climate Investment Funds (CIFs) ■■ The Development Programmes of the Commonwealth Secretariat (CommSec) ■■ European Bank for Reconstruction ■■ ■■ ■■ ■■ International Finance Corporation (IFC) International Federation of the Red Cross (IFRC) International Organisation for Migration (IOM) Joint United Nations Programme on HIV/AIDS (UNAIDS) ■■ Office of the High Commissioner for Human Rights (OHCHR) and Development (EBRD) ■■ United Nations Peacebuilding Fund ■■ European Commission Budget (EC’ion Budget) ■■ European Commission Humanitarian (PBF) ■■ The Private Infrastructure Development Group (PIDG) Aid and Civil Protection (ECHO) ■■ United Nations Development ■■ European Development Fund (EDF) ■■ Food and Agriculture Organisation (FAO) ■■ The Global Alliance for Vaccines and Immunisation (GAVI) ■■ Global Environment Facility (GEF) ■■ The Global Fund to Fight AIDS, TB and Malaria (GFATM) ■■ Global Facility for Disaster Reduction and Recovery (GFDRR) ■■ Global Partners in Education (GPE) ■■ ■■ ■■ Inter-American Development Bank (IADB) International Committee of the Red Cross (ICRC) International Development Association (IDA) Programme
 (UNDP) ■■ United Nations Educational, Scientific and Cultural Organisation (UNESCO) ■■ United Nations Children’s Fund (UNICEF) ■■ United Nations Population Fund (UNFPA) ■■ United Nations High Commission for Refugees (UNHCR) ■■ United Nations Entity for Gender Equality and the Empowerment of Women (UN WOMEN) ■■ UNITAID ■■ United Nations Office for the Coordination of Humanitarian Affairs (OCHA) ■■ World Food Programme (WFP) ■■ World Health Organization (WHO) Contents Ministerial foreword Executive summary
 Chapter 1 – Introduction Chapter 2 – The multilateral environment Chapter 3 – MAR Update methodology Chapter 4 – Findings of the MAR Update Chapter 5 – The use of the MAR Update Annex 1 – UK Multilateral Aid Review terms of reference Annex 2 – Assessment frameworks Annex 3 – MAR Update summary assessments Annex 4 – External reviewers’ commentary 5 7 11 15 21 33 88 93 97 108 183 1 DRIVING REFORM TO ACHIEVE MULTILATERAL EFFECTIVENESS Tables Table 1 Groupings of multilateral organisations Table 2 Selected multilateral results Table 3 MAR assessment framework Table 4 Value for money for UK aid funding of individual multilateral organisations Table 5 Reform priorities by multilateral organisation Table 6 Component progress ratings Table 7 Overall progress ratings Table 8 Score changes in the MAR Update Table 9 Detailed results for special measures agencies 16 19 23 24 25 27 28 36 37 Table 10 Summary of main reforms and outstanding issues by organisation type 66 Table 11 Spider diagrams Charts Summary chart Chart 1 MAR value for money and MAR progress ratings Total Official Development Assistance (ODA) from DAC countries to selected multilateral organisations 2011 Chart 2 ODA by country 2011 Chart 3 UK core multilateral ODA 2012 Chart 4 DFID model of value for money Chart 5 Evidence quality by MAR Update progress rating Chart 6 Multilateral organisations ranked by average MAR Update progress rating Chart 7 MAR value for money and MAR Update progress ratings across the
 multilaterals Chart 8 MAR scores for components assessed in the MAR Update Chart 9 Summary of progress by original MAR score Chart 10 Progress by reform component Chart 11 MAR scores and MAR Update progress ratings for financial resources management 69 9 18 18 19 21 32 34 35 40 41 42 43 2 Chart 12 Progress ratings for financial resources management by organisation type Chart 13 MAR scores and MAR Update progress ratings for partnership behaviour Chart 14 Progress ratings for partnership behaviour by
 organisation type Chart 15 MAR scores and MAR Update progress ratings for contribution to results Chart 16 Progress ratings for contribution to results by organisation type Chart 17 MAR scores and MAR progress ratings for strategic and performance management Chart 18 Progress ratings for strategic and performance management by organisation type Chart 19 MAR scores and MAR Update progress ratings for transparency and accountability Chart 20 Progress ratings for transparency and accountability by organisation type Chart 21 MAR scores and MAR progress ratings for cost and value consciousness Chart 22 Progress ratings for cost and value consciousness by organisation type Chart 23 MAR score and MAR Update progress ratings for gender Chart 24 Progress ratings for gender by organisation type Chart 25 MAR score and MAR Update progress ratings for fragile contexts Boxes Box 1 Box 2 Box 3 Box 4 Theory of change Food and Agriculture Organization (FAO) – making progress from “special measures” The Global Partnership for Education (GPE) – increasing transparency, predictability and flexibility in the allocation of its funding to developing countries International Federation of Red Cross and Red Crescent Societies (IFRC) – working effectively with the private sector to mitigate risks in flood prone communities Box 5 African Development Bank (AfDB) – tracking its performance to further improve its contribution to Africa’s development 44 46 47 49 50 52 53 55 56 57 58 60 61 63 29 39 45 48 51 3 Box 6 Box 7 Box 8 Box 9 The Global Fund to fight AIDS, TB and Malaria (GFATM) – strategically redirecting resources to areas where they will have the greatest impact The Joint United Nations Programme on HIV/AIDS (UNAIDS) – strategically managing running costs and supporting partners on value for money International Development Association (IDA) making progress on gender mainstreaming The Private Infrastructure Development Group (PIDG) – investing in much needed infrastructure in fragile
 states 54 59 62 64 4 Ministerial foreword It’s important that whether we give a pound directly, or through another organisation, our development budget is having the greatest possible impact on the lives of the poorest people in the world.
That is why the coalition government established the Multilateral Aid Review to take a long, hard look at the value for money offered by the 43 different multilateral organisations we were supporting.

This review had a major impact on our spending decisions.

While some high performing organisations received considerable increases in funding, some poor performing institutions had their funding stopped altogether.
The review also highlighted the strengths and weaknesses of multilateral organisations, and enabled us to give organisations a clear list of future priorities and essential reforms.

In addition, the review spurred an important debate on multilateral effectiveness, amongst the whole development community.
The original Multilateral Aid Review and this update make it clear that multilateral organisations continue to play a key role in reducing poverty around the world and in emergencies: the International Committee for Red Cross distributed food rations to 1.5 million people across Syria in 2012, and supported local water boards to provide clean drinking water to 12.5 million people.
Multilaterals can support the empowerment of girls and women: 17 countries where UN Women works adopted new or improved legislation to stop violence against women in 2012, adding to the 28 which had already done so, while UNICEF and UNFPA jointly supported communities in 15 countries to declare they were abandoning female genital cutting between 2008 and 2012.

Multilateral organisations are also helping to build the infrastructure that unlocks economic growth and jobs in developing countries.
Between 2010 and 2012 the African Development Bank ensured over 34 million people gained access to new transport connections and over 7.9 million people gained new or improved electricity connections.

This is all vital work but we need to keep up the momentum, making even greater progress to ensure all our investment is in the right places, on the right things and being spent in the right way.
Since becoming Development Secretary I have significantly increased DFID’s focus on driving efficiency and value for money on behalf of the taxpayer.

I have introduced ministerial sign off for all projects over £5 million.

I am also driving improvements in our procurement approach to get better deals for our money.

I have introduced greater transparency to show how development investment is delivering results.
As part of that, we will continue to review the effectiveness of multilateral institutions, to ensure every pound of our budget is being stretched as far as it can go.

5 Ministerial foreword This update shows that all organisations have made improvements over the two years since the original Multilateral Aid Review.

Some organisations were asked to make urgent reforms if they wanted our investment to continue, and they are making progress.
Other multilaterals, which were already doing well, are doing even better.

However not all organisations have made the changes we expected of them, and there is disappointing progress on some important thematic areas.

This includes the empowerment of girls and women, driving greater transparency and accountability, and keeping administration costs low.

These are really critical areas if we want value for taxpayer money and to lift the poorest people out of poverty for good.
The Multilateral Aid Review was never intended to be a one-off exercise and this update demonstrates our ongoing commitment to the multilateral effectiveness agenda.

Over the next year we will use the evidence from this update to work with multilaterals and other partners to bring about further change.

There will be a particular focus on these important thematic areas where progress has been too slow.
DFID will also be working with other governments and our international networks to improve our evidence base for the next review in 2015.

We owe it to developing countries, and the British taxpayer, to ensure every pound of the development budget is spent wisely, and I look forward to working with our development partners to take this forward.

rt Hon.

Justine Greening, secretary of state for international development 6 executive summary 1.
In 2011, the UK coalition government published the Multilateral Aid Review (MAR) – a systematic assessment of the performance of the multilateral organisations that the UK funds for development.

This report gives the results of the MAR Update, which has scrutinised the progress multilateral organisations are making on the important areas of reform that were identified as a result of the MAR.
The Department for International Development’s (DFID) significant investment in multilaterals is an essential part of the UK’s work to end extreme poverty.

2.

The MAR and this Update are an essential way of ensuring that we achieve value for money for the UK taxpayer from our significant investment in multilaterals.

In 2012/13, DFID gave around £3.25 billion of UK aid as core funding through the multilateral organisations.
Of this about £1,024 million went through the World Bank, £359 million through the global funds and £338 million through the United Nations (UN) and the remaining third went through the European Commission.

The MAR in 2011 3.

43 organisations were assessed under the MAR in 2011.

Nine were deemed to offer very good value for money for UK aid, 16 to offer good value for money, nine to offer adequate value for money, and nine to offer poor value for money.
The findings of the review were used to make decisions about UK aid funding.

Many very good and good value for money organisations received increases in UK aid funding.

At the same time the UK also withdrew voluntary core aid funding from four poor value for money organisations, and placed four others in “special measures”, demanding urgent improvements in performance.

4.

This Update shows there has been reform.


We asked some organisations to make urgent improvements to secure our continued investment, and they are making progress.

Other organisations which were already doing well are doing even better.

But we are also aware of slower progress.

Some organisations have not made the changes we expected of them, and there is disappointing progress on some important thematic areas.
This is a real concern to us as they include critical areas such as empowering girls and women, increasing transparency and accountability, and controlling administration costs.

The MAR Update 5.

This Update assesses the progress of the multilateral organisations against these reform priorities over the two years since the publication of the MAR in 2011.

It is not a full refresh of the MAR; each organisation has only been assessed against the specific reform priorities they were set.
37 organisations were assessed in three tranches, from late 2012 with the final tranche completed in October 2013.

7 executive suMMary 6.

The MAR attracted significant attention, both in the UK and internationally.

This commentary has been broadly positive.

It also points us to ways in which our work in this area can be improved further.
We have taken these comments into account in the MAR Update, where this has been possible: for example, we have clarified the assessment framework to meet the needs of multilateral organisations which set standards, engaged more with multilaterals during the process and increased our quality assurance.

7.

All organisations assessed have made some progress against the agreed reform agenda, and about half of them have done better than this.
Importantly, all of the UN “special measures” agencies are making steady progress towards emerging from “special measures”.

Six multilateral organisations have made enough progress in at least one reform area to increase their MAR score for that component.

Only one organisation, UNAIDS, has moved to a higher value for money category (in this case from adequate to good value for money for UK aid) – not surprising given that the MAR took place only two years ago.

8.
8.

The summary chart shows both the value for money assessment under the 2011 MAR, and the progress ratings from the MAR Update.

As a general rule, both very good and poor value for money organisations have made good progress on reforms, while adequate value for money organisations have tended to show less progress.

We are also interested in progress by reform area.


There has been good progress on key components such as financial resources management, partnership behaviour, and contribution to results, but there has been disappointing progress on the equally important components of gender, transparency and accountability, and cost and value consciousness.
8 summary chart: Mar value for money and Mar progress ratings GAVI PIDG ICRC ECHO AsDF EDF UNHCR CERF OCHA IFRC UNICEF IDA GFATM AfDF EBRD IFC PBF CIFs IFAD GEF WFP GPE GFDRR UNDP UNTAID UNFPA IADB EC budget OHCHR CDB UNAIDS UNIFEM/ UNWomen IOM WHO FAO UNESCO CommSec “Some” progress in most areas “Reasonable” progress in most areas 2.00 2.50 Average MAR Update progess rating 3.00 3.50 3.50 3.00 2.50 2.00 e r o c s R A M e g a r e v A 1.50 1.50 Key Very good value for money Adequate value for money Good value for money Poor value for money Progress in the MAR Update led to a score change in one or more components 9.
Following the MAR Update, we have continued to invest aid resources in the very good and good value for money organisations, with closer links to burden share and the need for other donors to match our commitment to the multilateral system, and stronger ties between funding and reform where progress is slower.
We have also continued to support adequate value for money and “special measures” organisations where they are making good progress on UK reform priorities, while keeping contributions flat or falling for those poorer performing organisations that are not reforming fast enough.

10.

Our work on multilateral effectiveness does not end with this MAR Update.

Tackling slow-moving and critical reforms, together with our partners, will be a key priority for the UK over the next few years.
We will also be working closely 9 DRIVING REFORM TO ACHIEVE MULTILATERAL EFFECTIVENESS executive suMMary with those organisations which have made relatively less progress, to ensure that more is done.

11.

We will carry out a full reassessment of value for money through a MAR in 2015, to inform future funding decisions and the reform agenda going forward.

This next MAR will also seek to address issues raised with us by multilateral organisations, and the informed commentary by our Parliament and others.
Over the next year, we will be working with others to ensure that the methodology and evidence base for future multilateral effectiveness assessments continues to improve and gives us a sound basis for making important decisions about where best to invest to secure humanitarian and development outcomes.

12.
12.

We would like to thank all those who have been involved in this MAR Update, particularly the multilateral organisations themselves, which have provided considerable evidence to enable this work to take place.

10 chapter 1 Introduction 1.

In 2011 the UK coalition government published its Multilateral Aid Review (MAR) – a systematic assessment of the performance of the multilateral organisations that the UK funds for development and humanitarian objectives.
This report gives the results of the MAR Update, which has scrutinised the progress multilateral organisations are making on the important areas of reform that were identified as a result of the MAR.

2.

The Department for International Development (DFID) leads the UK’s work to end extreme poverty.
DFID’s significant investment in multilaterals is a core part of our work to end the need for aid by creating jobs, unlocking the potential of girls and women and helping to save lives when humanitarian emergencies hit.

The MAR and this Update are an essential way of ensuring that we achieve value for money for the UK taxpayer from our development spending.

3.
3.

This MAR Update final report1 contains, in this first chapter an overview of the original MAR process and the public commentary on it, and explains the background to the MAR Update.

Chapter 2 gives an overview of DFID engagement with multilateral organisations.

Chapter 3 explains the methodology for the MAR Update and how it was developed.

The main body of the report is in chapter 4.


This gives the results for each of the multilateral organisations assessed, and analyses overall patterns of progress, across multilateral organisations and by component area.

The final chapter looks forward, and outlines the impact of this Update process and the way in which we intend to develop our work further.

The terms of reference for the MAR Update are in annex 1.

The MAR Update 4.
The MAR Update 4.

The MAR Update has been undertaken to see whether the reform we have asked for is taking place.

We understand that it can take time to implement change, and even more time to see its impact in improved performance in developing countries.

We believe though that it is right to keep our attention on the changes we are seeking, to recognise what has been done and to celebrate achievement – as well as to continue to press for improvement.
We do this because we know multilaterals are essential for development: whether or not they are doing this well, makes a vast difference to the lives of the many people who depend on them.

1 A short MAR Update interim report was published in July, which explained the Update process and gave the results for the first 24 agencies assessed.

This interim report did not provide any analysis, given that the process was underway and the results for all agencies were not yet available.
This full report essentially supersedes the earlier interim report.

11 introduction 5.

The Update was carried out in three tranches, from late 2012 with the final tranche being completed in October 2013.

The multilateral agencies involved under the Update have seen their full assessments in advance, and been given the opportunity to provide responses for publication on the DFID website.
These can be found at: https://www.gov.uk/government/collections/multilateral-aid-review Background – the Multilateral Aid Review 6.

The MAR assessed whether the UK’s core investment in multilateral organisations was value for money in achieving UK development and humanitarian objectives.
The organisations assessed under the MAR included a wide range of multilateral organisations: multilateral development banks and UN agencies, private sector instruments, global funds for health, the environment, and education, and humanitarian organisations.

These organisations were assessed against one objective assessment framework, which looked at the contribution an organisation made to UK development objectives and its organisational strengths, resulting in one overall value for money score.

7.
7.

The MAR had real impact.

DFID withdrew core aid funding from four organisations (the International Labour Organization (ILO), UN Human Settlements Programme (UN-HABITAT), UN Industrial Development Organization (UNIDO), UN International Strategy for Disaster Reduction (UNISDR) which had been assessed as being poor value for money in delivering UK aid objectives.
A further four organisations with a similar assessment (FAO, UNESCO, IOM, and CommSec) were placed in “special measures”, meaning that DFID demanded an urgent improvement in performance if the UK was to continue providing core funding2.

8.

The MAR was also taken into account in determining the amount of UK funding for other multilaterals.
In addition the strengths and weaknesses identified in the MAR resulted in the UK setting reform priorities in the areas which the expert DFID leads considered to be most significant and the most likely to have an impact across the work of the organisation.

Commentary on the Multilateral Aid Review 9.

The MAR attracted significant attention, both in the UK and internationally.
In the UK, the National Audit Office (NAO)3 scrutinised the process, and in the UK Parliament, both the Public Accounts Committee (PAC)4 and the International 2 The other organisation assessed as being poor value for money for the UK taxpayer was UNIFEM; this has merged into a new organisation UN Women.

3 National Audit Office (2012): The Multilateral Aid Review.
4 http://www.nao.org.uk/report/dfid-the-multilateral-aid-review/ Public Accounts Committee Report: The Department for International Development: The Multilateral Aid Review http://www.parliament.uk/business/committees/committees-a-z/commons-select/public-accounts- committee/news/multilateral-aid-review/ 12 Development Committee (IDC)5 have produced reports.

We welcome this interest and the informed commentary that has resulted.

10.
10.

The NAO concluded that DFID had shown international leadership in publically assessing multilateral organisations and found DFID had taken a logical approach to the methodology for the MAR.
It also noted areas where the MAR process could be improved: for example through considering further the position of those multilaterals which focus primarily on setting international standards and policies, and through the collection of more evidence from developing countries, and more consistent data from multilaterals themselves.

11.
11.

The PAC found the “Multilateral Aid Review was a significant step forward in assessing the performance of multilateral organisations” and welcomed the focus on costs and results delivered.

The PAC also encouraged attention to the multilateral system, including a better understanding of the way in which multilateral agencies’ roles related to each other, and examining whether there were gaps and overlaps, to support investment decisions.
The PAC also noted that a number of other governments had followed the UK’s example in undertaking similar assessments of multilateral organisations and that this was causing significant work for multilateral organisations; it therefore encouraged greater co-operation and collaboration between governments before a further full MAR in 2015.

12.

The IDC undertook their inquiry while the MAR Update process was underway.
They were therefore able to comment on some of the changes that had been made to the process.

It found that the MAR Update had made a “step in the right direction” by refining the assessment framework for those agencies which set international standards, and that the increased engagement between DFID and multilateral organisations during the process had been welcomed.
It also reflected on the issue of the burden caused by multilateral assessment processes and found that DFID had been successful in keeping this proportionate.

Like the PAC, it encouraged DFID towards more international collaboration, but added that joint donor assessments did not lessen the importance of, or the need for a UK MAR – as it was important to assess the contribution of multilaterals to UK development objectives.

13.
13.

Each of these reports raised the importance of collecting robust evidence and data about the performance of multilaterals, particularly in developing countries.

For example, the fact that multilateral organisations do not collect core data in a consistent manner was recognised as a constraint.

5 International Development Committee.


Multilateral Aid Review – Fourth Report of the Session 2012-2013. http://www.parliament.uk/business/committees/committees-a-z/commons-select/international-development- committee/news/substantive-press-notice-mar/ 13 DRIVING REFORM TO ACHIEVE MULTILATERAL EFFECTIVENESS introduction 14.

These challenges and others contained in the report will be addressed by DFID.
We are, for example, working with other donors to define a common approach to collecting administrative data from multilateral organisations.

Where it has been possible to address particular recommendations in the MAR Update process, we have done so (for more details, see chapter 3).

Where recommendations relate more to the development of a full MAR process, they will be taken into account as we work towards a full MAR in 2015.
We are very grateful for this informed and thoughtful commentary which we will consider seriously as we develop our thinking on multilateral effectiveness further.

14 chapter 2 The multilateral environment 1.

The MAR and this Update are undertaken to ensure that we are achieving value for money for the UK taxpayer through our investment in multilateral organisations.
This chapter presents the latest figures available for the funding of multilateral organisations, by the UK and all donors, and information about what is delivered as a result of that funding.

Multilateral organisations 2.

The multilateral organisations were established to enable national governments to work together on particular issues.

They are an essential part of the international system for humanitarian and development aid.

3.
3.

The MAR focused on those organisations which DFID funds and works with most closely.

These included a wide range of organisations with very different roles in development – for example, to respond to humanitarian need, to develop infrastructure to support economic growth, or to ensure that particular diseases are tackled in line with the best available evidence.

We placed these organisations into seven different groups in the MAR in order to analyse outcomes.
Table 1 sets out the organisations assessed in the MAR Update using the same groupings.
15 tHe Multilateral environMent table 1: Groupings of multilateral organisations Multilateral organisations number of organisations Multilateral development banks, with a focus on concessional funds AfDF, AsDF, CDBa, IDA Development finance institutions and funds, supporting private sector development Global funds for health, education and climate change EBRD, IFC, PIDG CIFs, GPE, GAVI, GEF, GFATM, UNITAID Humanitarian organisations (including UN) CERF, ECHO, GFDRR, ICRC, IFRC, IOM, OCHA, UNHCR, WFP UN organisations (excluding humanitarian) FAO, IFAD, OHCHR, PBF, UNAIDS, UNDP, UNESCO, UNFPA, UNICEF, UN Women, WHO European Commission (excluding humanitarian) European Commission budget instruments, EDF Other Total CommSec, IADB 4 3 6 9 11 2 2 37 a We primarily focus on the concessional funding window of the Caribbean Development Bank, the Special Development Fund.
4.

There have been some changes since the MAR.

In that exercise, we included 43 multilateral organisations – those which regularly received more than £1 million of core aid funding and the multi-donor trust funds which receive fully flexible funding from DFID.

We have assessed 37 multilateral organisations during the MAR Update process.

Of the original 43, five6 no longer receive core aid funding from DFID, and one has concluded its work7.
In addition, one organisation, UNIFEM, has been merged into a successor organisation, UN Women, which we have assessed in this MAR Update.

Finally, the Education for All – Fast Track Initiative (FTI) is now called the Global Partnership for Education (GPE).

Funding to the multilateral organisations 5.
Multilateral organisations receive funds from national governments as core aid funding (which is unrestricted, and may be used as the organisation thinks best, so long as it is in line with its mandate and agreed by the governing body) and as non-core aid funding (which must be spent on specified programmes, to achieve 6 7 The International Labour Organization, the United Nations Industrial Development Organisation, the United Nations International Strategy for Disaster Reduction, the United Nations Human Settlements Programme and the United Nations Environment Programme.
Expanded Delivering as One Funding Window for the achievement of the MDGs (EFW).

16 particular objectives or in named countries or regions).

Core aid funding is provided through membership fees, capital contributions and additional un- earmarked voluntary contributions.

The UK also funds multilaterals as part of its bilateral programming, mostly through country programmes but also as part of headquarters programmes.
Decisions about such funding are taken separately to our decisions about core aid funding.

They depend on a detailed consideration of the objectives that the country offices want to achieve and the partners available to deliver these, as well as taking into account evidence provided by the MAR.

They result in a decision about the best delivery route and partners, a process which is then tested through detailed business cases.

6.

The MAR set out our reasons for investing in multilaterals.
Multilateral organisations are able to work in a wider range of countries than individual donor governments, and often have the legitimacy to work in politically sensitive situations.

In doing so, they enable donor governments, such as the UK, to support humanitarian and development work in a far wider range of countries.

The scale of their operations enables them to develop and deploy specialist technical expertise, and to deliver global public goods.
Their ability to lead and co-ordinate humanitarian and development functions, can increase the impact of development interventions and reduce the transaction costs for both recipient and donor governments.

Some multilateral agencies also play an important role in developing international policies and technical guidance, and brokering international agreements, which guide and support the development activities of other multilateral and bilateral agencies.

Global funding for multilateral agencies 7.


Many international donors give funding to the multilateral agencies.

Chart 1 sets out the multilateral organisations which receive this funding, and chart 2 shows where the funding comes from.

The majority of funding is given to the European Commission (EC), which is both a development actor and a donor providing grants to other multilateral organisations, and the World Bank.

Chart 1 distinguishes between funding given as core aid and non-core aid.
Those which have the largest presence in developing countries making them natural and expert delivery partners (for example, the World Bank and UNDP), or are humanitarian organisations which receive additional funds targeted at specific emergencies (UNICEF, WFP and UNHCR).
17 DRIVING REFORM TO ACHIEVE MULTILATERAL EFFECTIVENESS tHe Multilateral environMent chart 1: total official development assistance (oda) from dac countries to selected multilateral organisations 20118 ) s n o i l l i m D S U ( 1 1 0 2 , A D O l a r e t a l i t l u M 15,000 12,000 9,000 6,000 3,000 0 O H C H R U N ES C O IFA D IA D B U N G A VI FA O U NFPA W H O Multilateral core AfD B AsD B U N H C R U N D P Multilateral non-core U NIC EF G FAT M W FP E U W B 8.
Chart 2 shows that the UK was the third largest aid donor in 2011, after the USA and Germany, and very closely followed by France and Japan.

Donors choose to make a mix of investments, giving core aid funding to multilateral organisations as well as investing in their own bilateral programmes.

This chart also shows the non-core funding that governments give to multilateral agencies through their bilateral programmes.
chart 2: oda by country 20119 1 1 0 2 , s t n e m e s r u b s i d A D O t e N ) s n o i l l i m D S U ( 35,000 30,000 25,000 20,000 15,000 10,000 5,000 0 D en m ark S witzerland Spain Czech Republic A ustria K orea Luxe m bourg Portugal Finland Belgiu m Iceland N e w Zealand Greece Ireland U nited States G erm any N orw ay A ustralia C anada N etherlands France S w eden Japan U nited Kingdo m Italy Bilateral Multilateral core Multilateral non-core Source: OECD DAC aggregate statistics, multilateral aid (core and non-core) by DAC donor, 2007-2011.
8 9 OECD DAC aggregate statistics, net ODA disbursement by DAC donor, 2011.

18 UK funding 9.

In 2012 the UK gave approximately £3.27 billion of ODA as core funding through multilateral organisations10.

Just over a third of this went through the EC instruments11.

Of the remainder, approximately £797 million went through the World Bank, £468 million through the global funds and £460 million through the UN.
Chart 3 shows this distribution of UK core multilateral ODA for 2012. chart 3: uK core multilateral oda 2012 UN and Commonwealth – £460 million Other international financial institutions – £258 million World Bank – £797 million EC instruments – £1,169 million Global funds – £468 million Other multilaterals – £118 million 10.

These are considerable investments which deliver significant development results.
Selected results information is shown in table 2. table 2: selected multilateral results United Nations Children’s Fund (UNICEF) ■■ In 2012 UNICEF contributed to 1,775 communities declaring abandonment of female genital cutting ■■ In 2012 in emergency, humanitarian, recovery or fragile situations, UNICEF assisted an estimated 2.1 million severely malnourished children through therapeutic feeding programmes World Health Organization (WHO) ■■ Since 1990 the number of children dying from preventable causes has fallen from around 12 million to around seven million and the global HIV, TB and malaria epidemics have been halted and reversed United Nations Development Programme (UNDP) ■■ In 2011-12 UNDP supported implementation of large-scale livelihood interventions in 15 crisis-affected settings, improving over two million livelihoods ■■ In 2012, supported electoral cycle management in over 60 countries, ensuring the participation of 17.3 million members of underrepresented and marginalised groups and strengthening 30 national electoral commissions 10 Source: Statistics on International Development, 2013.
11 European Commission (EC) instruments is used to refer to both the European Development Fund (EDF) and to external action instruments under Heading Four of the Multiannual Financial Framework in particular, the Development Co-operation Instrument (DCI).
19 DRIVING REFORM TO ACHIEVE MULTILATERAL EFFECTIVENESS tHe Multilateral environMent table 2: selected multilateral results International Development Association (IDA) Asian Development Bank (AsDB) Global Fund to fight AIDS, TB and Malaria (GFATM) Global Alliance for Vaccines and Immunisations (GAVI) Private Infrastructure Development Group (PIDG) Global Partnership for Education (GPE) European Commission (EC Budget) ■■ During the fiscal year 2011-12, IDA supported: ■■ the construction or rehabilitation of 57,252 km of roads ■■ social safety net programmes for 114 million people (2010-11) ■■ AsDB assistance in 2012 enabled 2.5 million new households to become connected to a water supply and 174,000 to electricity.
128.6 million people benefited from road projects ■■ 4.2 million people are currently receiving antiretroviral (ARV) therapy to treat HIV, an increase from 1.4 million five years ago; an additional 900,000 people received ARV therapy in 2012 ■■ 9.7 million new TB cases have been detected and treated, an increase from 2.9 million five years ago; 1.1 million TB cases were detected and treated in 2012 ■■ In 2012, it is estimated that GAVI: ■■ supported the immunisation of over 46 million children ■■ contributed to preventing more than 538,000 future deaths ■■ 39 PIDG-supported projects were fully constructed and operational by the end of 2012.
These projects are delivering new or improved infrastructure to 97.6 million people and have attracted over $10.4 billion of private sector investment in infrastructure ■■ In 2011, GPE helped train more than 110,000 teachers, deliver 18 million textbooks, and construct or restore almost 8,000 classrooms ■■ €220 million will be invested in projects that increase access to sustainable and affordable energy services for the poor living in rural and peri-urban areas.
Previous funding of €200 million was invested in 74 projects and is expected to benefit seven million people Global Environment Facility (GEF) ■■ UK funding contributed in 2012 to the following expected results: ■■ Conservation and management of 170 metric hectares of protected areas ■■ 500 million tonnes of CO2 equivalent avoided ■■ Sustainable use and management of biodiversity in 60 metric hectares of production landscapes and/or seascapes UN Women International Committee of the Red Cross (ICRC) ■■ In 2012, 17 countries where UN Women works adopted new or improved legislation on violence against women (in addition to 28 which had already done so), and three additional countries (cumulative total 23) incorporated temporary special measures for women in constitutions, legal frameworks and policies ■■ In 2012, ICRC: ■■ improved water, sanitation and construction work in 53 countries, helping some 22 million people worldwide ■■ supported 2.6 million people to benefit from agricultural infrastructure rehabilitation and construction 20 chapter 3 MAR Update methodology Introduction 1.
The MAR Update is focused on assessing progress, in particular whether reform has happened in the priority areas identified following the MAR.

This chapter sets out how we developed the methodology to make those assessments.

We first give a recap of the methodology used for the MAR, and then explain how we have used that to develop an approach for assessing progress.

We also explain how we have addressed some of the issues raised about the methodology and approach in this Update exercise.

2.
2.

The development of the original MAR was greatly aided by the involvement of two external reviewers, Dr Alison Evans, an independent consultant and formerly Director of the Overseas Development Institute, and Professor Lawrence Haddad, Director of the Institute of Development Studies.

We are therefore very grateful that they also agreed to undertake the role of external reviewer for this MAR Update exercise, providing challenge on the methodology for the Update and on the assessments themselves.
The MAR methodology and findings 3.

The MAR aimed to capture the value for money for UK aid of the whole organisation, and therefore took a broad view of the issue, ranging from control of costs through to the delivery of outcomes.

This follows the DFID model of value for money reproduced in chart 4.

This recognises that although it can be easier to measure some elements of this chain, such as output measures (e.g.
the numbers of health staff trained), what matters ultimately are the outcomes for which we are aiming (e.g.

a healthy population).

chart 4: dfid model of value for money Value for money Economy Efficiency Effectiveness Costs (£) Inputs Outputs Outcomes Qualitative Quantitative 21 Mar update MetHodoloGy 4.

The MAR assessment framework is based on both development and organisational theory.
Development theory says that organisations working in poor countries, trying to do things that matter and following principles such as harmonisation, transparency and accountability to beneficiaries, are more likely to be making a real difference for poor people and communities.

Organisational theory says that organisations that have certain characteristics, such as strong strategic management and a culture of seeking to minimise costs, are more likely to deliver results and offer value for money.
Our assessment criteria drew on both sets of theories.

Table 3 sets out the framework and the way in which we grouped the criteria together into ten components.
table 3: Mar assessment framework component criteria Critical role in meeting development objectives ■■ Important role in delivering key international development goals or humanitarian objectives, with country level evidence of this ■■ Important role in delivering UK development or humanitarian priorities, with country level evidence of this Attention to cross- cutting issues ■■ Performs well in fragile contexts ■■ Promotes gender equality ■■ Ensures its activities are low carbon, climate resilient and environmentally sustainable Focus on poor countries Contribution to results Strategic and performance management ■■ Allocates resources to countries that need it most or prioritises areas of greatest humanitarian need ■■ Allocates resources to countries where it will be best used ■■ Objectives are challenging e.g.
strives to reach the very poorest ■■ Strives for results at country level ■■ Demonstrates delivery against objectives ■■ Contributes to development or humanitarian results ■■ Has a clear mandate, and strategy and implementation plans to deliver it ■■ Governing body is effective at holding management to account ■■ Leadership is effective ■■ Measures results ■■ Has an effective evaluation function ■■ Governing body and management use results and evaluation evidence to improve decision-making ■■ Has good HR policies and practices Financial resource management ■■ Allocates aid transparently ■■ Funding is predictable ■■ Pro-actively manages poorly performing projects and programmes ■■ Ensures financial accountability ■■ Instruments are appropriate 22 table 3: Mar assessment framework Cost and value consciousness ■■ Challenges and supports partners to think about value for money ■■ Rates of return and cost effectiveness issues are important factors in decision-making ■■ Achieves economy in purchase of programme inputs ■■ Controls administrative costs Partnership behaviour ■■ Works effectively in partnership with others ■■ Implements social safeguard policies including incorporating beneficiary Transparency and accountability voice ■■ Has flexibility which enables a country-led approach ■■ Follows Paris/Accra principles in its approach to aid delivery ■■ Provides an effective leadership and co-ordination role in humanitarian settings ■■ Has a comprehensive and open disclosure policy ■■ Promotes transparency and accountability in partners/recipient ■■ Routinely publishes project documentation and project data ■■ Signatory of IATI and shows commitment to implementation ■■ Governing structures include effective partner country representation ■■ Partner country stakeholders have right of redress and complaint Likelihood of positive change ■■ Governing body and management continuously strive for improvement ■■ Evidence of progress against reform objectives in the past ■■ Opportunities to promote reform are anticipated 5.
Nine of these components were grouped into two indices: (i) the contribution to UK development objectives, which primarily assesses what organisations do; and (ii) organisational strengths, which primarily assesses how well they are organised and managed.

The likelihood of positive change component was not included in the composite indices and therefore had no impact on the MAR value for money score.
These composite indices were then used to determine a value for money score for each multilateral: poor value for money, adequate value for money, good value for money, and very good value for money.

Table 4 gives the overall results for the MAR, for those organisations included in the MAR Update.
23 DRIVING REFORM TO ACHIEVE MULTILATERAL EFFECTIVENESS Mar update MetHodoloGy table 4: value for money for uK aid funding of individual multilateral organisations adequate CDB EC Budget IADB OHCHR UNAIDS UNFPA WHO poor CommSec FAO IOM UNESCO UNIFEM very good AsDF ECHO EDF GAVI GFATM ICRC IDA PIDG UNICEF Good AfDF CERF CIFs EBRD GEF GFDRR GPE IFAD IFC IFRC OCHA PBF UNDP UNHCR UNITAID WFP Note: Organisations are listed in alphabetical order.
This table only includes those organisations that DFID continues to fund, and which are included in the MAR Update.

The exception is UNIFEM which is the predecessor organisation for UN Women.

The MAR Update methodology 6.


In developing the methodology for the MAR Update, we were seeking to address three issues – identifying precisely what to assess, considering how best to measure progress rather than an absolute level of performance, and whether we could take this opportunity to amend our approach to address some of the issues raised in commentary on the MAR.

Identifying what to assess – the reform priorities 7.
The findings of the MAR were used to identify specific areas of reform for all the agencies to which DFID continued to provide core aid funding.

The reform priorities of the multilateral organisations are based on the areas where the MAR identified particular weaknesses, but there is not a complete match between low MAR component scores and reform priorities.
This is for two main reasons: where organisations had a lot of challenges, reform priorities were set in the areas which were considered to be most important; and for organisations which were performing strongly, there were very specific areas of weakness to be addressed even if they were starting from a relatively higher level.
So, the MAR assessment formed the basis for setting the reform priorities, but actual reform priorities were set considering where action would have the most impact across the whole organisation, and on other factors, such as the priorities of other governments.

The reform priorities are also specific to the particular weakness identified in the MAR assessment and to the organisation.

For the purpose of analysis, each reform priority is assessed under the component where it is most relevant.
The 24 overall intention is that significantly higher levels of performance are evident by 2015.

8.

These reform priorities form the basis for the MAR Update assessment.

The assessment considers whether progress has been made against the specific reform priority and the MAR component (for example, financial resources management) to which it relates.

Table 5 sets out, for each multilateral organisation, the components which were assessed in the MAR Update12.
table 5: reform priorities by multilateral organisation ● ● ● s t l u s e r o t n o i t u b i r t n o c ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● AfDF AsDF CDB CERF CIFs CommSec ● EBRD EC Budget ECHO EDF FAO GAVI GEF GFATM GFDRR GPE ● ● 12 ‘Likelihood of positive change’ is not included as it was not taken into account in the calculation of the overall value for money score in the MAR.
25 DRIVING REFORM TO ACHIEVE MULTILATERAL EFFECTIVENESS Mar update MetHodoloGy table 5: reform priorities by multilateral organisation IADB ICRC IDA IFAD IFC IFRC IOM OCHA OHCHR PBF PIDG UN Women UNAIDS UNDP UNESCO UNFPA UNHCR UNICEF UNITAID WFP WHO ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● Assessing progress 9.
The second issue was to clarify how to measure progress, ‘the extent to which the UK’s reform priorities have been taken forward since the MAR was carried out’13.

We first assessed progress against each reform priority/component and assigned a progress rating.

These progress ratings were combined to give an overall progress rating for the organisation.
If there was sufficient evidence that an organisation was operating at a higher level than at the time of the original MAR assessment, we awarded an update of the MAR component score.

Finally, a change in a MAR component score could result in a change in the overall value for money score.

13 From the MAR Update terms of reference, 2013 (see annex 1 of this report).

26 10.

Progress was assessed on the basis of the changes that have happened since the MAR and up to the time of the MAR Update assessment.
Given that there was a relatively short time in which to see major change, the assessment was based on whether the changes made met our expectations or not, in terms of what that organisation could be expected to achieve between the MAR and its Update assessment.

We recognise that those organisations assessed in the first tranche (humanitarian organisations) and, to an extent, the second tranche, had less time to take reform forward.
Basing the assessment on our expectation of what could be achieved in the relevant timeframe ensured consistent assessments for the three tranches.

It is also the case that the organisations have continued to take action since the assessments, and that, for some, this has resulted in substantial change from the time of assessment14.

Table 6 sets out the progress ratings with their descriptions.
table 6: component progress ratings component rating description 4 significant progress: Reforms achieved to date surpass expectations by strongly addressing reform priorities significant progress 3 reasonable progress: Reforms achieved to date meet expectations by satisfactorily addressing reform priorities reasonable progress 2 some progress: Reforms achieved to date weakly address reform priorities and do not fully meet expectations some progress 1 little or no progress: Reforms achieved to date unsatisfactorily address reform priorities and any progress is significantly below expectations little or no progress 11.
The overall progress ratings and their descriptions are set out in table 7.

Where an organisation had an equal number of progress ratings (e.g.

two “some” progress ratings, and two “reasonable” progress ratings), the overall progress rating reflects the lower of the component progress ratings.

This is because our aim is to encourage continued reform.
14 For example, CERF has taken action to address the major issue identified under cost and value consciousness, the need to explain the value for money of its management charge.

This significant change is recognised in table 11.


27 DRIVING REFORM TO ACHIEVE MULTILATERAL EFFECTIVENESS Mar update MetHodoloGy table 7: overall progress ratings overall rating Basis for rating 4 3 2 1 significant progress in all components OR significant progress in most components outweighs reasonable, some or no progress in others.
reasonable progress in all components OR reasonable progress in most components outweighs some and no progress in others OR significant progress in some components balances some and no progress in others some progress in all or most components OR reasonable progress in some components balances no progress in others little or no progress in most components not balanced by significant, reasonable or some progress elsewhere 12.
An increase in component score was only awarded where there was sufficient evidence to demonstrate that the organisation is now operating at a higher level than it was in 2011.

The decision about whether the organisation should be awarded a higher MAR component score was based on the score descriptions under each component in the original MAR assessment framework.

Finally, a change in component score could result in a change to the overall MAR assessment of value for money.
Revising the methodology and our approach 13.

As we set out in chapter 1, while most of the commentary on the MAR has been positive, there have been a number of recommendations for improvement.

We have addressed some issues in the MAR Update: revising the assessment framework to meet the needs of multilateral organisations which set standards, engaging more with multilaterals during the process and increasing the quality assurance in the MAR process.

14.
14.

The question of the fit of the assessment framework to the role of agencies which set global policies and standards (often referred to as standard-setting or normative agencies) is perhaps the most significant issue raised with us on the original framework.
These standard-setting agencies told us that their work was not well recognised within the MAR assessment framework because although they supported agencies with direct development roles they did not directly deliver development outcomes themselves.

This meant that they could not provide the kind of evidence that we were seeking.

15.
15.

We therefore worked with these agencies to develop a more detailed MAR assessment framework for standard-setting agencies (see annex 2) which better reflected the types of work they do.
This has the same components, to ensure comparisons across all multilateral organisations can still be made, but includes criteria which relate to the way in which the activities of standard setting agencies are linked to impact in developing countries – that is, their theory of change (this 28 concept is explained in box 1).

We tested this revised framework through visits to India and Uganda.

We aimed to find out whether we could find evidence to confirm the organisations’ proposed theories of change.
These visits did improve our understanding of how the normative agencies’ work has an impact in beneficiary countries.

This gave us confidence that this clarified framework allows those agencies that do not deliver direct development outputs (e.g.

the number of children vaccinated, or numbers of roads built) to still get recognition for their work and the impact it is having on poverty reduction.
Box 1: theory of change At its heart, theory of change involves an exploration of the changes we want to help bring about and how we think the change processes might happen.

It involves the articulation of hypotheses about how change will happen and interrogation of the assumptions underpinning those hypotheses.
Theory of change thus goes beyond the assertion in a results chain that A leads to B, and explores how and why we think A will lead to B – the intermediate steps, the transmission mechanisms, the different possible causal pathways.

Also key is assessment of the evidence for these assumptions and hypotheses.

16.
16.

The second area we considered was the way in which we worked with multilaterals to ensure the assessments were well informed and reached the right conclusions, given feedback that multilateral organisations would have liked to have had more contact with us during the MAR process.

In response, for the MAR Update, we contacted the multilateral organisations in advance to specify the areas which would be assessed, and to circulate the assessment framework.
We also circulated the full assessments in draft, to allow agencies to fact check these and the evidence we were using, and to give them an opportunity to provide further evidence if they wished.

Finally, we shared a copy of the final full and summary assessments in advance of publication, and gave agencies the opportunity to provide a response for publication on the DFID website alongside the MAR Update documents.
We were pleased that multilateral organisations told the recent International Development Committee (IDC) inquiry on the MAR that they had welcomed this increased engagement.

17.

Finally, we improved the way we ensured the consistency and quality of the assessments.

For the MAR, the moderation process involved first considering the assessment for each multilateral organisation against those for similar multilaterals, and then against a wider range of assessments in a three stage process.
For the MAR Update, we put in place a moderation process with the following stages: ■■ peer review – where DFID staff responsible for doing the MAR Update assessments as well as staff with specific technical expertise (for example, on gender) considered the assessments.

The purpose of this moderation was to provide a challenge function during the drafting process.
29 DRIVING REFORM TO ACHIEVE MULTILATERAL EFFECTIVENESS Mar update MetHodoloGy ■■ Heads of departments moderation – where DFID Heads of Departments, largely Senior Civil Servants, considered the overall quality and consistency of the assessments.

They provided specific feedback on the proposed ratings, any proposed score changes, the quality of evidence and the overall strength of the narrative.

The comments from this group were influential in shaping the assessments.
■■ external reviewer and steering group moderation – where DFID directors with responsibility for multilateral engagement, policy and regional programmes, and the Chief Economist provided high level quality assurance.

They focused on the overall consistency of the assessments and the justification for the judgements being made.

The external reviewers ensured there was external challenge on the assessments and the evidence used.

18.
18.

These moderation processes were supported by the DFID MAR Update team, which undertook detailed consistency checks, considering the assessment of similar components against each other, to ensure that similar activities had been undertaken to achieve similar progress ratings.

Evidence Evidence sources 19.

Like the MAR, the Update drew on as many sources of evidence as possible.
The majority of evidence gathered was from the agencies themselves, with the strongest of this evidence being from independent reviews commissioned by the institutions.

Multilateral organisations also provided their annual reports, statements of policy, case studies, evaluations of specific programmes and project completion reports.
In addition to these sources, we gathered valuable evidence from the Multilateral Organisation Performance Assessment Network (MOPAN), assessments by other donors, in particular AusAID’s Australian Multilateral Assessment.

20.

The issue of gathering evidence of performance in developing countries is particularly important.

DFID country offices provided useful evidence and feedback.
We are also grateful for the evidence supplied by non-governmental organisations (NGOs): this is particularly important to us because of the different perspective and expertise that NGOs have (through, for example, working in the field with multilaterals), and as NGOs may be present in countries where DFID has no office.

21.

A number of country visits were undertaken to gather evidence for the MAR in 2011.

These visits did yield some interesting information, but overall their value was limited.
For the MAR Update we undertook exploratory visits to three countries where DFID does not have a country office (Mali, Senegal and Haiti).

This increased our understanding of these contexts.

However, the limitations evident in the MAR visits were particularly true of visits to countries where DFID is not present – it was difficult to talk to the right people, or to ensure that field 30 visits were well targeted to the areas we wanted to explore.
This was also a costly exercise in terms of DFID staff resource.

We took the view that as such visits did not yield the direct evidence that could be taken into account in the MAR Update, and were costly, we should not undertake more.

However, we did make contacts on these visits who later responded to a survey that we sent out, and evidence from this was used in some assessments.

Evidence quality 22.

Each MAR Update assessment includes an assessment of the quality of the evidence for each component.
Assessors rated the quality of evidence through considering the following issues: ■■ whether the evidence was up to date; ■■ whether the information was credible and consistent with other information; ■■ whether the key messages were clearly highlighted; ■■ whether the limitations to evidence were clear and discussed; ■■ whether the producer(s) of the evidence was/were independent; and ■■ whether the evidence is balanced.

23.
23.

The definitions and guidance we used are as follows: (i) High quality evidence should clearly set out what the intention of the paper is and then set about answering any questions identified.

It should normally include: ■■ a summary of any existing literature as well as an idea of where it sits within this literature.
It should also be fully referenced; ■■ a clear methodology should be set out for any research or evaluation, including any questions it is seeking to answer, and should be appropriate to address the central issues.

It should be open and transparent; any data used should be described and ideally widely available; and ■■ being independent and unbiased.

(ii) Medium quality evidence should adhere to the majority of the principles set out for high quality evidence but perhaps depart from them in some areas.
It may not be clear, for example, whether the evidence is independent and unbiased.

The methodology may not be as appropriate as it could be, or rigorous enough to address the central issues.

(iii) Low quality evidence will have major or numerous deficiencies.

The most common form of low quality evidence is anecdotal evidence or evidence that only draws on a handful of people’s perceptions.

It will most likely be inadequately referenced and not adhere to the principles of independence and lacking bias.
31 DRIVING REFORM TO ACHIEVE MULTILATERAL EFFECTIVENESS Mar update MetHodoloGy 24.

Information on the quality and quantity of evidence available was then used to make an assessment of the overall quality of the body of evidence on which we made our assessment.

Categories for the overall assessment of the body of evidence included very strong, strong, medium, limited and no evidence.

25.
25.

On average reviewers found the body of evidence to be of medium quality, with 66% of assessments based on such evidence.

Chart 5 shows that, in general a better body of evidence is correlated to better progress ratings.

No “significant” progress ratings were given where the body of evidence was limited, whilst no “little or no” progress assessments were made when a strong or very strong body of evidence was available.
chart 5: evidence quality by Mar update progress rating Evidence quality by MAR Update progress rating Significant Reasonable Some Little/no g n i t a r s s e r g o r p e t a d p U R A M 0% 100% Percentage of components receiving evidence rating by progress rating 40% 60% 80% 20% Limited Limited/medium Medium Medium/strong Strong Very strong 26.
This analysis also illustrates the importance of improving the evidence base, so that no multilateral organisation receives a lower progress rating due to lack of evidence rather than as a result of strong evidence showing poor performance.
We will aim to do this by working with: ■■ multilateral organisations to improve the evidence they can provide; ■■ the international system to support and develop collaboration on evidence collection, analysis and assessment, such as the Multilateral Organisation Performance Assessment Network (MOPAN); and ■■ other donors so that we can harmonise our approach and ask for similar evidence to be collected, and share our experience and insights.

32 chapter 4 Findings of the MAR Update Introduction 1.
This chapter analyses the findings of the MAR Update, describing the overall pattern of progress and the actions multilateral organisations have taken to address their reform priorities.

This is exciting: for the first time, we are able to consider not only multilateral organisations’ current performance levels but also where they have made good progress on important reform issues and where they have not.

2.
2.

As we have said, the MAR Update is not a full review of all MAR components but an assessment of progress against particular reform priorities agreed with DFID.

These are tailored to each organisation.

While they relate to improvement in a given component area, the actions they require can vary significantly, depending on the organisation’s starting point, and their role, mandate and business model.
Reform priorities were set primarily in areas of challenge and weakness, but again this judgement was institution-specific.

High performing organisations were sometimes asked to strengthen performance in areas that were already satisfactory or strong overall.

So while all organisations were asked to make progress on a challenging set of reforms, the nature of the challenge varied.

The institution-specific nature of the reform asks makes it more difficult to draw out common themes from the data.

3.
3.

The variation in the reform priorities also poses other challenges for the analysis.

First, the number of reform areas varies for each multilateral.

This means that the overall assessment of reform progress for each multilateral is based on a wider set of reform in some cases than in others.

This is further complicated by the timing of the assessments, with some organisations being assessed over a longer period of time than others.
Second, the number of multilaterals assessed under each component varies.

Where only a few organisations are assessed under a particular component, it is more difficult to draw firm conclusions about progress and the component as a whole.

4.

The analysis takes these challenges and caveats into account, using a number of different approaches to gain the best understanding we can from the information.
We look first at the overall progress across all multilateral organisations assessed under the MAR Update, then across all components.

There has been particular interest in the performance of those agencies in “special measures” and we include more detailed information about the progress they have made.

We then analyse progress under each component in more depth, considering the type of organisations and their starting position as well as their progress.
The analysis by type of organisation uses the groupings set out in 33 findinGs of tHe Mar update table 1, chapter 215.

Finally, short summaries of progress for each of the multilateral organisations assessed in the MAR Update are provided in table 11 at the end of this chapter.

Overall progress across the multilateral organisations 5.

Overall, there has been progress against the reform priorities set, although in some areas progress is less than expected.
Chart 6 ranks the multilateral organisations by progress made.

Amber shows that on average the organisation achieved “some” progress across its reform priorities and green shows “reasonable” progress on average.

No organisation was assessed as having made either “little or no” progress on average across its reform priorities, or “significant” progress on average – although these progress ratings were used for some individual components16.
chart 6: Multilateral organisations ranked by average Mar update progress rating16 3.5 3.0 2.5 2.0 1.5 1.0 0. 0.0 C D B m Sec U N D P U N FP A E C B u d g et C o m E D F G F D R R IA D B IFR C O H C H R U NIT AID C E RF E C H O U N-O C H A AfD F E B R D IFC U N H C R AsD F G A VI U NIC EF G EF IO M P BF PID G CIFs G F A T M ID A IF A D U N W o m e n U N ESC O H O F A O G PE W FP U N AID S IC R C 6.
Chart 7 shows the MAR Update average progress rating for each multilateral organisation, against their overall MAR value for money score.

This shows that organisations with a MAR overall value for money score of very good and those which had the lowest value for money score of poor were more likely to have 15 Throughout this report, the term “UN organisation” refers to those UN organisations focused on development.
16 UN Organisations more focused on humanitarian action are considered under “humanitarian organisations”.

This follows the terminology and groupings used in the MAR.

In charts 6 and 7, the overall average MAR Update progress ratings have been derived from the progress ratings for each component.

A score was allocated to each progress rating (from 1 for “little or no” progress to 4 for “significant” progress).

These were then averaged for each organisation to get an organisation average progress rating.
34 W made stronger progress.

There is a range of progress for organisations with a good MAR score.

Organisations classed as adequate in the MAR tend to have made slower progress.

There are, not surprisingly, exceptions in each case – in particular UNAIDS which made such good progress that an overall MAR value for money score change from adequate to good has been achieved.
One interpretation of this is that the best progress was achieved by those organisations which had the most capacity to change (those which were in any case strong performers), or had the most motivation (those which were in “special measures”).

However, this is speculative; we will be seeking to understand these results better over the next few months.
chart 7: Mar value for money and Mar update progress ratings across the multilaterals GAVI PIDG ICRC ECHO AsDF EDF UNHCR CERF OCHA IFRC UNICEF IDA GFATM AfDF EBRD IFC PBF CIFs IFAD GEF WFP GPE GFDRR UNDP UNTAID UNFPA IADB EC budget OHCHR CDB UNAIDS UNIFEM/ UNWomen IOM WHO FAO UNESCO CommSec “Some” progress in most areas “Reasonable” progress in most areas 2.00 2.50 Average MAR Update progess rating 3.00 3.50 3.50 3.00 2.50 2.00 e r o c s R A M e g a r e v A 1.50 1.50 Key Very good value for money Adequate value for money Good value for money Poor value for money Progress in the MAR Update led to a score change in one or more components 7.
Chart 7 also shows which organisations demonstrated a higher level of performance in a particular MAR component and which therefore received a score change on that component.

It is a considerable achievement to have 35 DRIVING REFORM TO ACHIEVE MULTILATERAL EFFECTIVENESS findinGs of tHe Mar update improved performance to such an extent in the two years since the MAR was published.

Table 8 highlights the score changes by organisation and by component.
table 8: score changes in the Mar update organisation score change CommSec 1 to 2 on financial resources management FAO GEF 1 to 2 on transparency and accountability 2 to 3 on partnership behaviour UN Women 1 to 2 on contribution to results UNAIDS UNESCO 2 to 3 on cost and value consciousness 1 to 2 on contribution to results and strategic and performance management Progress made by “special measures” organisations 8.
Following the MAR, four organisations were placed in “special measures”– meaning that they had to make urgent improvements for DFID to continue their funding.

These organisations all received a poor value for money for UK aid score in the MAR, but the UK coalition government continued to fund them due to their potential to make a significant development contribution.
Each of these agencies has made progress against reform priorities: three of the four made reasonable progress on average and three of the four achieved a score change against at least one component.

Table 9 shows their results in more detail, and box 2 provides an illustration of what this progress looks like in practice.
36 table 9: detailed results for “special measures” agencies* Multilateral priority reform components Mar 2011 score Mar update score Mar update progress rating overall rating commsec Critical role in meeting development (specifically international aid) objectives Contribution to results Strategic and performance management Financial resources management Cost and value consciousness fao Contribution to results Strategic and performance management Financial resources management Cost and value consciousness Transparency and accountability 2 3 2 2 2 1 1 2 2 2 2 1 2 2 2 2 1 2 2 2 2 2 little or no progress some progress some progress reasonable progress some progress significant progress reasonable progress some progress some progress reasonable progress * shaded rows indicate a change to the Mar score for that component.
37 DRIVING REFORM TO ACHIEVE MULTILATERAL EFFECTIVENESS findinGs of tHe Mar update table 9: detailed results for “special measures” agencies* Multilateral priority reform components Mar 2011 score Mar update score Mar update progress rating overall rating ioM Strategic and performance management Financial resources management Partnership behaviour unesco Contribution to results Strategic and performance management Cost and value consciousness Transparency and accountability 3 3 2 2 2 1 1 2 2 2 2 2 2 2 2 2 some progress reasonable progress reasonable progress reasonable progress reasonable progress reasonable progress some progress * shaded rows indicate a change to the Mar score for that component.
38 Box 2: food and agriculture organization (fao) – making progress from “special measures” FAO focuses on eradicating extreme poverty and hunger, particularly in rural areas of developing countries, where most of the world’s poorest people live.

The MAR found that FAO’s performance in developing countries was inconsistent, meaning that it was not always reaching the poor in the most effective way or having sufficient impact.

Yet change can happen.


New leadership has introduced greater prioritisation and a stronger focus on results.

FAO now works more strategically in helping countries to tackle hunger and food insecurity.

Performance management systems have been introduced for all staff and better recruitment is helping to ensure that the right people are in the right place, improving coordination and delivery of results, including in emergency situations.
The impact of these changes and continued attention to reform will provide FAO with a clearer focus on achieving results, and make it better able to help eliminate hunger and reduce rural poverty.

DFID/Robert Stansfield (Nepal, 2013).

Overall progress by reform component 9.

We are interested in the progress made against each reform component.

Chart 8 shows the MAR scores for the components and organisations assessed in the MAR Update and the number of organisations assessed under each component.
This shows that, predominantly, reform priorities have been set in areas of relative weakness (lower average MAR scores) although there are some set for organisations which already demonstrate ‘satisfactory’ or ‘strong’ performance in that area.
Chart 8 also shows that most organisations had reform priorities under 39 DRIVING REFORM TO ACHIEVE MULTILATERAL EFFECTIVENESS findinGs of tHe Mar update strategic and performance management (28), cost and value consciousness (23) and contribution to results (21), showing that these areas are considered to be both important and challenging.

On average, each organisation was assessed against four components.
We note that UN organisations feature in almost all the components – they are also the largest group of organisations included in the MAR Update (see table 1 in chapter 2).
chart 8: Mar scores for components assessed in the Mar update Decreasing average MAR score – relatively weaker components Critical role in meeting international (specifically international aid) objectives Climate change and environmental sustainability Partnership behaviour Cost and value consciousness Fragile contexts Contribution to results Strategic and performance management Financial resources management Transparency and accountability Focus on poor countries Gender 25 20 15 10 5 0 5 10 15 Strong Satisfactory Weak Unsatisfactory Number of organisations 10.
Chart 9 shows progress across components by the original MAR score.

Not surprisingly, it reflects the pattern shown in chart 7, in that there was most progress from the strongest, and weakest, starting positions, and more mixed progress in the middle.

Challenges clearly remain.

40 chart 9: summary of progress by original Mar score Strong Satisfactory Weak Unsatisfactory 60 40 20 0 20 40 60 Number of organisations 11.
Chart 10 shows progress by reform component: there has been progress across all reform components.

Better progress has been made on strategic and performance management, contribution to results, partnership behaviour and financial resources management, and slower progress has been made on cost and value consciousness, transparency and accountability and gender.

These are important areas and this is a matter for concern.
These results also need to be considered in the light of chart 9, which shows the original MAR scores.

Of the components which have made the strongest progress, strategic and performance management included organisations which had low MAR scores for this component, suggesting a step-change in performance in this component.
Of the components where progress has been slower, cost and value consciousness included organisations which had relatively higher MAR scores than is the case for gender and transparency and accountability.

This suggests that actual performance levels are lowest in these latter two, as they were starting from a low base and have made slow progress, whereas cost and value consciousness has made slow progress but original performance was relatively better.
We are disappointed that more progress has not been made in these areas, given their importance and priority both to the UK coalition government and to the achievement of development objectives.
41 DRIVING REFORM TO ACHIEVE MULTILATERAL EFFECTIVENESS Most to least reform findinGs of tHe Mar update chart 10: progress by reform component Climate change and environmental sustainability Financial resources management Partnership behaviour Contribution to results Strategic and performance management Fragile contexts Gender Transparency and accountability Cost and value consciousness Focus on poor countries Critical role in meeting development (specifically international aid) objectives 20 15 10 5 0 5 10 15 20 Number of organisations Significant Reasonable Some Little or no Progress by component 12.
The next section explores progress by component in more depth.

We look at these in order, from those components where there has been most reform to those where there has been the least.

To gain a better understanding of the progress made, we have analysed these in a number of ways, considering the main actions taken, the progress from the original MAR score, and progress by type of multilateral.
There were too few organisations under focus on poor countries and critical role in meeting development (specifically international aid) objectives to allow similar analysis to be undertaken.

Financial resources management 13.

Financial resources management looks at how multilateral organisations allocate, disburse and account for their resources.
We were looking for evidence that resources were allocated on a clear and transparent basis, predictable and long- term funding commitments, release of aid on schedule, flexibility to use a range of different aid instruments according to need, strong policies and processes for financial accountability and oversight, and a pro-active approach to managing poorly performing projects, curtailing them where necessary and redeploying the funding elsewhere.

42 14.
42 14.

There was a good level of performance on this component in the MAR in 2011, with 23 of the 43 multilateral organisations being rated as ‘satisfactory’ or ‘strong’.

This pattern has continued, with strong progress being made under this component in the MAR Update.

17 organisations were assessed under this component: of these 13 had a MAR score of ‘weak’ or ‘unsatisfactory’, but almost two-thirds of organisations assessed made “significant” or “reasonable” progress (chart 11).
chart 11: Mar scores and Mar update progress ratings for financial resources management17 CommSec GPE EC Budget IFAD IOM UNAIDS UNITAID WHO FAO IFRC OCHA OHCHR UNFPA EDF GAVI GFATM GEF Unsatisfactory Weak Satisfactory Strong MAR score Indicates a score change Significant Reasonable Some Little or no 15.

The reform priorities set for different organisations were very similar regardless of original MAR rating.
This is because MAR scores were related to how many of the component’s requirements the organisation fulfilled.

Reform priorities were set for organisations to address gaps – and the same issue might have been an important gap for an organisation with a ‘weak’ score as for one with a ‘satisfactory’ score.

16.
16.

Every organisation assessed against this component had reform priorities asking for improvements on at least one of the following: financial management and control, allocation models, anti-fraud and anti-corruption systems, disbursement speed, risk management or management of poorly performing programmes.
Accordingly, the main actions taken were in these areas, with organisations working to put in place better targeted and more transparent funding allocation models, improved management of poorly performing projects and improved financial management and control.
Differences in MAR Update progress ratings 17 The acronyms for organisation type used throughout in relevant graphs are as follows: UN – UN organisations (excluding humanitarian) Other – The Commonwealth Secretariat and the Inter-American Development Bank EC – European Commission (excluding humanitarian) MDB – Multilateral development banks with a focus on concessional funds GF – global funds for health, education and climate change H – humanitarian organisations (including UN humanitarian organisations) PSD – development finance institutions and funds supporting private sector development 43 DRIVING REFORM TO ACHIEVE MULTILATERAL EFFECTIVENESS findinGs of tHe Mar update arose from how far reforms had been implemented or the number of reform elements organisations had addressed.
However, weaknesses persist for a number of institutions particularly on continued slow disbursement speeds and issues with transparency in key decision making processes, for example funding allocation decisions.
chart 12: progress ratings for financial resources management by organisation type s n o i t a s i n a g r o f o r e b m u N 4 3 2 1 0 1 2 3 GPE GAVI GFATM UNITAID GEF IOM IFRC OCHA EC Budget EDF CommSec IFAD UNAIDS WHO FAO OHCHR UNFPA PSD H GF MDB EC Other UN Organisation type Significant Reasonable Some Little or no 17.

Chart 12 shows the progress rating by organisational type; the UN organisations and global funds form the largest groups under this component.
The global funds made good progress, with better performance being characterised by work to improve their aid allocation models.

Box 3 gives more detail on the GPE’s actions in this area.

18.

A number of agencies also increased the speed of their disbursements.

Less progress was made by UN organisations, three of which made “reasonable” progress, whilst the remaining three made “some” progress.

A common area of reform was the strengthening of financial management and administrative systems.
UN organisations have now reduced the number of issues raised by audits and acted to reduce the number of unaddressed issues from previous audits.

They have improved the content of their financial statements and put in place better financial controls.

19.

The EC has made “reasonable” progress on managing the EC budget instruments and the EDF.

For example, EC aid will in future be better targeted on the poor and fragile states and a number of wealthier countries will be “graduated” from bilateral aid.
For the EDF, steps are being taken to improve forecasting of commitments and disbursements.

The humanitarian organisations have made 44 progress against very different reform priorities, including putting in place new aid allocation frameworks, improved fraud and corruption policies and reforms to strengthen budget processes.

Progress on reducing disbursement times is constrained for certain organisations that are dependent on implementing partners to get funding to beneficiaries.
Box 3: the Global partnership for education (Gpe) – increasing transparency, predictability and flexibility in the allocation of its funding to developing countries A strong financial resources management system is important to ensure that financial resource allocation decisions are transparent, long-term and predictable, and can enable poorly performing projects to be identified and better managed.

Since 2011, GPE has increased the transparency and predictability of its allocations to countries.
Grants are now set indicatively in advance and published on their website, based on a formula which balances need for funding against capacity to deliver, and weighted to support key GPE priorities – including fragile states, girls’ education and domestic financing for education.

GPE has also expanded the range of agencies it works with to oversee funding in developing countries, so that recipient governments have more choice of partners to work with.
The aim of these reforms is to increase access to GPE funding, especially for fragile countries who may previously have found it difficult to access support.

The effect of GPE’s reforms has been to significantly increase country demand for grants and it has expanded the numbers of fragile states it supports by 50% over the last two years.

In 2013 it used a new accelerated funding mechanism for the first time to reach schools in Somalia for the start of the school year.
WaterAid/Suzanne Porter (Nigeria, 2006).

45 DRIVING REFORM TO ACHIEVE MULTILATERAL EFFECTIVENESS findinGs of tHe Mar update Partnership behaviour 20.

For partnership behaviour, we considered that this would be shown by organisations working effectively with a wide range of others (including national governments, civil society, and other multilateral organisations), and in a way that supports countries in leading their own development.
Organisations performing well would be taking a leadership role on the aid effectiveness agenda (set out most recently in Busan) and promoting the participation of women and marginalised groups.

For humanitarian agencies, we looked for effective leadership and good co-ordination in humanitarian settings and emergencies.

21.
21.

The MAR generally showed that this is an area of strength for multilaterals, with around three-quarters of the agencies assessed rated as ‘strong’ and ‘satisfactory’, and none as ‘unsatisfactory’.

In the MAR Update, partnership behaviour was a reform priority for 14 organisations, seven of which were rated as ‘satisfactory’ in the MAR and seven as ‘weak’.

Progress has been good, with nine “reasonable” and five “some” progress ratings.
Those organisations rated as ‘satisfactory’ have done marginally better, leaving some cause for concern about those organisations with a ‘weak’ MAR score and demonstrating only “some” progress.
chart 13: Mar scores and Mar update progress ratings for partnership behaviour CIFs GEF IFC IOM GFATM IDA UNHCR AfDF GAVI GFDRR GPE IFRC PBF UNICEF Unsatisfactory Weak Satisfactory Strong Indicates a score change MAR score Significant Reasonable Some Little or no 22.
Reform priorities focused on two main areas: the need for greater country ownership and use of country systems in project and programme design, and improved engagement with partners such as civil society, the private sector, other multilaterals and donors including DFID.

Organisations with a ‘satisfactory’ MAR score and a “reasonable” progress rating were more transparent about what partners could expect them to deliver.
More generally, organisations showing “reasonable” progress promoted both country ownership, with more alignment and use of country systems, and improved the way they worked with others.

Organisations awarded “some” progress tended to do one or the other.

For these 46 organisations, more needs to be done to improve the way they work with partners in developing countries.
chart 14: progress ratings for partnership behaviour by organisation type s n o i t a s i n a g r o f o r e b m u N 4 3 2 1 0 1 2 CIFs GAVI GEF GPE GFDRR IFRC IOM IFC AfDF UNHCR GFATM IDA PBF UNICEF PSD H GF MDB EC Other UN Organisation type Significant Reasonable Some Little or no 23.
Chart 14 shows that most types of multilateral organisations are represented, with global funds in particular having been set this as a reform priority reflecting the fact that their business model depends on working through others.

The climate funds have made “reasonable” progress by improving the way they work with civil society and the private sector, and by improving country ownership through greater involvement of country partners in programme design.
The health funds have increased capacity to handle country specific issues both within country and at headquarters.

However, it is too early to see the impact of some of these changes.

Humanitarian organisations have increased their capacity in humanitarian settings and are working more closely with other multilaterals, civil society and the private sector.

Box 4 shows how a partnership between the private sector and a humanitarian organisation is working.

24.
24.

Both of the multilateral development banks assessed have improved their use of country systems in project and programme design – one of the main weaknesses identified in the MAR.
47 DRIVING REFORM TO ACHIEVE MULTILATERAL EFFECTIVENESS findinGs of tHe Mar update Box 4: international federation of red cross and red crescent societies (ifrc) – working effectively with the private sector to mitigate risks in flood prone communities A multilateral organisation’s ability to work closely with others is crucial to successful intervention and many multilaterals are now seeking to work more closely with new partners in the private sector.
The IFRC has been working with Zurich Insurance Group and its Z Zurich Foundation since 2008; this has recently expanded significantly through the signing of a five-year memorandum with a particular aim of building community flood resilience.

This partnership combines the Z Zurich Foundation’s funds (up to 21 million CHF), Zurich’s risk, financial and project management expertise (an average of 500 working days each year) and the Red Cross Red Crescent experience of working with communities.
Both parties will provide dedicated resources, from regional and local Zurich and IFRC offices, to support Red Cross Societies in selected countries to implement long term disaster risk reduction programmes in flood prone communities.

Zurich and the IFRC aim to build their knowledge about floods and test the feasibility of implementing enhanced risk assessment and impact measurement approaches.

They will also support local capacity building and regional advocacy initiatives.
IFRC and Zurich also aim for this work to develop Red Cross Red Crescent's ability to deliver at scale, improve quality and accountability, and provide evidence of impact; and to increase Zurich employee satisfaction.

Contribution to results 25.

Under the heading of contribution to results, we were interested in whether multilateral organisations were setting and meeting challenging objectives and IFRC/Alana Torralba (Philippines, 2013).
48 making a consistently positive contribution to development or humanitarian results.

Higher performance would be indicated by the organisation, for example, having ambitious targets, or demonstrating innovation, or commitment to reaching the poorest.

We were also interested in where organisations could demonstrate they were having an impact in developing countries, and evidence of management resolving problems.

26.
26.

Chart 15 shows that contribution to results was a reform component for 21 organisations; and that most of the organisations for which it was a reform priority had MAR scores of ‘unsatisfactory’ or ‘weak’.
There has been good progress overall on this component, with both of the organisations rated as ‘unsatisfactory’ in the original MAR having achieved “reasonable” progress (enough in both cases to justify a score change to ‘weak’); and just over half of the organisations rated as ‘weak’ and ‘satisfactory’ have achieved “reasonable” progress.
chart 15: Mar scores and Mar update progress ratings for contribution to results UNESCO UN Women FAO AfDF GPE OHCHR PBF UNAIDS WHO CDB CommSec OCHA UNDP UNFPA IDA IFAD UNICEF WFP AsDF EBRD GFDDR Unsatisfactory Weak Satisfactory Strong Indicates a score change MAR score Significant Reasonable Some Little or no 27.
Many of the reform priorities set for these organisations were similar, and concentrated on the development, or better use, of results frameworks.

So, the main reforms undertaken focused on this: for example the revision of their results frameworks so that they included a comprehensive picture of the organisation’s work, enabling more detailed reporting of results.
A number of organisations are now also reporting results on a more regular basis and some organisations have improved systems for monitoring and reporting results in developing countries.

Organisations rated as having made “some” progress tended to have systems in place for reporting results in developing countries, but less evidence of whether and how these systems were working.
Organisations achieving a “reasonable” progress rating tended to have more evidence that their systems were producing reports on results in developing countries, giving them the ability to monitor and act on this information.

49 DRIVING REFORM TO ACHIEVE MULTILATERAL EFFECTIVENESS findinGs of tHe Mar update 28.

Chart 16 shows the progress ratings by organisation type.
UN organisations performed relatively poorly in the MAR on this component, specifically around results reporting and a lack of focus on their comparative advantage.

In the MAR Update they have made good progress.

There is evidence that they are prioritising and streamlining their strategic objectives to focus more on what they are best at.

This has then led to improved results frameworks (as noted above), both at headquarters and in country offices.
However, the organisations need to do more on using this improved data to drive improvements in performance and portfolio management.

There is also evidence that UN organisations improved their human resource capacity in developing countries.

29.

The UN organisations also include the four standard-setting agencies18.
As discussed in chapter 3, we took action to ensure that the MAR assessment framework explicitly recognised the work they do on setting standards and global policies (see also annex 2 which gives the clarified framework).

As a result of this, they have used theory of change principles to better articulate how their work improves the delivery of development and humanitarian results in developing countries.
These agencies all showed good progress: one achieved a score change from ‘unsatisfactory’ to ‘weak’ and another was rated as having achieved “significant” progress.

chart 16: progress ratings for contribution to results by organisation type FAO IFAD OHCHR PBF UNAIDS UNESCO UNICEF UNWomen WHO WFP GPE EBRD OCHA GFDRR AfDf IDA AsDf CDB CommSec UNDP UNFPA PSD H GF MDB EC Other UN Organisation type Significant Reasonable Some Little or no 30.
The MAR noted that a few of the multilateral development banks assessed on this component did not have a good management focus on results, some did not have enough of a focus on delivering poverty reduction, and many found it difficult to 18 FAO, WHO, UNESCO and OHCHR 50 demonstrate their contribution to outcomes.

The MAR Update has shown there has been a marked improvement (as illustrated in box 5) on AfDB’s work in this area.
The multilateral development banks have revised their results frameworks, for example, to include climate change, gender, budget management as well as focusing on measures to promote more inclusive growth.

However, there is more to do on ensuring that focusing on delivering results is fully embedded in all the multilateral development banks.
The humanitarian organisations already had good levels of performance on this component and the reform priorities assessed here were very specific to particular issues in the organisations.

They have however taken steps to improve their emergency response capacity.
Box 5: african development Bank (afdB) – tracking its performance to further improve its contribution to africa’s development It is important that multilaterals have systems in place to monitor results so that both the organisation and its donors can assess the impact they are having.

AfDB recently introduced development effectiveness reviews – a suite of publications that track its performance in contributing to Africa’s development.
The reviews track performance at the corporate level and AfDB’s performance in areas of special importance such as fragile states, regional integration and governance, and identify what has worked well and where AfDB can do better.

These reviews have been underpinned by strengthened systems for tracking results throughout the project lifecycle.
From 2014, AfDB’s management will have real-time information on the results for all on-going operations, broken down by sector, region and country – providing information to support the further improvement of AfDB’s performance.

Special attention has also been given to decentralisation and demonstrating results in developing countries.

The Bank is delegating more authority, building country capacity, and ensuring that results are included in its country strategies.
These measures enable AfDB to monitor its performance see whether they are meeting targets and use the information to inform future programmes to increase their effectiveness.

DFID/Shareefa Choudhury.

51 DRIVING REFORM TO ACHIEVE MULTILATERAL EFFECTIVENESS findinGs of tHe Mar update Strategic and performance management 31.

The ability of the multilateral organisations to achieve development or humanitarian objectives is often tied to the quality of their strategic and performance management systems.
The MAR looked for organisations with: a clear mandate and strategy; effective leadership and governance structures; a results culture and results based management systems; and an evaluation culture, holding independent evaluations and acting on the recommendations.

They should also manage their people well, with high quality human resource management systems, transparent and merit-based recruitment and promotion, and effective performance management systems.

32.
32.

The MAR identified that this was a generally challenging area for multilateral organisations.

Only 14 of the 43 multilateral organisations assessed were considered to have ‘satisfactory’ or ‘strong’ strategic and performance management systems.

It is not surprising therefore this component has the most organisations (28) assessed for the MAR Update.

Chart 17 shows that progress was mixed, with just over half of these (16) making “reasonable” progress and just under half (12) making “some” progress.
chart 17: Mar scores and Mar progress ratings for strategic and performance management s n o i t a s i n a g r o f o r e b m u N 15 10 5 0 5 10 15 UNESCO CERF FAO IADB IFRC UNAIDS UNITAID UN Women OCHA WFP ECHO CommSec EC Budget EDF GFDRR GPE IOM OHCHR UNDP UNFPA UNICEF WHO CIFs GEF GFATM IFAD GAVI AsDF Unsatisfactory Weak Satisfactory Strong Indicates a score change MAR score Significant Reasonable Some Little or no 33.
Reform priorities under strategic and performance management were very diverse and very specific for each organisation, given the wide range of issues this 52 component covers.

The main reforms undertaken are therefore varied reflecting this.

Organisations have increased their strategic focus by increasingly prioritising their activities and some have decentralised decision-making.

They have undertaken HR reform, and improved monitoring and evaluation through establishing independent evaluation units.
Many also improved their results frameworks and their ability to manage through these.

34.

Progress ratings were awarded based on the breadth of reforms undertaken against allocated priorities, the extent of progress on these reforms and the impact these reforms have had to date.
A “reasonable” progress rating reflects that organisations made good progress on reforms across all areas highlighted as organisational weaknesses, and “some” progress reflects either that reforms have yet to have real impact or that they have taken place in fewer areas.
chart 18: progress ratings for strategic and performance management by organisation type CERF ECHO IFRC OCHA WFP CIFs GEF GFATM UNITAID AsDF GFDRR IOM GAVI GPE EC Budget EDF IADB CommSec FAO IFAD UNAIDS UNESCO UN Women OHCHR UNDP UNFPA UNICEF WHO PSD H GF MDB EC Other UN Organisation type Significant Reasonable Some Little or no 35.

Chart 18 shows that this component was important across most organisational types.
The greatest concerns were raised around results based management in the EC and UN organisations and the human resource systems of both UN and humanitarian organisations.

36.

Issues highlighted by the MAR differed greatly between various types of organisation.

10 UN organisations were set strategic and performance management as a reform priority: half made “reasonable” progress and half made “some” progress.
As discussed in contribution to results, one important reform has been to improve results frameworks and to use the information from these to improve management decisions.

53 DRIVING REFORM TO ACHIEVE MULTILATERAL EFFECTIVENESS findinGs of tHe Mar update 37.

Good progress was made across humanitarian organisations, with important improvements made to strategies which now better define the aims and objectives of organisations and aid the targeting and planning of actions.
There has also been some strengthening of their evaluation functions.

Similarly, the global funds made good progress, in particular through increasing the use of results frameworks and better using these for management decisions.

Box 6 illustrates the comprehensive approach taken by GFATM.

Global funds have also made improvements to programme management, with reforms ranging from simplifying grant application forms to improving the availability of key project data.
The EC made some progress on improving their strategic and performance management.

Reforms undertaken for these two instruments were similar, including HR reforms and a results framework which has been delayed but is due to be implemented soon.
Box 6: the Global fund to fight aids, tB and Malaria (GfatM) – strategically redirecting resources to areas where they will have the greatest impact The quality of an organisation’s strategic and performance management systems underpin its ability to contribute to achieving development or humanitarian objectives.

In late 2011, the Global Fund to fight AIDS, Tuberculosis and Malaria’s Board agreed a new strategy for the period 2012-16.
The secretariat identified 20 “high-impact” countries: (i) that have 70% of the worldwide burden of HIV/AIDS, tuberculosis and malaria, and 54% of the fund grants; and (ii) where investments can produce the greatest results despite significant risks associated with being fragile, geographically challenging, and incredibly poor and with huge disease burdens.

Priority was given to recruiting staff for these country teams, and new departments were created to oversee funding in these countries.
The fund is also undertaking programme evaluations in all these countries to ensure that this activity and investment is producing the desired results.

12 evaluations have been completed and have already informed grant reviews or new funding applications.

A new organisational structure has also been put in place to support this work.

This has resulted in 75% of the secretariat now being dedicated to the core business of grant management, up from less than 50% previously.
There has been an 11% increase in the number of fund portfolio managers, a 60% increase in programme officers and a doubling in the number of procurement and supply management (PSM) specialists at the same time as overall headcount and operating expenses have been reduced significantly.

DFID/Thomas Omondi (Malaria Prevention, Kenya, 2007).

54 Transparency and accountability 38.


Under transparency and accountability we assess whether organisations make comprehensive information about their policies and projects readily available to outsiders; and whether they are accountable to those they work with, including donors, developing country governments, civil society, and direct beneficiaries.

39.

The MAR showed that performance was relatively weak, with just over half of the organisations having a score of ‘weak’ or ‘unsatisfactory’.
15 organisations, had transparency and accountability as a reform priority.

Chart 19 shows that, overall, progress on this component was also relatively slow – with just under half of the organisations making “some” or “little or no” progress.
chart 19: Mar scores and Mar update progress ratings for transparency and accountability FAO CERF ICRC UNHCR UNICEF UN Women WFP CDB PIDG UNESCO UNFPA IFRC UNITAID CIFs IADB Unsatisfactory Weak Satisfactory Strong MAR score Indicates a score change Significant Reasonable Some Little or no 40.
The main reform priorities set for the organisations were focused on improving transparency, specifically putting in place disclosure policies and making project and programme documentation and data more readily available.

Reforms on accountability focused on increasing organisations accountability to beneficiaries, donors and other stakeholders.

41.
41.

In response, many organisations have improved the information they publish on their websites, more organisations have signed up to IATI and a number have endorsed disclosure policies.

Organisations which were rated as making “reasonable” progress made more comprehensive reforms, going beyond other organisations in the information they published, for example publishing historic information on essential supply prices, and/or accountability, such as reforming regional governance bodies.
The outstanding weaknesses under transparency and accountability were that organisations needed to go further with their reforms, publishing more information online and making a presumption of disclosure.

55 DRIVING REFORM TO ACHIEVE MULTILATERAL EFFECTIVENESS findinGs of tHe Mar update Organisations making less progress on accountability needed to formalise changes, ensuring that necessary systems are in place and adhered to.

Overall, progress was mixed and less than we would have liked.
chart 20: progress ratings for transparency and accountability by organisation type s n o i t a s i n a g r o f o r e b m u N 4 3 2 1 0 1 2 CERF ICRC UNHCR WFP CIFs PIDG IFRC UNITAID CDB IADB FAO UNICEF UN Women UNESCO UNFPA PSD H GF MDB EC Other UN Organisation type Significant Reasonable Some Little or no 42.

Chart 20 shows progress by organisation type.

The MAR found that UN organisations were weak on transparency whilst the humanitarian organisations were weak on accountability.
Some reform has happened but less than we had hoped for.

The five UN organisations assessed under this component made important reforms to their transparency arrangements, increasing their engagement with IATI, with a number signing up to the standard and others publishing under it for the first time.

They also published better and more easily accessible data on their own websites.
Humanitarian organisations made key reforms to their accountability structures, in particular strengthening the role of advisory groups and revising the selection criteria for their membership.

More can be done on accountability to beneficiaries and other stakeholders including NGOs.

Both of the global funds assessed needed reforms to improve transparency.

Whilst one organisation made reasonable progress, signing up to IATI and re-designing their website, the other was not able to address concerns.
Cost and value consciousness 43.

This component looks at the drivers of spending by multilateral organisations, assessing whether or not concern for costs and value are important motivations for decision-makers.

The MAR sought evidence of organisations striving for economy in purchasing decisions and seeking to reduce administrative costs.
We wanted to see management and accountability systems with a strong focus 56 on achieving value for money in programme spend, including challenging and supporting partners to take a hard look at value for money in their policy and programme choices.

44.

Cost and value consciousness was a reform component for 23 organisations – showing its importance to DFID and the organisations involved in the MAR.
As discussed above, these organisations are on average starting from a higher position than the other components, with 10 having a ‘satisfactory’ or ‘strong’ MAR score.

However progress has been slow: only seven organisations achieved at least “reasonable” progress – although there is one “significant” rating, with a score change.
chart 21: Mar scores and Mar progress ratings for cost and value consciousness CommSec UNAIDS PBF UNESCO WHO FAO IFAD OHCHR UNDP UNFPA UN Women CDB CERF GAVI EBRD GFATM UNICEF AfDF AsDF EC Budget EDF IADB UNITAID Unsatisfactory Weak Satisfactory Strong Indicates a score change MAR score Significant Reasonable Some Little or no 45.
Reform priorities were mainly focused on the control of costs, particularly administration costs; improving efficiency; improving procurement practices; and addressing value for money issues amongst partners.

In line with these priorities, across the organisations reforms have focused on keeping administration costs to a minimum, both as absolute amounts and by reducing the proportion of administration to programme costs.
There has also been some focus on control of input costs through, for example, freezes on salary and recruitment or bigger strategic moves to reduce on-going costs by moving operations to lower cost locations.

Actions have also been taken to improve systems to enable a better understanding of costs, and to improve procurement processes.
Amongst the organisations achieving only “some” or “little or no” progress challenges remain in improving efficiency; and embedding value for money practices within their organisations and amongst partners.

46.

The organisations which achieved at least “reasonable” progress have gone beyond these reforms with evidence of them using cost information in decision making; and actively challenging and supporting partners on value for money and using this as a basis for future partner selection.
In a few cases, organisations 57 DRIVING REFORM TO ACHIEVE MULTILATERAL EFFECTIVENESS findinGs of tHe Mar update have taken steps to be more transparent about their costs, and to use their market position to influence the markets they are operating in and drive down input costs.
This is specifically the case for organisations starting from a higher position with a ‘satisfactory’ or ‘strong’ MAR score – where the reform priorities set for them were more ambitious, recognising their good baseline performance.
chart 22: progress ratings for cost and value consciousness by organisation type GAVI GFATM EBRD CERF UNITAID EC Budget EDF CommSec IADB AfDF AsDF CDB UNAIDS PBF UNESCO UNICEF WHO FAO IFAD OHCHR UNDP UNFPA UNWomen PSD H GF MDB EC Other UN Organisation type Significant Reasonable Some Little or no 47.
Chart 22 shows progress by organisation type, showing that all 11 UN organisations assessed under the MAR Update have this component as a reform priority.

This reflects the MAR finding that this was a challenging area for them.

There was a sense that much of the UN system did not focus on cost and did not seek efficiencies.
However, it was also recognised that their regulatory frameworks, the higher costs associated with their headquarters and the fact they have to maintain a presence in a very wide range of countries all had an adverse impact on this.

Against this background, there has been progress on making cost savings, most notably demonstrated by UNAIDS (see box 7).

There is evidence that overriding UN processes still make it difficult for some UN organisations to control their costs.
This also causes challenges for other multilaterals which work through the UN system, and are affected by UN rules such as those around cost recovery rates over which they have no direct control, as is the case for the CERF.

48.

The MAR also noted that the multilateral development banks, although they had a good level of performance, needed to do more to help partners to consider cost-effectiveness, for example by more systematically preparing low cost options.
The multilateral development banks have made mixed progress specifically on this issue of ensuring value for money through partners, but there has been some progress on cost control.

58 49.

The EC instruments have made improvements to procurement regulations and administration costs have been kept down.

However, there is more to do on striving for value for money through the programming process.
Box 7: the Joint united nations programme on Hiv/aids (unaids) – strategically managing running costs and supporting partners on value for money In seeking value for money an organisation is deliberately assessing which policy and programme choices achieve the greatest results at the least cost.

UNAIDS has introduced a strategic investment framework, to ensure investments in HIV/AIDS programmes in countries are more cost-effective, efficient and have the biggest impact.
It helps countries develop their HIV/AIDS programmes by posing key investment questions to help inform the decisions they need to make.

29 countries are now implementing this investment approach.

UNAIDS achieved a 13% efficiency saving (approx.

$42 million) in administration costs between 2011 and 2012 following a wide-ranging organisational restructure which included a significant reduction in staff numbers.
Furthermore, the decreased budget managed by the UNAIDS Secretariat was reallocated to its partner agencies.

They have also improved their ability to monitor their spend through a new results management tool.

UNAIDS Copyrighted Material/P.

Virot (Test strip to be labelled with client identification number – Mali, 2008).

Attention to cross-cutting issues 50.


This component captures cross-cutting policy issues which are important for delivering development outcomes and which are therefore high priority for the UK coalition government, namely gender, working in fragile contexts, and climate change and environmental sustainability.

Gender was a priority for a number of organisations – though least progress has been made on this component.
Most progress has been made on climate change and environmental sustainability, 59 DRIVING REFORM TO ACHIEVE MULTILATERAL EFFECTIVENESS findinGs of tHe Mar update although only five organisations were assessed under this component.

Working in fragile contexts is in the middle in terms of progress, but with only seven organisations being assessed and mixed progress.

Gender 51.

The purpose of this component is to assess whether or not multilateral organisations are prioritising gender equality.
We were looking for policies, structures and incentives to promote gender equality, either directly or through partnerships, and evidence of impact, particularly in developing countries.

52.

Performance on gender was one of the weakest assessment areas in the MAR, with two thirds of organisations receiving ‘weak’ or ‘unsatisfactory’ ratings.
Organisations that were weak on gender were also weak on related assessment components such as strategic and performance management which influences an organisation’s ability to deliver and make gender an institutional priority; and contribution to results, which limits an organisation’s ability to deliver and demonstrate impact for girls and women.
chart 23: Mar score and Mar update progress ratings for gender EBRD PIDG GPE AsDF IDA IFC AfDF CDB CIFs EC Budget ECHO EDF IADB Unsatisfactory Weak Satisfactory Strong Significant Reasonable Some Little or no MAR score 53.

The MAR identified gender as a reform priority for 13 organisations, 12 of which had MAR scores of ‘unsatisfactory’ or ‘weak’; they are therefore starting from a lower base compared to the other components.
As discussed above, this is particularly significant as less than half made “reasonable” progress, suggesting performance remains at a lower level than on other components.

This is a matter for concern.

54.

The MAR found that a lack of prioritisation on gender at a strategic and structural level had a knock on effect of weak gender mainstreaming processes and limited impact in developing countries.

Reform priorities were therefore set to include these areas.
So, organisations that progressed well took actions that demonstrated their strategic prioritisation of gender including conducting formal or external reviews on their gender policy and programmes, increasing their 60 technical capacity, or improving measurement and data collection including disaggregating data by gender.
However, the MAR Update has shown that a gender action plan does not necessarily indicate gender being a strategic priority for a particular organisation as it may not be implemented completely or consistently throughout the organisation.

In fact, some of the organisations making “some” progress have experienced delays in putting these plans in place and have more to do to implement them and meet their targets.
chart 24: progress ratings for gender by organisation type s n o i t a s i n a g r o f o r e b m u N 3 2 1 0 1 2 EBRD IFC PIDG AsDF IDA GPE ECHO CIFs IADB AfDF CDB EC Budget EDF PSD H GF MDB EC Other UN Organisation type Significant Reasonable Some Little or no 55.

Chart 24 shows that, when considered by type of organisation, the three private sector development (PSD) organisations showed the greatest progress.
This was the result of focused planning and leadership on gender resulting in significant gender focus in policies and strong evidence of mainstreaming.

They instituted strong gender monitoring frameworks and improved collection of gender disaggregated data.

However, not enough time has passed to assess whether better data collection has an impact on investment choices.

The EC has an ambitious gender plan which should have impact on the EC instruments.

There has been some achievement on staff training.


Both global funds have made gender a strategic priority and have increased their gender expertise to reflect this.

As organisations focused on action taken in partnership, both funds are demonstrating leadership on gender in these partnerships.

The biggest challenge for the multilateral development banks as a group is lack of mainstreaming and low levels of strategic prioritisation of gender.
All the banks have increased staff with gender expertise and there has been some evidence of an improvement to the proportion of projects with a focus on gender.

The multilateral development banks are also doing more to report gender disaggregated results.

However, there is still limited evidence of the impact in developing countries.
61 DRIVING REFORM TO ACHIEVE MULTILATERAL EFFECTIVENESS findinGs of tHe Mar update Box 8: international development association (ida) making progress on gender mainstreaming In many developing countries, women lack a voice in their households, communities, and governments, as well as access to resources.

IDA therefore works to increase women’s economic opportunities and participation through access to land, financial services and other resources.
The challenge is significant: it is still the case that less than 10 per cent of credit going to small farmers in Africa is directed to women, even though they are a majority of agricultural workers.

Following the MAR, IDA made gender one of its strategic priorities, and committed to take specific measures to accelerate progress on gender mainstreaming and gender- related Millennium Development Goals (MDGs).
As a result, IDA increased the proportion of projects which had fully considered gender issues from 70 per cent in 2011 to 86 per cent in 2012.

Afghanistan has benefitted from this effort: 2.7 million girls were enrolled in schools in 2012, up from 191,000 in 2002; 16,300 women have joined savings groups between 2010 and 2013; and nearly 60 per cent of the new enterprise groups are made up of women with access to technical support for rural development projects.
This empowerment of women is intended to secure greater equality in Afghanistan as the country continues to develop.

DFID/CARD-F (CARD-F Project, Afghanistan, 2013).

Working in fragile contexts 56.

The purpose of this component was to assess whether or not organisations are active in fragile states and effective in these contexts.

We looked for evidence of performance in fragile states and of adaptation to fragile contexts.
Such evidence would be, for example, whether they had staff equipped to work in contexts of 62 conflict and fragility, good quality policy and operational guidance which is mainstreamed and used, and reporting which is used to inform policy and programming.

57.

Working in fragile contexts was a reform component for seven organisations, five of which were assessed as ‘weak’ in the MAR and two as ‘satisfactory’.
Mixed progress has been achieved with four of the seven organisations achieving “reasonable” progress and three “some”.

chart 25: Mar score and Mar update progress ratings for fragile contexts s n o i t a s i n a g r o f o r e b m u N 4 3 2 1 0 1 2 3 GFDRR IDA PIDG AfDF IFC GPE GAVI Unsatisfactory Weak Satisfactory Strong Significant Reasonable Some Little or no MAR score 58.
Reform priorities were mainly focused on organisations increasing their presence and investments in fragile states; and putting in place or improving policies for working within these contexts.

In response, the main reforms undertaken across the organisations include the opening of offices in more fragile and conflict affected states and ensuring that working in fragile contexts is a strategic priority for more organisations.
However, across the organisations it is not clear what the impact of these changes have been in these specific contexts.

59.

Looking at this by type of organisation, the MAR found that the multilateral development banks generally lacked adequate in-country capacity in fragile states, and had a limited range of financing instruments and inflexible procedures which were not always suitable for fragile contexts.
The two multilateral development banks for which this was a reform priority have made good progress.

Both have made fragile states a priority, and this is shown by improved speed of deployment of resources to fragile states and increased capacity and presence in fragile states.

For private sector organisations, the MAR noted that they only did limited work in fragile contexts, and that neither private sector organisations nor global funds had operational guidance on operating in fragile contexts.
The private sector development organisations assessed in the MAR Update have improved their focus on fragile states with greater presence, and now have minimum targets for investment in fragile and conflict affected states.

Box 9 illustrates what the Private Infrastructure Development Group (PIDG) is 63 DRIVING REFORM TO ACHIEVE MULTILATERAL EFFECTIVENESS findinGs of tHe Mar update doing to address the issues of fragile states.
The global funds assessed have put in place policies and systems to deal with the specific issues of fragile states, and one has expanded its presence in fragile states.

Box 9: the private infrastructure development Group (pidG) – investing in much needed infrastructure in fragile states Poverty is underpinned by inadequate access to basic services including infrastructure.
Situations of conflict and fragility often result in the destruction of countries’ infrastructure, further distancing poor communities from essential services and markets.

Due to the challenges that they pose, fragile and conflict affected states attract especially low levels of private infrastructure investment resulting in huge and growing infrastructure deficits.

PIDG seeks to mobilise private sector investment in infrastructure in developing countries.
To date, 45% of PIDG-supported projects have been in fragile and conflict affected states – meaning $13 billion has been invested in sectors including energy, transport, and water.

In 2012, PIDG adopted specific targets for investment in these states.

This has a real impact on development, for example in Kenya the demand for power far outstrips the supply, hampering the socio-economic development of the country’s people.
By lending $32.8 million and helping put together an investor consortium, PIDG facility – Emerging Africa Infrastructure Fund – enabled Kenya Power and Lighting Company to access $163.8 million of private sector investment.

This has enabled the development, operation and maintenance of Rabai Power Plant, which now provides reliable power to 4.2 million Kenyans.

EAIF/PIDG (Rabai Power Plant, Kenya, 2012).

64 Climate change and environmental sustainability 60.
This component looks at whether multilateral organisations ensure that their development or humanitarian activities are low carbon, climate resilient and environmentally sustainable.

The assessment was based on whether organisations had a climate change and/or environment strategy in place, or a framework for guiding policies and resource allocation that incorporates climate change.

61.

In this Update, climate change and environmental sustainability was a reform priority for five organisations.
Three of these organisations were rated as ‘weak’ in the MAR, one as ‘satisfactory’ and another as ‘strong’.

Where relevant, reform priorities were mainly focused on increasing levels of support for climate projects; putting in place relevant policies; and addressing climate issues in operations and delivery.

Overall there has been good progress, with four of the five organisations achieving “reasonable” or “significant” progress.
Reforms undertaken include organisations increasing their climate change expertise through the recruitment of experts, or setting themselves climate targets, which they are exceeding.

The multilateral development banks and the private sector development organisation, assessed under this component, have prioritised climate and environmental concerns and included these in their overall objectives.

Summary of progress 62.


The analysis presented has given the results for all organisations in chart 7, showing that most progress has been made by organisations that performed strongly, and those which performed poorly in the MAR.
The analysis of progress against specific components shows that while there has been good progress on key components such as financial resources management, partnership behaviour, and contribution to results, there has been disappointing progress on the equally important components gender, transparency and accountability, and cost and value consciousness.
The more detailed analysis by component highlighted the actions taken by agencies to meet their reform priorities, and considers these by organisation type.

Table 10 provides a summary of the progress made by the different types of organisations, against the main weaknesses set out in the MAR report19 and do not cover all the reform priorities set for the different types of organisations assessed in the MAR Update.

19 As set out in table 7 on page 21 of the MAR.
65 DRIVING REFORM TO ACHIEVE MULTILATERAL EFFECTIVENESS findinGs of tHe Mar update table 10: summary of main reforms and outstanding issues by organisation type Main weaknesses identified in the Mar Gender and fragile contexts Multilateral development banks Private sector development Focus on poor countries, gender and fragile contexts Main reforms undertaken outstanding issues ■■ Further implementation of gender strategies and demonstrable evidence of impact ■■ More staff with gender expertise ■■ More projects focused on gender and increased reporting of results ■■ Faster deployment of resources to fragile states ■■ Presence in more fragile states ■■ Increased gender focus ■■ Greater use of gender disaggregated data and clear gender targets for specific sectors ■■ More, and higher quality, investments in fragile states in policies ■■ Gender mainstreaming and monitoring frameworks ■■ Improved collection of gender disaggregated data ■■ Presence in more fragile states ■■ Minimum targets for investment in fragile states Global funds Gender and fragile contexts ■■ More staff with gender ■■ More evidence of expertise ■■ Leadership on gender among partners ■■ Policies and systems in place on fragile states improved approaches to working in fragile states European Commission exc.
humanitarian Strategic and performance management and gender ■■ Introduction of a results framework, although delayed ■■ HR reform ■■ Improving evaluation ■■ Strengthening implementation of the Gender Action Plan 66 table 10: summary of main reforms and outstanding issues by organisation type UN exc.
Humanitarian Main weaknesses identified in the Mar Contribution to results, cost and value consciousness, strategic and performance management, transparency, and financial resources management Humanitarian Strategic and performance management and transparency Main reforms undertaken outstanding issues ■■ More streamlined strategic objectives and improved results frameworks ■■ Improved human resource capacity in developing countries ■■ Cost savings ■■ Strengthened financial management, audit and administrative systems ■■ More agencies signed up to, and implementing, IATI ■■ Better and more accessible data on websites ■■ Improved strategies which better define organisational objectives and target action on the ground ■■ Better partnership working with other multilaterals, civil society and private sector ■■ Some strengthening of evaluation ■■ Clearer role for advisory groups and revised selection criteria for membership in order to strengthen accountability ■■ Demonstrating results and better articulation of the connection between activities and outcomes in developing countries ■■ Continued focus on cost savings, especially administration costs ■■ More strategic use of financial resources ■■ Better articulation of the connection between activities and outcomes in developing countries ■■ Further strengthening of evaluation functions, in particular using evaluation findings to inform decision making ■■ Going further to improve accountability to beneficiaries and other stakeholders including NGOs 67 DRIVING REFORM TO ACHIEVE MULTILATERAL EFFECTIVENESS findinGs of tHe Mar update 63.
Finally, although this chapter has concentrated on the overview of progress made, identifying patterns and the most important reforms, it is the case that each organisation is different and each has their own story of change.

Table 11 provides short summaries of progress for each of the multilateral organisations assessed in the MAR Update.

The diagrams reproduce the summary of progress from the MAR, with an overlay of the results of the MAR Update.
Two-page summary assessments which give more detail for each organisation are provided in annex 3.
68 table 11: spider diagrams20 contribution to uK development objectives organisational strengths summary of progress afdf MAR VFM: Good MAR Update rating: Contribution to results 2 Climate change/ environment Criticality to international aid objectives 4 3 2 1 0 Criticality to UK aid objectives Transparency and accountability Focus on poor countries Cost and value consciousness 4 3 2 1 0 AfDF has improved its results reporting systems.
It has increased staff presence and authority in fragile states with some evidence of performance improving in these countries.

The Climate Change Action Plan now has more clearly defined objectives.

The new gender strategy was severely delayed and the AfDF need to do more to explore lower cost options in policy and programmes.
Partnership behaviour Gender Fragile contexts Financial resources management Strategic and performance management asdf MAR VFM: very good MAR Update rating: Contribution to results 3 Climate change/ environment Criticality to international aid objectives 4 3 2 1 0 Criticality to UK aid objectives Transparency and accountability Focus on poor countries Cost and value consciousness 4 3 2 1 0 AsDF has put in place a new results framework which captures inclusive growth and includes ambitious gender targets.
It has improved its capacity in developing countries and increased private sector finance for climate change mitigation.

There is limited evidence of efficiency gains, and efforts to address poor project performance have yet to have impact.
Partnership behaviour Gender Fragile contexts Financial resources management Strategic and performance management 20 The colour of the shaded area within each spider diagram reflects the original MAR score for that composite index (ie strong (light green), satisfactory (dark green), weak (amber) and unsatisfactory (red) – see page 14 of the MAR for details.
The seven score changes (see table 8 above) are shown as lighter shaded areas for the respective organisations, if the score change would change the original MAR score for the composite index the score change area is shaded to reflect what that new score would be.
6 9 DRIVING REFORM TO ACHIEVE MULTILATERAL EFFECTIVENESS 7 0 contribution to uK development objectives organisational strengths summary of progress cdB MAR VFM: adequate MAR Update rating: Contribution to results 2 Climate change/ environment Criticality to international aid objectives 4 3 2 1 0 Criticality to UK aid objectives Transparency and accountability Focus on poor countries Cost and value consciousness 4 3 2 1 0 CDB has started work to improve its operational policies, strategies and guidelines, and its corporate oversight.
It has improved its transparency through a new information disclosure policy, increasing the independence of its evaluations and improving its website content.

There is still limited evidence on its efficiency gains and value for money improvements and further work is needed on results management, value for money and gender mainstreaming.
Partnership behaviour Gender Fragile contexts Financial resources management Strategic and performance management cerf MAR VFM: Good MAR Update rating: Contribution to results 2 Climate change/ environment Criticality to international aid objectives 4 3 2 1 0 Criticality to UK aid objectives Transparency and accountability Focus on poor countries Cost and value consciousness 4 3 2 1 0 Partnership behaviour Gender Fragile contexts Financial resources management Strategic and performance management CERF has improved the way it prioritises funding to under-funded emergencies.
It has also strengthened its reporting and is holding partner agencies to greater account for how funds are used.

The CERF will need to continue its work with UN partners to improve the speed at which they disburse funds to NGOs and to improve its ability to demonstrate results.

The value for money of the management charge had not been appropriately examined by the CERF; this has been acted on following the MAR Update assessment.
I F I N D N G S O F T H E M A R U P D A T E 7 1 contribution to uK development objectives organisational strengths summary of progress cifs MAR VFM: Good MAR Update rating: Contribution to results 3 Climate change/ environment Criticality to international aid objectives 4 3 2 1 0 Criticality to UK aid objectives Transparency and accountability Focus on poor countries Cost and value consciousness 4 3 2 1 0 CIFs have commissioned an independent gender review and are taking forward its recommendations but it is too early to see impact.
They have implemented revised results frameworks and improved disbursement speeds.

They improved their transparency through signing up to IATI.

It is too early to see the results of these reforms in developing countries.
Partnership behaviour Gender Fragile contexts Financial resources management Strategic and performance management commsec MAR VFM: poor MAR Update rating: Contribution to results 2 Climate change/ environment Criticality to international aid objectives 4 3 2 1 0 Criticality to UK aid objectives Transparency and accountability Focus on poor countries Cost and value consciousness 4 3 2 1 0 CommSec has made reasonable progress on financial management, and has had unqualified financial statements for the last two years – achieving a score change for this component from 1 to 2.
There is more to do on other reforms, particularly on prioritising its work in line with international aid objectives and in areas where it has comparative advantage.
Partnership behaviour Gender Fragile contexts Financial resources management Strategic and performance management 7 1 DRIVING REFORM TO ACHIEVE MULTILATERAL EFFECTIVENESS 7 2 contribution to uK development objectives organisational strengths summary of progress eBrd MAR VFM: Good MAR Update rating: Contribution to results 2 Climate change/ environment Criticality to international aid objectives 4 3 2 1 0 Criticality to UK aid objectives Transparency and accountability Focus on poor countries Cost and value consciousness 4 3 2 1 0 Partnership behaviour Gender Fragile contexts Financial resources management Strategic and performance management ec Budget MAR VFM: adequate MAR Update rating: Contribution to results 2 Climate change/ environment Criticality to international aid objectives 4 3 2 1 0 Criticality to UK aid objectives Transparency and accountability Focus on poor countries Cost and value consciousness 4 3 2 1 0 Partnership behaviour Gender Fragile contexts Financial resources management Strategic and performance management EBRD has made reasonable progress on implementing its new gender action plan, and shows commitment to a more strategic consideration of gender issues.
On climate change, it has expanded energy efficiency lending into new and difficult markets.

Some progress has been made towards articulating the impact of EBRD’s work on people’s lives, but it has more to do to broaden its results framework.

The EBRD performs well currently on cost and value consciousness.

This should be reinforced through the implementation of the results of internal reviews of personnel policies, procurement and processes in 2013 and 2014.
The EC has made some progress against the Gender Action Plan but more attention is needed, particularly at senior management level.

The EC is expected to implement a new results framework in mid-2014 but progress on this has been slower than expected.

There has been little progress on evaluation, where more needs to be done to ensure follow-up to evaluations.

The DCI instrument of the EC budget will focus more on the poorest and most fragile countries.
More attention needs to be placed on value for money in programming aid.
7 3 contribution to uK development objectives organisational strengths summary of progress ecHo MAR VFM: very good MAR Update rating: Contribution to results 2 Climate change/ environment Criticality to international aid objectives 4 3 2 1 0 Criticality to UK aid objectives Transparency and accountability Focus on poor countries Cost and value consciousness 4 3 2 1 0 Partnership behaviour Gender Fragile contexts Financial resources management Strategic and performance management ECHO has shown leadership and innovation in working with DevCo and EEAS on resilience- building initiatives in the Horn of Africa and the Sahel.
It has also developed a ‘gender marker’ to track gender-sensitive projects – this now needs to be routinely used across their programmes.

There is some evidence of climate change/environment being taken into account in planning and evaluation, but more work is needed to strengthen processes for measuring the environmental impact of its projects.
edf MAR VFM: very good MAR Update rating: Contribution to results 2 Climate change/ environment Criticality to international aid objectives 4 3 2 1 0 Criticality to UK aid objectives Transparency and accountability Focus on poor countries Cost and value consciousness 4 3 2 1 0 Partnership behaviour Gender Fragile contexts Financial resources management Strategic and performance management The EC has made some progress against the Gender Action Plan but more attention is needed, particularly at senior management level.
The EC is expected to implement a new results framework in mid-2014 but progress on this has been slower than expected.

There has been little progress on evaluation, where more needs to be done to ensure follow-up to evaluations.

The new EDF allocation model will increase the focus of resources on the poorest and most fragile countries.

More attention needs to be placed on value for money in programming aid.
7 3 DRIVING REFORM TO ACHIEVE MULTILATERAL EFFECTIVENESS 7 4 contribution to uK development objectives organisational strengths summary of progress fao MAR VFM: poor MAR Update rating: Contribution to results 3 Climate change/ environment Criticality to international aid objectives 4 3 2 1 0 Criticality to UK aid objectives Transparency and accountability Focus on poor countries Cost and value consciousness 4 3 2 1 0 FAO has increased its focus on results, having introduced new results frameworks at country and corporate levels.
New leadership is introducing a greater focus on value for money and has achieved significant savings.

HR practice is improving with better recruitment processes, but performance management needs to be strengthened.

Transparency and accountability are improving, although the introduction of some new systems has been delayed.
Partnership behaviour Gender Fragile contexts Financial resources management Strategic and performance management Gavi MAR VFM: very good MAR Update rating: Contribution to results 3 Climate change/ environment Criticality to international aid objectives 4 3 2 1 0 Criticality to UK aid objectives Transparency and accountability Focus on poor countries Cost and value consciousness 4 3 2 1 0 GAVI has approved a new supply and procurement strategy and has achieved significant price reductions on vaccines.
They have increased engagement with civil society and clarified the roles and responsibilities of partners.

GAVI has also implemented a new policy on fragile states, but it is too early to see impact.

They are taking steps to improve the management of cash-based programmes, but changes are yet to have sufficient impact.
Partnership behaviour Gender Fragile contexts Financial resources management Strategic and performance management 7 5 contribution to uK development objectives organisational strengths summary of progress Gef MAR VFM: Good MAR Update rating: Contribution to results 3 Climate change/ environment Criticality to international aid objectives 4 3 2 1 0 Criticality to UK aid objectives Transparency and accountability Focus on poor countries Cost and value consciousness 4 3 2 1 0 Partnership behaviour Gender Fragile contexts Financial resources management Strategic and performance management GEF introduced ways of supporting recipient countries to align GEF funding with their national strategies and increased the number of implementing agencies, to give countries a greater choice of partner.
It has also strengthened its results based management framework, to provide a better overview of its portfolio.

Early evidence suggests an improvement in the average time taken from project approval to implementation, however, more evidence of progress on the majority of current projects is required.
GfatM MAR VFM: very good MAR Update rating: Contribution to results 3 Climate change/ environment Criticality to international aid objectives 4 3 2 1 0 Criticality to UK aid objectives Transparency and accountability Focus on poor countries Cost and value consciousness 4 3 2 1 0 Partnership behaviour Gender Fragile contexts Financial resources management Strategic and performance management GFATM has increased its awareness and control of costs.
This has been shown through reduction of staff numbers and operating expenses, but better value for money in costs and outcomes needs to be spread throughout its operations.

Human resources are now more focused on those countries with the greatest needs.

Their new funding model is more reliant on national strategies and there is greater stakeholder participation, though it is too early to see full impact of these changes in the reduction of burdens placed on implementing countries.
7 5 DRIVING REFORM TO ACHIEVE MULTILATERAL EFFECTIVENESS 7 6 contribution to uK development objectives organisational strengths summary of progress Gfdrr MAR VFM: Good MAR Update rating: Contribution to results 2 Climate change/ environment Criticality to international aid objectives 4 3 2 1 0 Criticality to UK aid objectives Transparency and accountability Focus on poor countries Cost and value consciousness 4 3 2 1 0 GFDRR is making progress to mainstream disaster risk management.
It now needs to demonstrate the impact of these investments.

GFDRR is now actively working to improve its monitoring and reporting of results; this is work in progress.

GFDRR is effectively supporting the development of risk financing frameworks, and has improved its engagement in fragile states and with civil society.
Partnership behaviour Gender Fragile contexts Financial resources management Strategic and performance management Gpe MAR VFM: Good MAR Update rating: Contribution to results 3 Climate change/ environment Criticality to international aid objectives 4 3 2 1 0 Criticality to UK aid objectives Transparency and accountability Focus on poor countries Cost and value consciousness 4 3 2 1 0 Partnership behaviour Gender Fragile contexts Financial resources management Strategic and performance management GPE has improved its results reporting but needs to strengthen this further and use results to drive performance.
It has adapted its model to make it easier for fragile countries to access support and increased the number of fragile states it funds.

GPE has increased its engagement with countries on gender and improved its financial management including making grants to countries more transparent and predictable.

It has strengthened its leadership and management but it is too early to see the impact of this.
7 7 contribution to uK development objectives organisational strengths summary of progress iadB MAR VFM: adequate MAR Update rating: Contribution to results 2 Climate change/ environment Criticality to international aid objectives 4 3 2 1 0 Criticality to UK aid objectives Transparency and accountability Focus on poor countries Cost and value consciousness 4 3 2 1 0 IADB has improved its results measurement and monitoring, particularly at the project level.

They have made some progress on gender.


They met targets on gender diversity of staff, but have more to do to make gender an integral part of loan operations.

Policy on information disclosure needs further clarification and its independent complaints mechanism needs reform.
Partnership behaviour Gender Fragile contexts Financial resources management Strategic and performance management icrc MAR VFM: very good MAR Update rating: Contribution to results 3 Climate change/ environment Criticality to international aid objectives 4 3 2 1 0 Criticality to UK aid objectives Transparency and accountability Focus on poor countries Cost and value consciousness 4 3 2 1 0 Partnership behaviour Gender Fragile contexts Financial resources management Strategic and performance management ICRC has made good progress on climate change and the environment, with a new strategic framework and further work to understand and mitigate the environmental impact of its activities.
It has also made progress on increasing transparency, particularly with donors and continues to ensure beneficiaries are taken into account in programmes and planning.

More evidence of accountability to beneficiaries, including through the increased use of evaluation, is needed.
7 7 DRIVING REFORM TO ACHIEVE MULTILATERAL EFFECTIVENESS 7 8 contribution to uK development objectives organisational strengths summary of progress ida MAR VFM: very good MAR Update rating: Contribution to results 3 Climate change/ environment Criticality to international aid objectives 4 3 2 1 0 Criticality to UK aid objectives Transparency and accountability Focus on poor countries Cost and value consciousness 4 3 2 1 0 Partnership behaviour Gender Fragile contexts Financial resources management Strategic and performance management IDA has prioritised working in fragile contexts.
It has increased its focus on gender, but this has yet to be fully integrated as part of its operations.

It has improved results reporting across the organisation and is modernising procedures to improve partnership working.

It has better budget and work programme information, but could do more to discuss cost effectiveness with partners.

It is too early to determine the impact in developing countries of current corporate strategy reforms.
ifad MAR VFM: Good MAR Update rating: Contribution to results 3 Climate change/ environment Criticality to international aid objectives 4 3 2 1 0 Criticality to UK aid objectives Transparency and accountability Focus on poor countries Cost and value consciousness 4 3 2 1 0 IFAD has increased its focus on results, introduced impact evaluation and made progress on HR reform.

It has strengthened its financial management and made some cost savings.
IFAD needs to do more to improve efficiency: an action plan has been developed but it is too early to judge impact.
Partnership behaviour Gender Fragile contexts Financial resources management Strategic and performance management 7 9 contribution to uK development objectives organisational strengths summary of progress ifc MAR VFM: Good MAR Update rating: Contribution to results 2 Climate change/ environment Criticality to international aid objectives 4 3 2 1 0 Cost and value consciousness 4 3 2 1 0 Criticality to UK aid objectives Transparency and accountability Focus on poor countries IFC has made progress on targeting investments at women, including increasing access to finance for women-owned businesses.
It has made efforts to work more with others.

More progress is needed to increase investment in fragile contexts, and expand its portfolio across low income countries.
Partnership behaviour Gender Fragile contexts Financial resources management Strategic and performance management ifrc MAR VFM: Good MAR Update rating: Contribution to results 2 Climate change/ environment Criticality to international aid objectives 4 3 2 1 0 Criticality to UK aid objectives Transparency and accountability Focus on poor countries Cost and value consciousness 4 3 2 1 0 IFRC has worked to put new policies and systems in place, with a focus on performance management of national societies, although their ability to have impact on these societies remains limited.
There is improved joint working with agencies outside the Red Cross Movement, but sustained effort is needed to ensure accountability to donors.
Partnership behaviour Gender Fragile contexts Financial resources management Strategic and performance management 7 9 DRIVING REFORM TO ACHIEVE MULTILATERAL EFFECTIVENESS 8 0 contribution to uK development objectives organisational strengths summary of progress ioM MAR VFM: poor MAR Update rating: Contribution to results 3 Climate change/ environment Criticality to international aid objectives 4 3 2 1 0 Criticality to UK aid objectives Transparency and accountability Focus on poor countries Cost and value consciousness 4 3 2 1 0 IOM has put in place measures to strengthen its humanitarian response capacity and also improve its leadership of the Camp Coordination and Camp Management Cluster.
It has introduced a results based approach to project management to monitor the progress of its projects.

Evaluation remains under-resourced and the absence of an institutional results framework remains a weakness.
Partnership behaviour Gender Fragile contexts Financial resources management Strategic and performance management ocHa MAR VFM: Good MAR Update rating: Contribution to results 2 Climate change/ environment Criticality to international aid objectives 4 3 2 1 0 Criticality to UK aid objectives Transparency and accountability Focus on poor countries Cost and value consciousness 4 3 2 1 0 OCHA has improved its support to humanitarian leadership.
It has also improved its staff planning, deploying staff more quickly and reducing vacancy rates.

More work is needed to ensure that OCHA achieves better field capacity.

More progress is needed on information management and supporting the link between humanitarian needs, appeals and programming.
Partnership behaviour Gender Fragile contexts Financial resources management Strategic and performance management 8 1 contribution to uK development objectives organisational strengths summary of progress oHcHr MAR VFM: adequate MAR Update rating: Contribution to results 2 Criticality to international aid objectives 4 3 2 1 0 Transparency and accountability Criticality to UK aid objectives Cost and value consciousness 4 3 2 1 0 OHCHR has introduced results frameworks across regional and country offices and rolled out a new performance monitoring system which allows staff to plan, monitor and report on results.
This has strengthened results management and fostered a stronger results culture within the organisation.

Greater progress needs to be made on establishing an evaluation function and pursuing efficiency savings.
Partnership behaviour Gender Fragile contexts Financial resources management Strategic and performance management pBf MAR VFM: Good MAR Update rating: Contribution to results 3 Gender Criticality to international aid objectives 4 3 2 1 0 Fragile contexts Criticality to UK aid objectives Focus on poor countries Transparency and accountability Cost and value consciousness 4 3 2 1 0 PBF has introduced new reporting procedures with a stronger focus on results, and now requires partners to demonstrate better their strategic contribution to peacebuilding.
It is more proactively seeking to improve value for money through project design and working with partners.

PBF urgently needs to strengthen the effectiveness of the management structures that oversee how the fund is spent in developing countries.
Partnership behaviour Financial resources management Strategic and performance management 8 1 DRIVING REFORM TO ACHIEVE MULTILATERAL EFFECTIVENESS 8 2 contribution to uK development objectives organisational strengths summary of progress pidG MAR VFM: very good MAR Update rating: Contribution to results 3 Climate change/ environment Criticality to international aid objectives 4 3 2 1 0 Criticality to UK aid objectives Transparency and accountability Focus on poor countries Cost and value consciousness 4 3 2 1 0 Partnership behaviour Gender Fragile contexts Financial resources management Strategic and performance management PIDG has introduced a formal policy on fragile and conflict affected states and is now monitoring and reporting gender disaggregated data.
PIDG needs to continue to improve in these areas by implementing recommendations from a strategic review, which included scaling up work in water, sanitation and agriculture.

PIDG has taken steps to improve transparency including introducing a disclosure policy, improving information on its website and producing IATI compliant reports but transparency remains weak.
unaids MAR VFM: adequate MAR Update rating: Contribution to results 3 Criticality to international aid objectives 4 3 2 1 0 Transparency and accountability Criticality to UK aid objectives Cost and value consciousness 4 3 2 1 0 UNAIDS has improved its results framework and developed a web-based tool to collect results from partner agencies.

It made significant efficiency savings and moved to one administration system.
UNAIDS resources are now more focused on strategic priorities, but more needs to be done to target resources to countries with the highest need.
Partnership behaviour Gender Fragile contexts Financial resources management Strategic and performance management 8 3 contribution to uK development objectives organisational strengths summary of progress undp MAR VFM: Good MAR Update rating: Contribution to results 2 Climate change/ environment Criticality to international aid objectives 4 3 2 1 0 Criticality to UK aid objectives Transparency and accountability Focus on poor countries Cost and value consciousness 4 3 2 1 0 Partnership behaviour Gender Fragile contexts Financial resources management Strategic and performance management UNDP has improved its strategic direction, corporate planning and results framework.
It showed leadership in improving the Resident Coordinator system and encouraging IATI implementation.

It has reduced staff recruitment times, but needs to do more on staff performance.

There are some improvements to procurement, but clearer targets are needed for cost control.

There are indications of improved performance in developing countries though more progress is required to improve programme management and results reporting.
unesco MAR VFM: poor MAR Update rating: Contribution to results 3 Climate change/ environment Criticality to international aid objectives 4 3 2 1 0 Criticality to UK aid objectives Transparency and accountability Focus on poor countries Cost and value consciousness 4 3 2 1 0 UNESCO has improved its results based management frameworks and their coverage.

It increasingly implements evaluation recommendations and focuses more on its areas of strength.
It has made further savings on its overhead costs and put in place a new financial disclosure policy.

More work on transparency is needed.
Partnership behaviour Gender Fragile contexts Financial resources management Strategic and performance management 8 3 DRIVING REFORM TO ACHIEVE MULTILATERAL EFFECTIVENESS 8 4 contribution to uK development objectives organisational strengths summary of progress unfpa MAR VFM: adequate MAR Update rating: Contribution to results 2 Climate change/ environment Criticality to international aid objectives 4 3 2 1 0 Criticality to UK aid objectives Transparency and accountability Focus on poor countries Cost and value consciousness 4 3 2 1 0 UNFPA has improved its integrated results framework and has addressed issues in financial management, having implemented IPSAS and received an unqualified financial statement, and improved its procurement.
It has signed up to IATI and established an evaluation function.

These reforms now need to demonstrate impact throughout the agency and in particular in developing countries.
Partnership behaviour Gender Fragile contexts Financial resources management Strategic and performance management unHcr MAR VFM: Good MAR Update rating: Contribution to results 2 Climate change/ environment Criticality to international aid objectives 4 3 2 1 0 Criticality to UK aid objectives Transparency and accountability Focus on poor countries Cost and value consciousness 4 3 2 1 0 UNHCR has published a formal disclosure policy, and is now sharing programme information.
Although UNHCR has taken steps to improve its partnership behaviour, such as its cluster leadership, and with WFP and UNICEF, cooperation with other agencies in-country remains an area for improvement.
Partnership behaviour Gender Fragile contexts Financial resources management Strategic and performance management 8 5 contribution to uK development objectives organisational strengths summary of progress unicef MAR VFM: very good MAR Update rating: Contribution to results 3 Climate change/ environment Criticality to international aid objectives 4 3 2 1 0 Criticality to UK aid objectives Transparency and accountability Focus on poor countries Cost and value consciousness 4 3 2 1 0 UNICEF has improved its procedures for operating in humanitarian crises, but needs to increase the speed with which it disburses funding to partners on the ground.
It is demonstrating greater attention to value for money and transparency, including through procurement savings and implementing IATI.

Its results reporting needs to be improved and it could contribute more to UN coordination in developing countries.
Partnership behaviour Gender Fragile contexts Financial resources management Strategic and performance management unitaid MAR VFM: Good MAR Update rating: Contribution to results 2 Gender Criticality to international aid objectives 4 3 2 1 0 Fragile contexts Criticality to UK aid objectives Focus on poor countries Transparency and accountability Cost and value consciousness 4 3 2 1 0 UNITAID has a new strategy to define funding priorities and is more robust in appraising projects.
It has improved financial management systems, but could be stronger in seeking better value for money.

More reform on transparency is needed, especially in publishing project documents in a consistent and accessible manner.
Partnership behaviour Financial resources management Strategic and performance management 8 5 DRIVING REFORM TO ACHIEVE MULTILATERAL EFFECTIVENESS 8 6 contribution to uK development objectives organisational strengths summary of progress un women N/A (UNIFEM – poor value for money) MAR Update rating: Contribution to results 3 Gender wfp MAR VFM: Good MAR Update rating: Contribution to results 3 Climate change/ environment Criticality to international aid objectives 4 3 2 1 0 Fragile contexts Criticality to international aid objectives 4 3 2 1 0 Transparency and accountability Cost and value consciousness 4 3 2 1 0 Partnership behaviour Financial resources management Strategic and performance management UN Women has been assessed as a new organisation, facing legacy issues from its main predecessor organisation, UN’s Development Fund for Women (UNIFEM).
It has introduced a results framework and reporting, and has acted on evidence to improve its results.

UN Women has signed up to IATI and is implementing a disclosure policy.

It needs to continue to improve its performance in-country.

It needs to further demonstrate its cost and value consciousness.

Criticality to UK aid objectives Focus on poor countries Cost and value consciousness 4 3 2 1 0 WFP has improved its emergency response capacity and has put in place measures to decrease response times.
It has also strengthened its approach to value for money.

WFP has shown commitment to improving results reporting, evaluation, transparency and risk management.

It must now ensure these reforms are consistently applied in the field to deliver appropriate programmes.
Partnership behaviour Criticality to UK aid objectives Transparency and accountability Focus on poor countries Gender Fragile contexts Financial resources management Strategic and performance management I F I N D N G S O F T H E M A R U P D A T E 8 7 contribution to uK development objectives organisational strengths summary of progress wHo MAR VFM: adequate MAR Update rating: Contribution to results 3 Climate change/ environment Criticality to international aid objectives 4 3 2 1 0 Criticality to UK aid objectives Transparency and accountability Focus on poor countries Cost and value consciousness 4 3 2 1 0 WHO has developed clear results frameworks that link activities to health and development outcomes.
It has better information about its use of resources and its programmes.

It has fully implemented IPSAS and strengthened its internal control framework and audit capacity.

It has also reduced costs, for example through its relocation of staff to Kuala Lumpur.

More progress is needed on improving performance management systems, particularly for staff and results.
Partnership behaviour Gender Fragile contexts Financial resources management Strategic and performance management 8 7 DRIVING REFORM TO ACHIEVE MULTILATERAL EFFECTIVENESS chapter 5 The use of the MAR Update 1.

This chapter explains how we have used the MAR Update in making funding decisions and in taking forward our areas of reform for multilateral organisations.

It also sets out our plans for future assessments of multilateral effectiveness.

Funding 2.
Funding 2.

All DFID funding to multilateral organisations is related to performance.

Following the MAR, DFID announced funding increases to the best performing organisations and withdrew voluntary core aid funding from four organisations that were assessed as poor value for money for UK aid.

The vast majority (95%) of the UK’s funding to multilateral organisations now goes to organisations that were assessed as very good (73%) or good (22%) value for money for UK aid.

3.
3.

As with the MAR, the progress reviews and updated assessments from the MAR Update have informed UK decisions on core aid funding through multilateral organisations.

The timings of these vary.

Many organisations, for example the UN and humanitarian organisations, are funded on a regular annual or biennial basis.
Others – for example, the global funds, international financial institutions and the European Development Fund – are funded through regular replenishments and for most of these, no funding decision has been taken since the completion of their MAR Update assessment.

4.
While the funding for each organisation is decided on its own specific circumstances, some common factors are taken into account: ■■ delivery: what outputs and outcomes will be achieved through that funding; ■■ ■■ ■■ ■■ strategic fit: how this fits with what we in the UK are trying to achieve in humanitarian action and development; reform: the extent to which they have been able to reform in line with our expectations; comparison with other delivery routes: whether investment in this organisation offers the best way of delivering our objectives or whether another partner would be better; and contributions from others: whether UK funding will help to catalyse, or risk displacing, funding from others.
5.

Although we present the funding decision here in terms of the MAR and MAR Update assessment results, these other factors are also very important, as explained below: 88 ■■ MAR score of very good or good and on average “reasonable” progress: the UK government has continued to invest significant resources in these organisations, while also asking for further improvements.

We can expect them to continue to deliver good, and improving, value for money for our investment.
We will consider funding increases based on their expected results where there continues to be a good strategic fit with DFID priorities and complementarity with the DFID bilateral programme.

■■ MAR score of very good or good and on average “some” progress: organisations with these characteristics are very similar to those above – they remain high performing organisations which deliver important development and humanitarian results.
However, given that progress on reform has been slower than we would have liked, we are emphasising the need for further reform by, for example, making some funding conditional on progress on a specific reform, or by taking overall reform progress into account in more frequent reconsideration of budget levels.

■■ MAR score of adequate or poor and on average “reasonable” rating: funding for organisations with lower MAR scores was carefully considered following the 2011 exercise.
Funding continued where organisations had an especially good fit with the UK’s objectives, or a particular niche in the international system for development and humanitarian objectives that others cannot fill.

The UN agencies which were in “special measures” fall within this description.

We are pleased at the progress they made and we are therefore continuing to fund their UK assessed contribution, subject to further review in 2015.
The evidence of reform progress from the MAR Update has also given us the confidence to consider increases to the adequate value for money organisations in this group, which we had previously held down at 2010 levels.

■■ MAR score of adequate or poor and on average “some” rating: as discussed, these organisations are important to development objectives and to wider UK strategic interests.

All have made some progress against their UK reform priorities, but below the level that we were looking for.
Given their importance to the UK, DFID is continuing to fund all of these organisations, with an increasing emphasis on the need for progress on reform objectives.

The European Commission budget is fixed under international agreements.

Funding to the other organisations in this group will remain at or below 2010 levels, meaning that the real value of UK contributions will continue to fall.

Further action on reform 6.
Following the MAR in 2011, we identified specific reform priorities for each of the multilateral organisations that DFID continued to fund, and placed particular stress on strengthening seven key aspects of performance across the multilaterals: ■■ Accountability for results ■■ Delivery of efficiency savings and value for money in programming 89 DRIVING REFORM TO ACHIEVE MULTILATERAL EFFECTIVENESS tHe use of tHe Mar update ■■ Human resource management ■■ Transparency and accountability ■■ Delivering for girls and women ■■ Working in fragile contexts ■■ Partnership behaviour 7.
As shown in chapter 4, progress has been uneven across these important areas.

We are concerned in particular at slow progress on delivering for girls and women, transparency and accountability, and the delivery of efficiency savings and value for money in programming.

These will be important areas of focus for us going forward.

8.

We recognise that meaningful reform takes time.


Following the MAR Update, we have therefore kept the same broad reform priorities for each organisation, adjusting them only to reflect what has been achieved so far, and the remaining challenges.

The summary assessments, summaries of progress in Table 11 and the discussion in chapter 4 of the components and common issues which need addressing give more detail on the areas of focus for the next two years.
We would also point out that a future MAR – as discussed below – will be a full reconsideration of every component.

We will be looking to see higher levels of performance across all of these priorities at the time of this assessment.

9.

Having said this, we will be giving particular attention to the areas where progress is disappointing.
The first step will be to understand better the reasons for this: we will be doing further analysis on the Update findings, and we would also be very interested in hearing from multilateral organisations themselves about their understanding of the issues involved.

We will also be working to refine our view and description of what good performance looks like in these, and other component areas.
This will build on what we have learned since the MAR in 2011, in particular through the work of those multilateral organisations which have made better progress than others.

We will also continue to work closely with other donors to develop a common view on good practice and shared expectations for multilateral organisations.

We will then be in a better place to ensure that the right actions are taken to bring about better performance and for this to be demonstrated through a future MAR.
Future plans for multilateral effectiveness assessment 10.

The UK coalition government is committed to ensuring that the UK taxpayer receives value for money from their investment in multilateral organisations.

We continue to work with multilateral organisations to support them being as effective as they can be – recognising their crucial importance to delivering humanitarian and development aid.
This MAR Update demonstrates the importance we attach to undertaking assessments of multilateral effectiveness 90 and reinforces that this cannot be a one-off exercise.

The government has therefore already committed to undertaking a further MAR in 2015.

This will be a full reassessment of agencies across a complete set of components, unlike this Update which focused attention on the particular areas where we had asked organisations to improve.

11.
11.

As a MAR in 2015 will be a comprehensive re-appraisal of multilateral organisations, it is also a good opportunity to reconsider and refresh the methodology for this multilateral assessment.

In 2014 we will be taking forward work to develop an approach for a future MAR.

Our intention is to give multilateral organisations sufficient advance notice about the exact nature of the assessment.
There has already been extensive commentary on the MAR methodology as we outlined in chapter 1, and, where possible, we have made consequent adjustments to the methodology for this Update, as discussed in chapter 3.

We now will be considering the other issues identified through this commentary, the feedback we have already received from multilateral organisations, and our own experience of undertaking a MAR and an Update.
The IDC found that our increased engagement with multilaterals during the Update process had been welcomed and asked us to continue this.

We intend to do so – such involvement is valuable to us, and we will seek the views of multilateral organisations, in particular, on both the methodology and the process for the MAR as we develop our thinking.

12.

We have already made a number of public commitments on what our approach will involve.
We will continue to assess agencies as now across the range of their performance.

The IDC asked us to consider including those agencies we stopped funding following the MAR in 2011.

We have said that, given the intensive nature of a MAR assessment, we would not want to subject an agency to this unless there was a prospect of renewing funding and that we therefore propose to undertake a light-touch review of these agencies to decide whether they will be included in a full MAR.
The NAO and PAC asked us to consider wider multilateral systems issues and we will be carrying out an exercise to map gaps and overlaps in a sector in the multilateral system during 2014.

We have already discussed, in chapter 3, the importance of improving the evidence base in particular the evidence about performance of multilaterals in country.

On the details of the methodology, we will be considering whether there are other issues that warrant attention.
The IDC have impressed on us the importance of gender and the need to more specifically recognise the specific issue of violence against girls and women.

We will also seek to respond to the work on defining the post-2015 development agenda: this will not be achieved without multilaterals and there may be ways it can and should be reflected in MAR assessments.

13.

We will not be working alone.


We recognise that our assessment causes a burden, and while we were pleased that the IDC judged that the effort caused by the MAR was proportionate to the benefits, we are not complacent.

We are working through a senior-level donor network to improve the availability of data on administration costs, through work to standardise the way in which multilaterals collect and report on it.
We are active members of MOPAN (the 91 DRIVING REFORM TO ACHIEVE MULTILATERAL EFFECTIVENESS tHe use of tHe Mar update Multilateral Organisation Performance Assessment Network) which is itself redefining its purpose, activities and methodology in time for 2015.

We are keen to see MOPAN become the main source of information and evidence on multilateral organisations, to support donor action and assessments.
We will also work to align our methodology to further reduce the burden on multilateral organisations.

Finally, we agreed with the PAC that we should work with other donors to share assessments and align methodology on a bilateral basis where we have common interests.

Conclusion 14.

This report marks the end of the MAR Update process and the first cycle of MAR assessments.
However, the discussion in this chapter on the continuing funding implications for multilateral organisations, our continuing expectations for and efforts on reform, and our plans for a further MAR in 2015 demonstrate that our work is ongoing.

Ensuring multilaterals are effective, through engagement and assessment, is an integral part of our work which continues between the more public examinations carried out in the MAR.

15.
We acknowledge the effort that has gone into the MAR Update and thank all those who have worked with the government on this: colleagues in DFID and across government, the external reviewers, who have scrutinised and challenged the approach and methodology and the conclusions reached; all of the NGOs who submitted evidence and shared their experience of multilaterals; and the multilateral organisations which have been assessed, and provided large amounts of evidence and engaged constructively with the process.
We are grateful.

92 annex 1 UK Multilateral Aid Review terms of reference 1.

Background 1.1 The UK’s Multilateral Aid Review (MAR) was published on 1 March 2011.

It provided, for the first time, a comprehensive overview of the strengths and weaknesses of the multilateral organisations that DFID works with.

1.2 The MAR confirmed that the multilateral system is a critical complement to what the UK government can do alone.

But it also found evidence of significant weaknesses.
Since the MAR was published DFID has drawn on its value for money assessment to decide on funding through multilateral organisations, communicated its key reform priorities to each multilateral organisation and engaged closely both with the institutions themselves and with other stakeholders to promote reform.

2.

Purpose 2.1 In order to ensure that contributions to multilateral organisations continue to offer the best value for money for UK aid, dfid needs to update the Mar assessments in 2013.

3.
3.

Scope 3.1 The Update will consider all of the multilateral organisations which were covered by the MAR and which continue to receive core aid DFID funding, or where DFID continues to be involved in key financing decisions.

3.2 The review will assess only the components relevant to the uK’s reform priorities, while also checking for backsliding elsewhere (e.g.

failure to implement agreed policies).

We will also be receptive to evidence of significant improvements in other MAR component areas.
The reform priority areas will vary depending on the MAR assessments and will be communicated to each multilateral organisation in writing.

3.3 The Update will focus on the extent to which the UK’s reform priorities have been taken forward since the MAR was carried out.

Progress on the reform priority components will be given a narrative assessment and scored.
Where reforms have been sufficient to justify a change in component score, this will be given and used to update the overall MAR assessment of value for money as appropriate.

93 3.4 The updated assessments will inform dfid decisions on funding through multilateral organisations, although the review will not itself make funding recommendations.

All DFID funding through multilateral organisations is related to performance.
The modality for this varies, and will be made clear to each organisation in advance of the Update.

4.

Methodology 4.1 Given the partial nature of the Update, it would not be appropriate to make significant changes in the methodology at this stage.

The updated assessments will therefore be based on the original Mar methodology.

However, there will be an even stronger emphasis on ensuring high quality country-level evidence for the assessments.
4.2 We have worked closely with the normative agencies to develop a clarified framework for these assessments.

The UK National Audit Office (NAO) published its Value for Money Study of the MAR on 19 September 2012.

DFID will respond, as appropriate, to their recommendations.

4.3 As with the MAR, external reviewers will provide a quality assurance function to ensure the robustness of the exercise.

5.
5.

Timing 5.1 Updated assessments for each agency will be carried out and published in line with dfid business needs over 2013, linked to the DFID business case cycle and to key decision points over financing.

The timings will be communicated to each multilateral organisation in writing.

5.2 In order to ensure consistency of treatment across organisations, the assessments will be considered in three batches, completed in the spring, summer and autumn of 2013.
Multilateral organisations will be informed in writing of the timing of their assessment.

6.

Stakeholder engagement 6.1 DFID is already engaged in dialogue with each multilateral organisation over the MAR findings and their implications for the reform agenda.

This engagement will continue.

It will include clearly specifying the improvements that DFID will look to see in the updated assessments, and the links between progress on reform and future funding.
It will also include dialogue with the institutions over the evidence base for the updated assessments.

6.2 DFID will monitor progress against the key reform priority areas at country level.

Monitoring reports will draw on inputs from other stakeholders, including government, civil society, the private sector and other donors as appropriate, and will be shared with the multilateral organisations.
DFID will conduct regular monitoring in many of the countries where it is present, and will carry out periodic visits to a sample of other countries.

In some cases, DFID will carry out 94 UK MULTILATERAL AID REVIEW TERMS OF REFERENCE monitoring and visits jointly with other donors.

DFID will inform multilateral organisations of the countries where they will be monitored and reviewed.
6.3 UK civil society has been invited to contribute to the exercise, and is considering how best to draw on their networks in-country to provide evidence for the updated assessments.

6.4 The views of developing country governments are important to DFID.

A particular effort will be made to seek these views in advance of the updates.
6.5 As with the original MAR, the updated assessments will draw on a wide range of other evidence sources, including Mopan assessments (Multilateral Organisation Performance Assessment Network).

6.6 Other UK government departments, including the Foreign and Commonwealth Office and Her Majesty’s Treasury, will be involved in updating the assessments as appropriate, and consulted about possible exits or proposals for significant changes in funding.
7. Review team and governance 7.1 The updated assessments will draw on contributions from across the UK government as well as a wide range of other stakeholders.
These will include: ■■ DFID institutional teams, who will be responsible for pulling together the evidence for their institutions, proposing any changes to scores, developing the narrative on progress on reform, and leading discussions with other UK government departments, as well as the engagement with their multilateral institutions ■■ the International Directors’ Office (IDO), who will co-ordinate the evidence- gathering process, advise on methodology issues, ensure that the assessments are consistent, and write the final report ■■ Missions, delegations and permanent representations will contribute to these judgements as well as playing a major role in stakeholder management ■■ DFID country offices, who will engage in regular monitoring of performance at country level ■■ DFID policy teams, who will advise on the performance of the multilateral organisations in relation to thematic or sectoral objectives ■■ Other UK government departments, who will give evidence of progress as appropriate 95 DRIVING REFORM TO ACHIEVE MULTILATERAL EFFECTIVENESS 7.2 The work will be overseen by a steering group which comprises the directors of the International Divisions, Policy Division, and two Regional Divisions, as well as the Chief Economist.
The steering group is chaired by Mark Bowman, Director General.

7.3 external reviewers will contribute to the development of the methodology for updating the assessments, and help to ensure that the updates themselves are sufficiently robust.

8.

Reporting 8.1 Updated assessments will be published in two batches in the summer of 2013 and the autumn of 2013.
8.2 Once all the assessments are completed, DFID will publish a report which reviews progress against the Mar reform priorities, highlighting both successes and areas where more effort is needed.

24 October 2012 96 UK MULTILATERAL AID REVIEW TERMS OF REFERENCE annex 2 Assessment frameworks summary of framework Mar component criteria 1.

Critical role in meeting development objectives 1.1 Is the MO critical in the delivery of the MDGs and poverty reduction?
1.2 Is it critical in the delivery of other international development goals (for example economic growth, adapting to climate change, conflict and humanitarian objectives)?

1.3 Does it fill a critical gap in the international development and humanitarian architecture and at a regional and country-level (e.g.

shocks, disease, climate change, cross-border infrastructure), meeting gaps in knowledge, making the system more effective e.g.
through providing a common platform for other donors, or innovating to create more effective instruments.

This implies that the organisation plays a leading role in this respect.

The MOs should be assessed on how critical they are to: ■ Key DFID development and humanitarian priorities as set out in more detail below.

■ Development or humanitarian objectives in countries/regions that are important to the UK.

■ UK government development objectives more broadly (for example prosperity in the Caribbean).
DFID Priorities: ■ Wealth creation – generate growth, stimulate the private sector and trade and improve infrastructure.

■ Governance and security – peace building, conflict prevention, public financial management, human rights, as well as stabilisation spending and the Global Conflict Pool.

■ Direct delivery of the MDGs (particularly for girls and women) – health and nutrition, education, water and sanitation, humanitarian assistance and food aid.
■ Climate change – mitigate and adapt to the impact of climate change.

■ Respond to humanitarian disasters – ensure basic needs of those affected are met 97 summary of framework: continued Mar component criteria 2.

Attention to cross-cutting issues 2.1 Is there clear evidence that the MO performs effectively in fragile states?

2.2 Does the MO have specific policy and/or operational guidance on working in and on these contexts (e.g.
around use of Fragile States principles, conflict sensitivity/Do No Harm, including social safeguards, political/social/conflict analysis)?

Is this guidance of good quality?

Is this guidance mainstreamed and used?

2.3 Are agency personnel equipped to work in contexts of conflict and fragility (e.g.
by clear mandates and guiding principles of engagement; adequate staffing at country level) 2.4 Does the MO produce annual or more frequent monitoring reports which include specific attention to operating in fragile contexts?

Do such reports inform policy and programming?

2.5 Does it have policies, structures and incentives to promote gender equality and is there evidence of these having an impact?
2.6 Does it have and use partnerships to promote gender equality and is there evidence of these having an impact?

2.7 Does evidence and information on gender equality inform policy and programming and is there evidence of these having an impact and improving policy choice?

2.8 Is there country-level evidence of the mainstreaming of gender or of the impact of gender policies 2.12 Does the MO have environmental and climate safeguards in place?

Do these meet our baseline standard?


Are all development or humanitarian interventions guided by the outcomes of the environmental/climate safeguard procedure?

Are these monitored and reviewed?

2.13 Are climate change, development and environmental impacts measured?

Are climate change and environment indicators incorporated into all performance/results frameworks?

98 ASSESSMENT FRAMEWORKS summary of framework: continued Mar component criteria 3.
Focus on poor countries We will use indices of need (numbers of poor people, human development indicators and fragility) and effectiveness (strength of institutional and policy environment) to construct a country ranking for all low income and lower middle income countries.

The higher up the ranking, the more likely it is that aid to that country will contribute to the UK’s poverty reduction objectives.
We will look at how the multilateral development organisations allocate their country based aid, and compare this with our ranking.

Organisations that give a large share of their aid to countries high up in the ranking are, in purely geographic terms, more likely to be contributing to poverty reduction.

We will describe them as having a good focus on poor countries (FoPC).

The FoPC scores will be banded together to generate scores for focus on poor countries on a range of 1 to 4.
Multilateral organisations whose principal focus is not development and which focus on humanitarian responses and conflict, climate change and disaster risk reduction allocate their resources against different objectives.

In each of these cases we will qualitatively assess the MOs against their own objectives by looking at how well their allocation fits the incidence of the problem.

This will provide the relevant measure of need overall.
Where possible, effectiveness is will be addressed through consideration of how well allocation is driven by evidence of country and/or situation specific contexts.

In all cases, a judgement will be made on the balance of evidence on need and effectiveness to determine the FoPC scores.

4.

Contribution to results 4.1 Are its objectives sufficiently challenging?

For example, is the MO striving for continuous improvement and striving to reach the very poorest groups?
Is it benchmarking itself against similar organisations?

Is it taking risks and innovating to deliver better results?

4.2 Is there evidence that the management is doing all that it can to deliver results at country level (striving for results, holding staff to account for delivery, pro-actively intervening to turn around problem parts of the portfolio etc)?

4.3 Can it demonstrate delivery against its objectives including at a country-level?
4.4 Can it demonstrate a significant contribution to development (outputs or outcomes), humanitarian results or poverty reduction?

99 DRIVING REFORM TO ACHIEVE MULTILATERAL EFFECTIVENESS summary of framework: continued Mar component criteria 5.

Strategic and performance management 5.1 Does it have a clear mandate?

Is there a clear line of sight from the mandate to strategy and implementation plans?

5.2 Is its governing body effective at holding management to account?
Does it use results and evaluation evidence to challenge management and effectively steward performance and strategic decision-making?

5.3 How effective is its leadership?

Is it using results and evaluation evidence to drive improvements in performance and strategic decision-making?

5.4 Do its HR policies encourage good performance?

Does it award jobs transparently and on the basis of merit and experience?

5.5 Are systems in place to effectively measure results?
Does it have a comprehensive results framework that covers the whole of the organisation’s activities and the whole of the results chain from inputs through to impact?

5.6 Does it have an effective evaluation function?

Are evaluations acted upon?

6.

Financial resource management 6.1 Does it use a clear and transparent system to allocate aid?

6.2 Do its financial systems allow and encourage the making of predictable i.e.

long term commitments?

6.3 Are aid flows released according to agreed schedules?
6.4 Does it pro-actively manage poorly performing projects and programmes, curtailing them where necessary and recycling savings into better performing parts of the portfolio?

6.5 Does it have strong policies and processes for financial accountability (risk management, anti-corruption, quality of external audits, fiduciary risk)?

6.6 Do its financial systems give it the flexibility to use the right instruments in the right situations (e.g.

in fragile states)?
in fragile states)?

100 ASSESSMENT FRAMEWORKS summary of framework: continued Mar component criteria 7.

Cost and value consciousness 7.1 Does it challenge and support development partners to think about economy, efficiency and cost effectiveness in key policy and programme choices?

8.
8.

Partnership behaviour 7.2 Do its systems (including pressure from the governing bodies and shareholders) require senior management to take account of return and cost/effectiveness (is there evidence that shareholders actively challenge senior management on such issues or question choice)?
7.3 Is it aware of and does it strive for economy in the purchase of programme inputs (in other words, its approach to procurement is driven by cost control, it has targets for procurement savings, prices achieved are monitored and reported on)?

7.4 Does it strive for reductions in administrative costs and is not profligate (is there evidence of targets and cost control)?
8.1 Does it work effectively in partnership with others (includes partner countries, other multilateral organisations, bilaterals, NGOs, civil society, research institutions, private sector etc)?

The scoring should give greater weight to those partnerships which are most important to the MOs role and mandate.

8.2 Does it implement social safeguard policies, including incorporating beneficiary voice into its policies and programmes (e.g.
through participatory approaches to programme design and implementation)?

In particular do policies promote the participation of girls and women, and the most marginalised, including indigenous peoples and people with disabilities?

8.3 Does it have the flexibility to enable and reinforce the country- led approach (e.g.

is it flexible in the policy choices it can support, does it use instruments clients want, does it apply low and appropriate conditionality etc)?
8.4 Does it provide aid and technical assistance in a way most likely to lead to sustained development results (e.g.

Paris/Accra type approaches) and does it take a leadership role on this agenda.

8.5 Does it provide an effective leadership and co-ordination role in humanitarian settings (if applicable)?

101 DRIVING REFORM TO ACHIEVE MULTILATERAL EFFECTIVENESS summary of framework: continued Mar component criteria 9.
Transparency and accountability 9.1 Does it have a disclosure policy, and does that policy specify a presumption of disclosure – i.e.

that information should be made publicly available unless there is a clear case for withholding it?

Is the list of exceptions justifiable and based on the MO’s commercial, security, data protection or other policies and regulations?

9.2 Does it encourage transparency and accountability in delivery partners and recipients, by putting its aid on budget?
9.3 Does it routinely publish project and policy documentation and are these easy to find?

Does it publish timely, detailed data about projects it is funding or implementing?

9.4 Is the multilateral signed up to the International Aid Transparency Initiative (IATI) and is it actively participating?

Has it published a plan to implement phase 1 IATI/the new common standard by 2015?

9.5 Are partner countries well represented (e.g.


through seats, votes etc) in the governing mechanisms of the MO, such that they have an impact on decision making.

9.6 Do stakeholders (government, civil society, other key groups) from partner countries have a mechanism through which they have the right to redress or complain about the MOs policies and programmes?

For humanitarian MOs: 9.7 Is the MO certified by the Humanitarian Accountability Partnership?
Has it undertaken any other humanitarian transparency or accountability project (e.g.

peer review on accountability to disaster-affected populations)?

9.8 Does the MO have systems and tools to ensure adequate participation of disaster/conflict affected groups in needs assessments, monitoring and evaluation?

102 ASSESSMENT FRAMEWORKS Clarified assessment framework for Multilateral Aid Review assessment summary of framework: Mar component criteria 1.
Critical role in meeting development objectives 1.1 Does the MO play a critical role in global governance structures for sustainable development and/or addressing the MDGs or other international development or humanitarian goals?

1.2 Does it develop norms and standards or global public goods that are critical for the achievement of the MDGs and poverty reduction, or for the delivery of other international development or humanitarian goals?
1.3 Does it provide a convening function for international bodies and develop norms and standards which are perceived to be important by member states, including developing countries as demonstrated by take- up (ratification/incorporation into national regulatory framework, guidelines, priorities or other country-led processes)?
1.4 Does the MO play a critical role (in country and internationally) in supporting and/or challenging countries to implement norms and standards in the area of its mandate that are important for development?

1.5 Does it generate evidence and formulate policy advice that is world-leading in terms of originality, significance and rigour, and is widely available and used by policy-makers, including in development agencies and/or developing countries?
1.6 Is it a world leader in identifying, understanding, developing and brokering solutions to global, regional or cross-border threats in the area of its mandate?

Does it understand the relationships between these threats?

1.7 At the country level, is the organisation critical in creating an enabling environment for the delivery of the MDGs and poverty reduction or other international development and humanitarian goals?
1.8 To the extent that the organisation has in-country activities that are not related to its normative work, are these critical in the delivery of the MDGs and poverty reduction, or in the delivery of other international development and humanitarian goals?

Does it fill a critical gap in the international development and humanitarian architecture at regional and country level?

1.9 [where relevant] Does the MO provide critically important advice and assistance in humanitarian situations?
103 DRIVING REFORM TO ACHIEVE MULTILATERAL EFFECTIVENESS summary of framework: continued Mar component criteria 2.

Attention to cross-cutting issues 2.1 Does the MO take full account of gender, climate and environmental dimensions in its normative work, evidence and policy products?

2.2 Does the MO take account of gender and climate and environmental dimensions when deciding which areas to prioritise in normative/standard-setting work?
2.3 Are its norms, standards, and delivery of pathways for their implementation relevant to fragile contexts and enhance resilience to crises and threats?

2.4 Does the MO tailor its policy products and assistance to ensure relevance in situations of conflict or fragility?
2.5 To the extent that the organisation has in-country activities that are not related to its normative work, does it pay adequate attention to gender, climate change and environmental sustainability considerations and the special circumstances of fragile contexts, as set out in the main MAR framework?

3.

Focus on poor countries 3.1 Does the MO utilise a global approach where appropriate but have an operational focus on the needs of the most vulnerable countries and population groups?
Is the global approach sensitive to the needs of poor countries through an attention to voice and capacity development in these countries?

4.

Contribution to results 3.2 Does it prioritise resources available for in-country support to those developing countries where there is most need for its services?

4.1 Does the MO articulate the theory of change that translates normative and standard-setting work into impact on the ground?
Can it demonstrate, through its monitoring and evaluation systems that this impact is taking place and that milestone/progress markers deemed necessary to achieve this impact are being achieved?

4.2 Are normative and standard-setting products relevant, have the backing of key stakeholders and ambitious enough to demand significant improvements in practice?
4.3 Are country and regional level implementation activities related to norms and standards work designed and managed with a focus on achieving results?

4.4 Is there evidence of global knowledge products, policy advice or thought leadership leading to tangible change in policy or practice?

4.5 Does the MO demonstrate results in addressing identified regional and global issues?

104 ASSESSMENT FRAMEWORKS summary of framework: continued Mar component criteria 5.
Strategic and performance management 4.6 To the extent that the organisation has in-country activities that are not related to its normative work, is there clear evidence that these are contributing to results at country level?

5.1 Does it have a clear mandate?

Is there a clear line of sight from mandate to strategy and implementation plans?

5.2 Does the MO develop norms and standards which are strategically aligned with its mandate?

5.3 Is its governing body effective at holding management to account?
Does it use results and evaluation evidence to challenge management and effectively steward performance and strategic decision-making?

5.4 How effective is its leadership?

Is it using results and evaluation evidence to drive improvements in performance and strategic decision-making?

5.5 Do its HR policies encourage good performance?

Does it award jobs transparently and on the basis of merit and experience?

5.6 Are systems in place to effectively measure results?
Does it have a comprehensive results framework that covers the whole of the organisation’s activities and the whole of the results chain from inputs through to impact?

5.7 Does it have an effective evaluation function?

Are evaluations acted upon?

5.8 Do the norms and standards developed by the organisation complement and reinforce each other and are they linked up where appropriate with other organisations’ work?
5.9 Does the MO have a clear dissemination strategy which actively promotes the use of its evidence and policy guidelines and does it monitor this?

6.

Financial resource management 6.1 Does it use a clear and transparent system to allocate aid?

6.2 Does it pro-actively manage poorly performing projects and programmes, curtailing them where necessary and recycling savings into better performing parts of the portfolio?
6.3 Does it have strong policies and processes for financial accountability (risk management, anti-corruption, quality of external audits, fiduciary risk assessment)?

6.4 [where relevant] To the extent that it disburses aid, does it deliver predictable financing by, for example, making long-term commitments and disbursing aid according to agreed schedules?

105 DRIVING REFORM TO ACHIEVE MULTILATERAL EFFECTIVENESS summary of framework: continued Mar component criteria 7.
Cost and value consciousness 7.1 Does it encourage partners to think about economy, efficiency and cost effectiveness in all that it does, including in normative and policy work and global public goods?

8.

Partnership behaviour 7.2 Do its systems (including pressure from the governing bodies and members) require senior management to take account of return and cost effectiveness (is there evidence that shareholders actively challenge senior management on such issues or question choice)?
7.3 Is it aware of and does it strive for economy in the purchase of programme inputs (in other words, is its approach to procurement driven by cost control, does it have targets for procurement savings, are prices achieved monitored and reported on)?

7.4 Does it strive for reductions in administrative costs and is not profligate (there is evidence of targets and cost control)?
8.1 Does it work effectively in partnership with others (includes partner countries, other multilateral organisations, bilaterals, NGOs, civil society, research institutions, private sector etc)?

The scoring should give greater weight to those partnerships which are most important to the MOs role and mandate.

8.2 Is the organisation’s policy agenda broadly endorsed by its membership and put into practice in a way that ensures widespread take- up of its products?
8.3 Does the MO incorporate beneficiary voice, particularly of girls and women and marginalised groups, into its policies and programmes, including in developing norms and standards, policy advice and guidelines and global public good products?

8.4 Does it adapt norms, standards and global public good products appropriately to different country and regional contexts?

8.5 Does the MO provide aid and technical assistance in a way most likely to lead to sustained development results (e.g.
Paris/Accra type approaches) and does it take a leadership role on this agenda?

8.6 [where applicable] Does the MO effectively fulfil its role as a humanitarian cluster lead?

106 ASSESSMENT FRAMEWORKS summary of framework: continued Mar component criteria 9.

Transparency and accountability 9.1 Does it have a disclosure policy, and does that policy specify a presumption of disclosure – i.e.

that information should be made publicly available unless there is a clear case for withholding it?
Is the list of exceptions justifiable and based on the MO’s commercial, security, data protection or other policies and regulations?

9.2 Does it routinely publish project and policy documentation (including the latest versions of all normative instruments, and reports on implementation of norms) including financial data and performance reports and are these timely and easy to find?

9.3 Is the multilateral signed up to the International Aid Transparency Initiative (IATI) and is it actively participating?
Has it published a plan to implement phase 1 IATI/the new common standard by 2015?

9.4 Are partner countries well represented in the governing mechanisms of the organisation – and of any international treaty bodies it hosts – and do they have an impact on decision making?

9.5 Do stakeholders (government, civil society, other key groups) from partner countries have a mechanism through which they have the right to redress or complain about the MOs policies and programmes?
9.6 Are norms and standards developed and monitored in a transparent and unbiased way, including through clearly distinguishing between independent experts and government representatives?

9.7 Does the MO have systems and tools to ensure adequate participation of disaster/conflict-affected groups in needs assessments, monitoring and evaluation?
107 DRIVING REFORM TO ACHIEVE MULTILATERAL EFFECTIVENESS annex 3 MAR Update summary assessments component rating description 4 significant progress: Reforms achieved to date surpass expectations by strongly addressing reform priorities significant progress 3 reasonable progress: Reforms achieved to date meet expectations by satisfactorily addressing reform priorities reasonable progress 2 some progress: Reforms achieved to date weakly address reform priorities and do not fully meet expectations some progress 1 little or no progress: Reforms achieved to date unsatisfactorily address reform priorities and any progress is significantly below expectations little or no progress overall rating Basis for rating 4 3 2 1 significant progress in all components OR significant progress in most components outweighs reasonable, some or no progress in others.
reasonable progress in all components OR reasonable progress in most components outweighs some and no progress in others OR significant progress in some components balances some and no progress in others some progress in all or most components OR reasonable progress in some components balances no progress in others little or no progress in most components not balanced by significant, reasonable or some progress elsewhere 108 african development fund (afdf) Multilateral aid review (Mar) 2013 update progress rating: 2 MAR 2011: Good Value for Money for UK Aid progress assessment summary reasonable progress in building partnerships, on results, and on tackling climate change and environmental sustainability.
some progress on gender issues, enhancing effectiveness in fragile contexts and cost and value consciousness.

Baseline African Development Fund, as part of the African Development Bank, supports poorer African countries in infrastructure, governance, regional integration, tertiary and vocational education, and agriculture and food security.

The MAR highlighted several strengths: • • • • AfDF’s geographical focus fits well with DFID’s priorities.

It has a strong focus on wealth creation and governance.
It has generally good relationships with partner governments.

It has an independent evaluation department helping to shape policy.

The MAR also highlighted several weaknesses: • • Delays and limited in-country capacity hinder performance.

It is not yet able to demonstrate outputs for all its programmes and projects and it is not always strongly focused on poverty.

A need to improve the mix and specialisation of skills of staff in fragile contexts.
There is weak performance on climate change and environmental sustainability, fragile contexts and gender.
• • DFID’s reform priorities for the MAR Update were: • Improved focus on gender (particularly on results), enhanced effectiveness of programmes in fragile contexts and better defined policies on climate change and environmental sustainability – assessed under attention to cross-cutting issues (gender, fragile contexts and climate change and environmental sustainability); Embedded results framework in the Bank’s business and its culture focused on results – assessed under contribution to results; Improved effectiveness in administration budgets and value for money in programmes – assessed under cost and value consciousness; Improved project performance and partner coordination through further decentralisation – assessed under partnership behaviour.
• • • summary of overall progress Improvements that have been demonstrated in all reform components since the MAR are: Climate change and environmental sustainability action plan finalised with more clearly defined objectives; staff presence in fragile contexts increased with some evidence of performance improving in developing countries; improved systems and frameworks in place to allow the AfDB to better measure AfDF’s contribution to results; improved cost-effectiveness of project management and swifter project delivery; and maintenance of strong partnership with its clients and improved standing and visibility with other donors.
In three areas progress has been less than expected: a delayed new gender policy; limited progress in responding to the evaluation on fragile contexts; and limited dialogue on unit costs of programmes.
109 DRIVING REFORM TO ACHIEVE MULTILATERAL EFFECTIVENESS Mar reform component progress against reform priorities Mar 2011 score progress rating Mar update score, if any change attention to cross-cutting issues (gender) There is some evidence that gender is being considered in programme planning and results are being disaggregated by gender.

A revised gender framework and action plan has been delayed by over two years.
attention to cross-cutting issues (fragile contexts) The decentralisation road map is mostly on-track and greater delegated authority to the field but there is slow progress in responding to some recommendations of fragile contexts evaluation.

attention to cross-cutting issues (climate change and environmental sustainability) The climate change and environmental sustainability action plan has been finalised and has more clearly defined objectives.
There is mixed evidence on how much impact they are having in developing countries, though it is still early days.

contribution to results The most notable achievement has been the annual reporting of the Bank’s results framework with thematic and country specific reports.

cost and value consciousness There have been improvements in the costs of transactions and speed of delivery of AfDF projects – notably in fragile contexts.
However, there is limited progress in improving the dialogue with clients on lower cost options in policy and programme choices.

partnership behaviour There is evidence of improvements in co-ordination with donors.

There has also been good performance against the Paris Declaration targets.

There is an agreed framework for engaging with civil society organisations.
2 2 2 2 3 3 Some progress Some progress Reasonable progress Reasonable progress Some progress Reasonable progress 110 MAR UPDATE SUMMARY ASSESSMENTS asian development fund (asdf) Multilateral aid review (Mar) 2013 update progress rating: 3 MAR 2011: Very Good Value for Money for UK Aid progress assessment summary reasonable progress made with respect to cross-cutting issues of gender, climate change and environmental sustainability and strategic and performance management.
some progress made on results and cost and value consciousness.

Baseline The Asian Development Fund (AsDF) is the part of the Asian Development Bank which lends at very low interest rates and provides grants to the less developed regional members of the Bank to promote inclusive and environmentally sustainable growth.

The MAR highlighted several strengths: • • The AsDF plays a critical role in sustainable economic development across Asia.
There is a strong results focus and good integration of climate change and environmental sustainability into existing development work.

There is good financial management, and relatively low development costs for a development bank.

• The MAR also highlighted several weaknesses: • • • The AsDF has a good policy on gender equality but limited evidence of impact.

There is no evidence of emphasis on securing cost effectiveness in projects.
There is only partial progress in tackling weaknesses in HR policies and practices.
DFID’s reform priorities for the MAR Update were: • Address the needs of girls and women through the design and implementation of projects – assessed under attention to cross-cutting issues (gender); Tackle climate change and environmental sustainability, including leveraging private sector finance for projects in low income countries – assessed under attention to cross-cutting issues (climate change and environmental sustainability); Greater focus on inclusive growth in country strategies and projects – assessed under contribution to results; Effective decentralisation of staff (and devolution of decision making) to support policy dialogue with partner governments – assessed under strategic and performance management; Support to partners to achieve value for money and striving to reduce administration costs – assessed under cost and value consciousness.
• • • • summary of overall progress The Bank has demonstrated progress on all criteria but some weaknesses remain at project level implementation and reporting.

The new results framework has a stronger results focus including demonstrating evidence of inclusive growth, and a coherent strategy for private sector development, and improved business processes and reporting on portfolio performance.
The new results framework includes value for money indicators, but more substantial evidence on cost and value consciousness at project level is needed to warrant a higher score.

Reasonable progress has been achieved in its human resource management, especially on decentralisation to country missions and improved capacity.
The Bank has also raised the gender profile across the programming and agreed more ambitious gender targets in its results framework, but it is too early to see changes in developing countries.

However, there is more systematic evidence of an enhanced contribution to tackling climate change and environmental sustainability, including a substantial rise in private sector leveraging in climate change and environmental sustainability mitigation.
111 DRIVING REFORM TO ACHIEVE MULTILATERAL EFFECTIVENESS Mar reform component progress against reform priorities Mar 2011 score progress rating Mar update score, if any change attention to cross-cutting issues (gender) There is a stronger gender focus in the revised results framework and country strategies and improved gender monitoring.
attention to cross-cutting issues (climate change and environmental sustainability) There is strengthened staff capacity on climate change and environmental sustainability, and increased private sector leveraging in climate change and environmental sustainability mitigation.

contribution to results Revised results framework focuses more on inclusive growth and enabling private sector development.
There is a stronger results focus at project, programme and country levels, but efforts to redress falling project performance have yet to bear fruit.

strategic and performance management There has been improved capacity in developing countries through further devolution and some progress in policy dialogue, but not yet demonstrated expected improvements in project performance.

There is more transparency in recruitment of senior staff.
cost and value consciousness There has been progress on business processes, including procurement capacity and reporting on portfolio performance but limited evidence of efficiency gains in programmes.
2 3 3 4 3 Reasonable progress Reasonable progress Some progress Reasonable progress Some progress 112 MAR UPDATE SUMMARY ASSESSMENTS the caribbean development Bank (cdB) Multilateral aid review (Mar) 2013 update progress rating: 2 MAR 2011: Adequate Value for Money for UK Aid progress assessment summary some progress in addressing most reform priorities identified in the Mar.

slow implementation and too early to demonstrate evidence in projects and in developing countries.
cost value consciousness remains the most challenging reform priority to be addressed.

The Caribbean Development Bank provides development finance across the Caribbean including Commonwealth countries and UK Overseas Territories.

Baseline The MAR highlighted several strengths: • • There is an increasing focus on results.

There are strong partnerships with governments, ability to work with other donors and willingness to accept expertise from others (IBRD, IADB).
There is good control of administrative budgets and the process for senior management recruitment is transparent and merit based.

• The MAR also highlighted several weaknesses: • • • • Weaknesses in HR policies and practices exist.

Strategies and results frameworks need to be improved.

There is limited expertise in cross-cutting areas with limited impact on operations.

The large number of small projects makes reporting of impact difficult.
DFID’s reform priorities for the MAR Update were: • • • • Delivering for women, girls and disadvantaged males, especially youth through mainstreaming gender in operations – assessed under attention to cross-cutting issues (gender); Strengthening project performance and demonstrating results on the ground – assessed under contribution to results; Focusing on value for money in bank administration and programme choices – assessed under cost and value consciousness; Provisioning public access to policies and project information to support accountability and introducing a disclosure policy based on open access – assessed under transparency and accountability.
summary of overall progress Since the MAR, the Bank has demonstrated some progress in its management for results, including the revision of its results framework.

It has started work to improve its operational policies, strategies and guidelines and its corporate oversight (e.g.

in risk management and internal audits).


In addressing its long term issue of high vacancies which has impacted the Bank’s capacity to implement reforms, the CDB implemented a major review of staff skills and structures and has filled 30 key posts (representing 15 per cent increase in staffing) which is a positive step forward.
It has also improved its transparency through a new information disclosure policy, greater independence for the evaluation function and by some strengthening of project and policy content on its website.

However, progress has been slow in implementing the reform programme across most of the key reform areas.

Further work is needed for the Bank to demonstrate real improvements in the areas of results management, value for money and gender mainstreaming.
Implementation of the agreed reforms needs to be accelerated.

113 DRIVING REFORM TO ACHIEVE MULTILATERAL EFFECTIVENESS Mar reform component progress against reform priorities Mar 2011 score progress rating Mar update score, if any change attention to cross-cutting issues (gender) There has been improved capacity and some sex disaggregated data in the new results framework, but too early to demonstrate any changes on the ground in gender emphasis or outputs.
contribution to results The Bank has started to report on results in the annual Development Effectiveness Review.

There is evidence that the Bank is moving towards more results-based management.

It is too early to see changes at project and programme levels.

cost and value consciousness There is good financial control, but there is still limited evidence on efficiency gains and value for money considerations in its operations.
transparency and accountability New information disclosure and corporate communications policies exist.

There are initial steps towards sharing more project and country data on the Bank’s public website.

2 2 2 2 Some progress Some progress Little or no progress Some progress 114 MAR UPDATE SUMMARY ASSESSMENTS central emergency response fund (cerf) Multilateral aid review (Mar) 2013 update progress rating: 2 MAR 2011: Good Value for Money for UK Aid progress assessment summary Has taken reforms seriously.
improved results reporting, and has continued to be open and transparent.

impact level reporting and improving disbursement schedules remain a challenge.

committed to addressing outstanding areas of concern around value for money.

The CERF is a UN humanitarian fund which enables timely humanitarian assistance to reach those affected by natural disasters and protracted conflict.
Baseline The MAR highlighted several strengths: • • • CERF fills a crucial gap by pre-positioning donor funding to ensure a timely response.

It has a clear mandate, leadership is strong and evaluations are acted upon.

It is well administrated and its management has improved the transparency and timeliness of the aid allocation process.

It meets UK objectives and enables DFID to deliver on humanitarian objectives.
• The MAR also highlighted several weaknesses: • • There is no systematic reporting on results at the beneficiary level.

There are no accountability safeguards beyond those provided by the individual recipient agencies.

Lack of timeliness in projects implemented through NGOs.

There is an additional 3% management charge at secretariat level.
• • DFID’s reform priorities for the MAR Update were: • Improving: (i) the prioritisation process for CERF applications; (ii) the performance management framework; and (iii) results reporting – assessed under strategic and performance management; Improving the value for money of the CERF management charge and ensuring timely disbursement of funds to NGOs and implementers – assessed under cost and value consciousness; Providing additional accountability safeguards – assessed under transparency and accountability.
• • summary of overall progress There is a commitment to investigate reforms and take action where appropriate.

Reasonable progress has been made against all the reform priorities.

It is too early to assess the impact in the field.

Achievements to date include: • An increased focus on prioritisation.

Prioritisation strategies are required before CERF proposals are developed.

At headquarters, the CERF examined its approach to underfunded emergencies to ensure priorities are met.
Improved results reporting.

The Performance and Accountability Framework (PAF) leads to better results reporting at outcome, output and input level.

Capturing the impact of CERF funding remains a challenge.

Continued focus on the accountability of implementing partners.

The review of the PAF later this year provides an opportunity to address this further.

• • Challenges remain: • • Reducing fund disbursement times from UN agencies to implementing partners.
An examination of management and administrative charges.

The value for money of the management charge remains a concern and action to address this should be prioritised.

115 DRIVING REFORM TO ACHIEVE MULTILATERAL EFFECTIVENESS Mar reform component progress against reform priorities Mar 2011 score progress rating Mar update score, if any change strategic and performance management CERF continues to have a clear mandate and strong leadership which uses results and evaluation to improve performance.
It has an improved results framework, and effective evaluation function.

Challenges remain in capturing impact of funds.

cost and value consciousness Partners continue to be challenged on cost issues.

Overhead rates have been examined.

Concerns remain that the CERF has not yet been able to ensure that the 3% running costs for the CERF secretariat represent value for money.

transparency and accountability CERF remains open and transparent.

Accountability is improving at headquarters.


Initiatives are underway to improve accountability to partners and beneficiaries.
2 2 2 Reasonable progress Little or no progress Reasonable progress 116 MAR UPDATE SUMMARY ASSESSMENTS • • • climate investment funds (cifs) Multilateral aid review (Mar) update 2013 progress rating: 3 MAR 2011: Good Value for Money for UK Aid progress assessment summary strong commitment at headquarters to reform and an increased focus on results, but too early to see the full benefits in developing countries.
Baseline The CIFs are made up of four programmes implemented by the multilateral development banks: the Clean Technology Fund (CTF); the Pilot Programme for Climate Resilience (PPCR); the Forest Investment Programme (FIP) and the Scaling Up Renewable Energy Programme (SREP).

The MAR highlighted several strengths: • Meets a critical gap in delivering climate change outcomes, offers finance at scale and innovation, and informs future climate change architecture.
Flexibility to use a variety of financing instruments with some innovation, strong audit function.

Low administrative costs and a challenge function for finance and budgetary issues.

• • The MAR also highlighted several weaknesses: • Patchy experience with mixed evidence on working with developing country stakeholders other than governments.

Lengthy process to design results frameworks.

No common approach or agreed methodology to decide the allocation of funds between pilots in the four programmes.
DFID’s reform priorities for the MAR Update were: • Develop policies, structures and incentives to promote gender – assessed under attention to cross-cutting issues (gender); Ensure systems are in place to effectively measure results, including development impact – assessed under strategic and performance management; • Work effectively in partnership with others, including ensuring country ownership – assessed under partnership behaviour; Improve the transparency of the CIFs – assessed under transparency and accountability.
• summary of overall progress The CIFs continue to make progress across all reform priorities.

Concerns about how gender was taken into account were targeted through an independent gender review.

Revised results frameworks were implemented for three of the programmes, whilst work continues on core indicators for the fourth.

Concerns around speed of disbursement are being addressed.

Greater transparency has been achieved through signing up to the International Aid Transparency Initiative.
117 DRIVING REFORM TO ACHIEVE MULTILATERAL EFFECTIVENESS Mar reform component progress against reform priorities Mar 2011 score progress rating Mar update score, if any change attention to cross-cutting issues (gender) An independent gender review of the CIFs was conducted and recommendations are being taken forward including the recruitment of a senior gender specialist within the CIF administrative unit.
Multilateral development banks are ensuring that gender experts are included in all missions and that at least one indicator disaggregated by gender is included in each project where feasible.

strategic and performance management Revised results frameworks, toolkits and guidance for monitoring and reporting have been developed.

An independent evaluation is now fully underway.

The CIFs have used performance evidence to improve their slow disbursement rates.
partnership behaviour Actions have been taken to improve MDB collaboration in developing countries and strengthen country ownership.

Since 2011 there is more evidence that CIFs are working with stakeholders, including the private sector and civil society, as they move from the design to implementation phase.
transparency and accountability Reasonable progress has been made in enhancing the transparency of the CIFs, including making executive sessions open to all, improvements made to the clarity and content of annual reports, and signing up to the International Aid Transparency Initiative.
2 3 2 3 Some progress Reasonable progress Reasonable progress Reasonable progress 118 MAR UPDATE SUMMARY ASSESSMENTS the commonwealth secretariat Multilateral aid review (Mar) 2013 update progress rating: 2 MAR 2011: Poor Value for Money for UK Aid progress assessment summary performance improving, particularly in area of financial management.

Much less evidence of stronger contribution to development objectives – difficulty in agreeing new strategic plan is key indicator in this regard.
Baseline The Commonwealth Secretariat is the main inter-governmental agency of the Commonwealth, an association of 54 independent states.

It works to strengthen democracy and support development across the membership, and to make the Commonwealth’s voice heard on global issues.

The MAR highlighted several strengths: • The Secretariat has a unique role in the international system that allows it to influence across and beyond its membership.
It has access to a network of networks which enables it to promote south-south and north- south cooperation.

It has a niche role in supporting and representing the needs of small states.

The MAR also highlighted several weaknesses: • The Secretariat’s development programmes do not make a critical contribution to international development objectives.

It has insufficient focus on results.

Its financial resources management requires considerable improvement.
DFID’s reform priorities for the MAR Update were: • Greater focus and prioritisation whilst exiting from development activities where it does not have a comparative advantage – assessed under critical role in meeting development objectives; Focused interventions with increased evidence of follow-through, and demonstration of innovative approaches (e.g.
drawing on new technology) – assessed under contribution to results; Strong leadership from the Secretary-General, and a results-based approach driving budgeting, project design, and decision-making – assessed under strategic and performance management; Financial statements and systems getting a clean bill of health from auditors, and evidence of learning from bad experiences – assessed under financial resources management; Administration costs transparently classified and tightly controlled, a Secretariat-wide procurement policy developed and adhered to, and evidence of the public sector management programme impacting on partner governments’ approach to cost control – assessed under cost and value consciousness.
• • • • • • • • summary of overall progress The Secretariat has made reasonable progress in financial management, with sufficient positive movement to warrant a score change from 1 to 2, some progress in results, strategic and performance management and cost and value consciousness, but little or no progress in its contribution to development objectives.
119 DRIVING REFORM TO ACHIEVE MULTILATERAL EFFECTIVENESS Mar reform component progress against reform priorities Mar 2011 score progress rating Mar update score, if any change critical role in meeting development objectives The Secretariat continues to make modest contributions on important development issues, including through outreach and advocacy.
However, the Secretariat’s proposals for a strategic plan focused on a small number of areas where they can add value did not secure consensus, and the agreed plan is broader-based.

contribution to results The Secretariat developed new partnerships and sought to innovate through the use of information technology.

Management has been seen to intervene pro-actively in project management.

Programmes appear to be delivering against their objectives, but there is little evidence of increased impact.
strategic and performance management There has been some progress with the introduction of results-based management, and the new strategic plan includes a stronger results framework.

The HR function has been strengthened, but there are still weaknesses to be addressed, and the new plan is not as tightly prioritised as the initial draft.

financial resources management The accounts for the last two years have been unqualified, and a comprehensive programme of action has been taken to address weaknesses.
There have also been indications of a more pro-active approach to programme management.

cost and value consciousness The Secretariat has defined administration costs more clearly, but as yet there is little concrete evidence of improved cost consciousness and control.
2 2 2 1 1 Little or no progress Some progress Some progress 2 Reasonable progress Some progress 120 MAR UPDATE SUMMARY ASSESSMENTS european Bank for reconstruction and development (eBrd) Multilateral aid review (Mar) 2013 update progress rating: 2 MAR 2011: Good Value for Money for UK Aid progress assessment summary reasonable progress made with respect to cross-cutting issues of gender and climate change and environmental sustainability.

some progress made on results and cost and value consciousness.
Baseline The EBRD provides finance to businesses from Central Europe and the Western Balkans to Central Asia (and now the Middle East and North Africa) supporting transition towards democratic market economies.

The MAR highlighted several strengths: • • EBRD has a leading role in supporting transition and climate finance in the region.

It has a comprehensive results and performance system with evidence of strong strategic stewardship by the Board and pro-active portfolio management.
It has flexible, innovative use of financial instruments.

It has active budget management – evidence of active re-prioritisation.

• • The MAR also highlighted several weaknesses: • • Its geographical focus does not match with DFID’s.

The link between the impact of EBRD’s programmes on transition, and their impact on people’s lives, is not always well articulated.

• Management support for gender was not strong.


• It has strong partnership behaviour during a crisis, but sometimes it is criticised for working against sector reforms.
DFID’s reform priorities for the MAR Update were: • Increase levels of Bank support to climate change and environmental sustainability mitigation particularly in more innovative and risky projects – assessed under attention to cross-cutting issues (climate change and environmental sustainability); Implementation of the new Gender Action Plan – assessed under attention to cross-cutting issues (gender); Continued efforts to measure the wider development impact of transition activities – assessed under contribution to results; A more explicit focus on cost-effectiveness in administration budgets and project design – assessed under cost and value consciousness.
• • • summary of overall progress Reasonable progress has been made with respect to implementing a new Gender Action Plan and there is evidence of a strong commitment to a more strategic consideration of gender issues.

Expansion of energy efficiency lending into new and difficult markets is the prime example of reasonable progress in terms of the climate change and environmental sustainability component.
Some progress has been made with respect to broadening and extending the results framework to capture the impact of projects on people’s lives.

Improved transparency regarding cost-cutting and re-prioritisation, cost-sharing to fund technical assistance activities and a strategic approach to procurement are the principal examples of the Bank’s progress on cost and value consciousness.
121 DRIVING REFORM TO ACHIEVE MULTILATERAL EFFECTIVENESS Mar reform component progress against reform priorities Mar 2011 score progress rating Mar update score, if any change attention to cross-cutting issues (gender) There is evidence of a strong commitment to a more strategic consideration of gender in project design as well as risk and mitigation.
Reasonable progress has been made implementing the gender action plan, evidenced by new initiatives focused on entrepreneurship and financial inclusion directed at women.

attention to cross-cutting issues (climate change and environmental sustainability) EBRD has deepened its use of innovative instruments, including in countries less responsive to energy lending, and penetrated the domestic housing market in its countries of operation.
The third phase of the Sustainable Energy Initiative was approved including extension to adaptation activities.

contribution to results Some progress has been made towards articulating the impact of EBRD’s work on people’s lives through two rigorous impact assessments and consideration is being given to the broadening of the results framework in two possible ways.

cost and value consciousness Assessment is from a high base.
Progress is evident in: i) more transparent cost-cutting and re-prioritisation during the 2013 budget process; ii) seeking cost-sharing opportunities for technical cooperation activities; and iii) moving towards a strategic approach to procurement, away from a transactional approach.
1 4 3 4 Reasonable progress Reasonable progress Some progress Some progress 122 MAR UPDATE SUMMARY ASSESSMENTS european commission Budget (ec Budget) Multilateral aid review (Mar) update 2013 progress rating: 2 MAR 2011: Adequate Value for Money for UK Aid progress assessment summary progress on aid allocation and ensuring staff have development expertise.

some progress on gender and a results framework.

More progress needed on evaluation and managing for value for money.
Baseline The EC Budget provides funding for the EU’s external aid programmes throughout the world, with the exception of the African, Caribbean and Pacific regions.

It is comprised of a number of geographical and thematic instruments.

The MAR highlighted several strengths: • The EC budget funds 90 country programmes, many of which are UK priorities but do not receive UK aid.

The budget instruments are key for wider UK priorities on EU enlargement, the Neighbourhood and the Middle East.
Financial accountability is strong and well established.

The MAR also highlighted several weaknesses: • • • • There is limited focus on the poorest countries.

There is little evidence of a uniform approach to gender.

There is no overall results framework.

Non-budget support assistance has less of a focus on value for money.
DFID’s reform priorities for the MAR Update were: • Strengthen gender mainstreaming in practice and measurement of impact of gender work – assessed under attention to cross-cutting issues (gender); Strengthen results-based approach to aid, including by implementing a results framework – assessed under strategic and performance management and cost and value consciousness; Improve ability to recruit and maintain development expertise – assessed under strategic and performance management; Develop a more effective evaluation function – assessed under strategic and performance management; Allocate resources according to needs and performance – assessed under financial resources management; Keep administrative costs under control – assessed under cost and value consciousness.
• • • • • • • summary of overall progress A Gender Action Plan has been introduced but more attention is needed, particularly by senior management.

Some progress is being made on implementing a results framework, which should be ready by mid-2014.

This should give more opportunity to manage for value for money.

Progress has been made in ensuring that delegations have the right expertise to deliver their programmes.

The evaluation function has made less progress than expected.
The development cooperation instrument will increase its focus on the poorest and most fragile states.

123 DRIVING REFORM TO ACHIEVE MULTILATERAL EFFECTIVENESS Mar reform component progress against reform priorities Mar 2011 score progress rating Mar update score, if any change attention to cross-cutting issues (gender) Around 50% of gender action plan targets have been met but there is evidence of weak management commitment.
strategic and performance management Implementation of a results framework has started and one should be in place by mid-2014.

There have been improvements in human resource management.

financial resources management Greater focus on the poorest countries will be delivered by graduating some countries from bilateral aid and the implementation of allocation criteria agreed in the Agenda for Change.
cost and value consciousness Improvements have been made to procurement regulations and OECD data suggests administrative costs are in line with other multilaterals.

The results framework should provide an opportunity to better manage for value for money.
2 2 2 3 Some progress Some progress Reasonable progress Some progress 124 MAR UPDATE SUMMARY ASSESSMENTS european community Humanitarian office (ecHo) Multilateral aid review (Mar) update 2013 progress rating: 2 MAR 2011: Very Good Value for Money for UK Aid progress assessment summary progress on important reform priorities including processes to ensure gender taken into account across programmes, and steps to link relief to longer-term development.
limited progress on integrating environment and climate change into humanitarian response.

Baseline ECHO is the European Commission’s Directorate-General (DG) for Humanitarian Aid and Civil Protection.

It is the second largest international humanitarian donor, after the US, disbursing £1.1 billion in 2012 to a wide range of large-scale emergencies and protracted and forgotten crises.
The MAR highlighted several strengths: • • • ECHO’s work of humanitarian aid and its continual improvement is aligned with UK priorities.

ECHO’s role in disbursing EU funds quickly in emergency situations is crucial.

Strong mandate and policy framework for dealing with fragile and conflict sensitive situations.

The MAR also highlighted several weaknesses: • • • Weak link between humanitarian assistance and longer-term development.
Committed to gender equality but has not yet been embedded into operational practice.

No formal system to assess ECHO’s environmental impact.
DFID’s reform priorities for the MAR Update were: • Embed gender considerations into operational practice – assessed under attention to cross- cutting issues (gender); Systematically assessing the environmental impact of ECHO’s activities – assessed under attention to cross-cutting issues (climate change and environmental sustainability); Strengthen the link between humanitarian interventions and longer-term development, through working with other parts of the EU – assessed under strategic and performance management.
• • summary of overall progress ECHO has demonstrated its commitment to addressing the UK’s reform priorities.

At headquarters, there has been progress on the gender priority and linking relief and longer-term development.

At the operational level, the full impact of these changes is yet to be fully felt, but only two years after the MAR we would not expect to see substantial evidence of change across all field operations.
There has been some progress on assessing the environmental impact of ECHO’s activities.

125 DRIVING REFORM TO ACHIEVE MULTILATERAL EFFECTIVENESS Mar reform component progress against reform priorities Mar 2011 score progress rating Mar update score, if any change attention to cross-cutting issues (gender) ECHO has developed a new gender policy and gender (and age) marker to track gender-sensitive projects.

This now needs to be embedded across ECHO’s operations.
attention to cross-cutting issues (climate change and environmental sustainability) Progress demonstrated in scale-up of ECHO’s work on disaster risk reduction and introduction of context-based approach to environmental impact assessment.

Still needs to put in place processes for measuring the environmental impact of ECHO’s projects.
strategic and performance management Progress has been demonstrated through leadership and innovation in joint ECHO/DEVCO (Commission DG for Development Cooperation)/EEAS (European External Action Service) working in the Horn of Africa and the Sahel on flagship resilience-building initiatives.

Further progress needed in implementing the Commission Action Plan on resilience and demonstrating impact on the ground though joint working between humanitarian and development actors.
2 2 2 Some progress Some progress Reasonable progress 126 MAR UPDATE SUMMARY ASSESSMENTS european development fund (edf) Multilateral aid review (Mar) update 2013 progress rating: 2 MAR 2011: Very Good Value for Money for UK Aid progress assessment summary progress on aid allocation and ensuring staff have development expertise.

some progress on gender and a results framework.

More progress needed on evaluation and managing for value for money.
Baseline The European Development Fund (EDF) is the main instrument for EU aid to African, Caribbean and Pacific countries and the Overseas Countries and Territories.

It is managed by the European Commission (EC) but is not part of the EC budget.

The MAR highlighted several strengths: • • • • The EDF is critical for progress on the MDGs and poverty reduction.

Financial accountability is strong and well established.

Aid is allocated on a needs and performance basis, based on published criteria.
High levels of budget support with results-based tranches encourage partner countries to look at value for money issues.

The MAR also highlighted several weaknesses: • • • There is little evidence of a uniform approach to gender.

There is no overall results framework in place.

Non-budget support assistance has less of a focus on value for money.
DFID’s reform priorities for the MAR Update were: • Strengthen gender mainstreaming in practice and measurement of impact of gender work – assessed under attention to cross-cutting issues (gender); Strengthen results-based approach to aid, including by implementing a results framework, and improve the evaluation function – assessed under strategic and performance management and cost and value consciousness; Improve ability to recruit and maintain development expertise – assessed under strategic and performance management; Allocate resources according to needs and performance – assessed under financial resources management; Release aid on schedule – assessed under financial resources management; Keep administrative costs under control – assessed under cost and value consciousness.
• • • • • summary of overall progress A Gender Action Plan has been developed but more attention is needed, particularly by senior management.

Some progress is being made on implementing a results framework, which should be ready by mid-2014.

This should give more opportunity to manage for value for money.

Progress has been made in ensuring that delegations have the right expertise to deliver their programmes.

The evaluation function has made less progress than expected.
The new aid allocation model will lead to increased resources for poor countries and fragile states.

127 DRIVING REFORM TO ACHIEVE MULTILATERAL EFFECTIVENESS Mar reform component progress against reform priorities Mar 2011 score progress rating Mar update score, if any change attention to cross-cutting issues (gender) Around 50% of gender action plan targets have been met but there is evidence of weak management commitment.
strategic and performance management Implementation of a results framework has started and one should be in place by mid-2014.

There have been improvements in human resource management.

financial resources management A new aid allocation model will lead to increased resources for poor countries and fragile states.

The 2012 disbursement to commitment ratio is over 100%.

cost and value consciousness Improvements have been made to procurement regulations and administrative costs have been kept down.
The results framework should provide an opportunity to better manage for value for money.

2 2 3 3 Some progress Some progress Reasonable progress Some progress 128 MAR UPDATE SUMMARY ASSESSMENTS food and agriculture organization (fao) Multilateral aid review (Mar) 2013 update progress rating: 3 MAR 2011: Poor Value for Money for UK Aid progress assessment summary progress against all reform priorities, with sufficient improvement to warrant score change in area of transparency and accountability.
new leadership introduced greater focus on strategic prioritisation and results.

Human resource reform remains priority.

Baseline FAO leads on food and agriculture issues in the UN system, covering standard-setting and facilitation of international treaties, policy and technical support, and emergency response.

The MAR highlighted several strengths: • FAO has a role in contributing to global food and nutrition security and delivering the MDGs, particularly on hunger and poverty.
FAO pays attention to cross-cutting issues, including performance in fragile contexts.

FAO has an inclusive partnership approach, providing a neutral international platform to agree international treaties.

• • The MAR also highlighted several weaknesses: • • • • FAO has insufficient focus on results, particularly in developing countries.

FAO has insufficient human resources management for better performance and accountability.

There is no culture of value for money across the organisation.
There is reduced transparency, including in financial management and in dialogue between senior management and membership.
DFID’s reform priorities for the MAR Update were: • Reporting on outputs and outcomes at a country and organisational level for a more consistent focus on results – assessed under contribution to results; • Merit-based appointments (especially country representatives) and a better staff performance management system in use – assessed under strategic and performance management; • More consistent, strategic performance in developing countries and better management of poorly-performing projects – assessed under strategic and performance management and financial resources management; Commitment to, and implementation of, a plan to improve efficiency – assessed under cost and value consciousness; A culture of transparency starting with a presumption of disclosure – assessed under transparency and accountability.
• • summary of overall progress FAO has introduced greater prioritisation and more focus on results through streamlined strategic objectives and new results frameworks at country and corporate levels.

Work is continuing on finalising indicators.

Recruitment processes have been improved and performance management systems introduced for all staff.

However, human resource reform remains a priority.


New leadership is introducing a greater sense of value for money and significant additional savings have been achieved.

Transparency and accountability are improving but there have been delays in ensuring adherence to the International Public Sector Accounting System (IPSAS) and to introducing the FAO’s new Enterprise Risk Management policy.
129 DRIVING REFORM TO ACHIEVE MULTILATERAL EFFECTIVENESS Mar reform component progress against reform priorities Mar 2011 score progress rating Mar update score, if any change contribution to results Strategic objectives have been streamlined from 11 to 5, establishing greater prioritisation and focus on results.

A new results framework for 2014-15 has been developed, including the translation of normative work to impact in developing countries.
Finalising indicators is due to be completed by the end of 2013. strategic and performance management New country programming frameworks are ensuring that all country offices have results frameworks for a more consistent, strategic approach, linked to the revised corporate results framework.

The performance evaluation management system is now used by all FAO staff.

Recruitment processes have improved, including in developing countries.

Human Resource reform remains a priority.


financial resources management FAO has started to improve the performance of decentralised offices with delegations of authority and reinforced capacity in procurement.

A new global resource management system has been introduced, enhancing efficiency and financial management.

This needs to be rolled out to all country offices in 2013.

Finalisation of FAO’s enterprise risk management policy and of IPSAS- compliant systems have been deferred.
cost and value consciousness FAO management has been more proactive in identifying efficiency savings since the MAR, with over US$19 million savings channelled into priority areas in addition to $34.5 million requested by membership.

Awareness of value for money is increasing, from a low baseline, but will take time to become embedded across the organisation.

A long- term plan for efficiency has not yet been developed.
transparency and accountability Since the MAR, FAO has introduced an internal audit disclosure policy and a financial disclosure policy for staff.

A whistleblower policy is being implemented.

New leadership is introducing a greater sense of accountability, including more dialogue with membership.

FAO is taking forward plans to publish more project information and to sign up to IATI, but an implementation plan for this is still to be developed.
2 2 2 2 1 Significant progress Reasonable progress Some progress Some progress 2 Reasonable progress 130 MAR UPDATE SUMMARY ASSESSMENTS Global alliance for vaccines and immunisation (Gavi) Multilateral aid review (Mar) 2013 update progress rating: 3 MAR 2011: Very Good Value for Money for UK Aid progress assessment summary continues to deliver results with activity across all reform priorities.

impact of increased focus on market shaping being felt.
Key changes have begun to support health system strengthening.

Baseline The GAVI Alliance is a public-private partnership committed to saving children’s lives and protecting people’s health by increasing access to immunisation in developing countries.

The MAR highlighted several strengths: • GAVI has made a significant contribution to MDG 4 by increasing finance for vaccinations, including from innovative sources, and has improved coverage of new and underused vaccines.
It has a strong partnership with governments, civil society and the private sector.

It provides highly cost-effective health interventions with vaccines selected on strict criteria for health impact and cost effectiveness with appropriate administration costs.

It has effective financial oversight, with a proactive Audit and Finance Committee, an internal auditor and a robust Transparency and Accountability Policy.
The MAR also highlighted several weaknesses: • • The need for more focus on market shaping to reduce prices and secure sustainable supply.

It has relatively poor performance of cash based programmes, particularly Health Systems Strengthening support.

The need for a more systematic evidence based approach to working in fragile contexts.

There is lack of clarity in roles and responsibilities of key partners, such as WHO and UNICEF, and on the inclusion of civil society.
• • DFID’s reform priorities for the MAR Update were: • Better approach to working in fragile settings through the development of a policy and systematic evidence of performance in fragile contexts – assessed under attention to cross-cutting issues (fragile contexts); Greater focus on outcomes and performance, specifically in relation to the delivery of its cash based programmes – assessed under strategic and performance management and financial resources management; Stronger performance on influencing markets and more strategic approach to procurement for sustainable and affordable vaccine supply – assessed under cost and value consciousness; Further alignment and clarity in roles and responsibilities of partners, including civil society – assessed under partnership behaviour.
• • • • • • summary of overall progress GAVI continues to be a high performing institution providing a very cost-effective health intervention.

From the 2013 MAR high baseline, evidence collected for the 2013 MAR Update demonstrates GAVI’s on-going commitment to improvement across the DFID reform priorities and in all at least some evidence of commitment being translated into implementation exists.
However workstreams are at different stages and for some it is too early to judge impact in developing countries.
131 DRIVING REFORM TO ACHIEVE MULTILATERAL EFFECTIVENESS Mar reform component progress against reform priorities Mar 2011 score progress rating Mar update score, if any change attention to cross-cutting issues (fragile contexts) A new policy has been approved – ‘GAVI and fragile contexts: a country by country approach’ – and specific roles for UNICEF and WHO in addressing equity and poor performance are outlined in the 2013-14 business plan.

It is too early to see the impact in developing countries.
strategic and performance management Commitment to improve the management of cash based, particularly health systems strengthening, programmes exists.

Evaluation recommendations have been made e.g.

to improve monitoring and links with immunisation outcomes, but changes are yet to have sufficient impact.
financial resources management Additional measures are included in the transparency and accountability policy for overseeing cash-based support, introduction of country scorecards and other M&E mechanisms, and a strong recent record in recovering funds lost through corruption and fraud.

Some issues remain in individual countries, but overall evidence shows systems to be effective at detecting and handling cases of misuse.
cost and value consciousness The new supply and procurement strategy has been approved and significant price reductions for pentavalent, rotavirus and human papilloma virus (HPV) vaccines will enable more children to be immunised.

However, the pace of implementing the strategy is slower than expected with delays in the production of vaccine roadmaps and scope for further progress in ensuring sustainably lower prices without supply disruptions.
partnership behaviour There is more systematic engagement with civil society, clarification of roles and responsibilities of WHO and UNICEF and increases in the number of country reporting officers.

Changes remain at an early stage of implementation but there is some evidence of improved communication in developing countries.
3 3 3 4 3 Some progress Some progress Reasonable progress Reasonable progress Reasonable progress 132 MAR UPDATE SUMMARY ASSESSMENTS Multilateral aid review (Mar) update 2013 progress rating: Global environment facility (Gef) 3 MAR 2011: Good Value for Money for UK Aid summary progress has been made across all three reform priority areas.

significant progress has been made to improve country ownership.
progress assessment Baseline GEF works in partnership with a number of implementing agencies, providing new and additional grants and concessional funding to meet the agreed incremental costs of measures to achieve agreed global environmental benefits.

The MAR highlighted several strengths: The GEF fulfils a critical niche.

• Establishment of the System for Transparent Allocation of Resources (STAR) supports the • allocation of resources to those countries where support will have the greatest impact.
The GEF has set indicators to ensure implementing agencies achieve value for money.

• The MAR also highlighted several weaknesses: • Improvements in programmatic approaches and project cycle management are needed to improve efficiency of the GEF.

Improvements are required in partnership working, including improving demand driven approaches and improved working with recipient country national entities.
DFID’s reform priorities for the MAR Update were: • Increased support to countries in preparing programmes to maximise the benefits of GEF support – assessed under partnership behaviour; Increased use of programmatic approaches to maximise impact of the GEF support – assessed under partnership behaviour; Streamline project cycle management arrangements to minimise transaction costs and avoid delays in the approval process – assessed under financial resources management; Improved results-based management framework with clear output targets and indicators – assessed under strategic and performance management.
• • • • summary of overall progress The introduction of national portfolio formulation exercises and a pilot to increase the number of implementing agencies will support recipient countries to align GEF funding with national priorities and increase country choice of agency and therefore country uptake.

This has resulted in a positive score change under the partnership behaviour.
GEF has made reasonable progress across other reform priorities, developing a results based management framework and streamlining project cycle management, to avoid delays in the approval process.

133 DRIVING REFORM TO ACHIEVE MULTILATERAL EFFECTIVENESS Mar reform component progress against reform priorities Mar 2011 score progress rating Mar update score, if any change strategic and performance management GEF has continued to develop results based management (RBM) through a new RBM framework.
Reform of the Annual Monitoring Report is now complete which will provide the GEF Council with more timely analysis of the portfolio.

GEF is also implementing a number of improvements to the presentation and operational use of programme data.

financial resources management Early evidence suggests a reduction in the average time lag from project approval to CEO endorsement, from 22 months to 18 months, but at present the time lapse for 75% of projects is unknown.
Reduced agency fees have been established for all 10 implementing agencies; falling to 9.5% for grants up to $10 million and 9% for grants above $10 million.

partnership behaviour GEF has introduced national portfolio formulation exercises (NPFEs) which are voluntarily undertaken by recipient countries to align GEF funding with national priorities and strategies and also increase countries choice of agency and country uptake.

To date 42 countries have undertaken an NPFE.


A pilot to accredit up to 10 new GEF agencies is currently on-going.

This will give recipient countries a greater choice of agencies to work with.
3 3 2 Reasonable progress Some progress 3 Reasonable progress 134 MAR UPDATE SUMMARY ASSESSMENTS the Global fund to fight aids, tB and Malaria (GfatM) Multilateral aid review (Mar) 2013 update progress rating: 3 MAR 2011: Very Good Value for Money for UK Aid progress assessment summary significant reforms, all within 18 months, show a strong level of commitment to improvements, though it is too early to realise the full benefits in developing countries.

overall progress has been reasonable.


Baseline GFATM is a global public/private partnership that raises and disburses funds to prevent and treat HIV/AIDS, tuberculosis and malaria.

Since its creation in 2002, it has become the biggest multilateral funder of health related MDGs.

The MAR highlighted several strengths: • The fund finances a range of high impact interventions throughout pre-pregnancy, pregnancy, birth and childhood, with an important impact on MDGs 4 and 5.
Its beneficiary voice is reasonably well embedded into all layers of governance.

The fund’s decision to publish and require recipients to publish procurement data has been a major driver for a range of innovations in transparency.

The MAR also highlighted several weaknesses: • There are clear weaknesses and bottlenecks in the business model which impede faster progress and even more impressive results.
The fund places heavy burdens on countries and partners, and its own systems and requirements often take precedence.

The time between grant approval and disbursement is not quick enough which results in large ‘cash balance’ on the Global Fund’s books.
DFID’s reform priorities for the MAR Update were: • To reduce the costs the fund imposes on recipients and partners, through reform of the business model – assessed under strategic and performance management and partnership behaviour; To have better value for the money spent through the fund, by securing lower prices, more effective use of cash balances, and operational efficiencies – assessed under financial resources management and cost and value consciousness; To have continued focus on the poorest and the most vulnerable (high disease burden areas) – assessed under strategic and performance management and financial resources management; To develop policies that address the longer-term sustainability of GFATM-funded programmes (including better M and E) – assessed under strategic and performance management.
• • • • • • • summary of overall progress The scope of the fund’s reform over the past 18 months has been far-reaching: substantial and difficult reforms encompassing all elements of its structure and operations from strategy, governance, organisational transformation and implementation through a new funding model have been undertaken rapidly for an organisation of this size.
135 DRIVING REFORM TO ACHIEVE MULTILATERAL EFFECTIVENESS Mar reform component progress against reform priorities Mar 2011 score progress rating Mar update score, if any change strategic and performance management The scope of the fund’s reform over the past 18 months has been very substantial, encompassing all elements of its structure and operations.

The pace of change has been very rapid and the secretariat now has a clear remit to improve performance.
financial resources management There has been substantial reform at all levels designed to address slow disbursement of grants and other financial management challenges.

The new funding model gives countries greater funding predictability, and a more specific risk management approach is now applied to each grant.

New financial management systems are being put in place and implemented.
cost and value consciousness Robust recruitment processes for senior appointments, including the Chief Financial Officer (CFO), Treasurer and Head of Procurement, plus a reduction in staff numbers of 7.4% and in operating expenses of 12%, and a further 5% planned for 2013, show evidence of increased cost consciousness.

New management and the CFO have given assurances that there will be further improvements which we will monitor.
partnership behaviour There is greater stakeholder participation in grant renewal decisions and in new disease committees.

Fund portfolio managers have been strengthened to engage better with countries.

Country coordinating mechanisms and local fund agents will be regularly reviewed; the new funding model will be more reliant than previously on national strategies.
3 3 3 2 Reasonable progress Reasonable progress Reasonable progress Some progress other relevant information The fund came out well in the original MAR.

It had expanded rapidly as it attempted to scale up treatment for the three diseases and was delivering results at scale; but as it had expanded, its costs were increasing and its processes and requirements had become ever more burdensome on implementing countries.
A variety of events during 2011, sparked by press reports on fraud and corruption in some programmes exacerbated this situation.

The board decided that it could no longer continue with the old approach.
It established a high level panel to look at the fund’s fiduciary and oversight systems and processes, and the report and comprehensive recommendations of this high level panel, coupled with other negative events in 2011, proved the catalyst for a transformational reform process, both in pace and breadth, which has rolled out since.
136 MAR UPDATE SUMMARY ASSESSMENTS Global facility for disaster reduction and recovery (Gfdrr) Multilateral aid review (Mar) 2013 update progress rating: 2 MAR 2011: Good Value for Money for UK Aid progress assessment summary Has made some progress in addressing reform priorities.

steps have been taken to improve performance in fragile contexts but it is too early to assess impact in developing countries.

More progress needs to be made against contribution to results.
Baseline GFDRR’s core mandate is to integrate disaster risk management (DRM) into development planning and implementation.

It works in some of the most disaster-prone countries in the world.

The MAR highlighted several strengths: • • GFDRR has a clear mandate, purpose and strategy.

It has the ability to leverage significant additional investment to support Disaster Risk Reduction (DRR) activities, in particular from the World Bank.

It has a responsive secretariat with high calibre staff.
• The MAR also highlighted several weaknesses: • There have been initial challenges in scaling up activities, and programmes are of variable quality and consistency.

There has been weaker performance in fragile and conflict-affected states.

It is not effectively co-ordinating with donors and engaging with civil society.
DFID’s reform priorities for the MAR Update were: • Greater integration and investment of DRM in target country development policies and poverty reduction strategies – assessed under strategic and performance management; Adoption of risk financing frameworks in GFDRR priority countries – assessed under attention to cross-cutting issues (fragile contexts); Results-based management system developed and implemented – assessed under contribution to results; Integration of civil society into disaster risk reduction work in priority countries – assessed under partnership behaviour.
• • • • • summary of overall progress GFDRR has demonstrated some progress and continues to show commitment in addressing issues raised in the original MAR and other reviews.

These are: • • • Prioritising partnerships with civil society; Supporting the development of risk financing frameworks; Growing engagement in fragile contexts, including a focus on learning how to operate more effectively in these environments.

This learning now needs to be translated into best practice, lessons learned and guidance.
Challenges remain: • Although GFDRR can demonstrate that it is helping to mainstream DRM, in particular in World Bank programmes, it is not yet demonstrating what impact these investments are having on the ground due to weak monitoring and evaluation systems.
137 DRIVING REFORM TO ACHIEVE MULTILATERAL EFFECTIVENESS Mar reform component progress against reform priorities Mar 2011 score progress rating Mar update score, if any change attention to cross-cutting issues (fragile contexts) GFDRR has growing investment and improved performance in fragile contexts, which it should now capitalise on to generate lessons learned and guidance.
contribution to results GFDRR is making progress in mainstreaming disaster risk management, in particular leveraging World Bank finance, but is not yet demonstrating the impact its investments are having.

strategic and performance management Efforts have been made to improve monitoring and reporting of results, but this remains work in progress.

There is commitment from leadership to improve this.

There is strong donor demand for demonstrating results on the ground.
partnership behaviour Considerable effort has been made in developing a civil society strategy, which now needs implementing.

It needs to support the World Bank in playing a stronger role in coordinating donor investment in disaster risk management.
2 3 2 3 Reasonable progress Little or no progress Some progress Reasonable progress 138 MAR UPDATE SUMMARY ASSESSMENTS Global partnership for education (Gpe) Multilateral aid review (Mar) 2013 update progress rating: 3 MAR 2011: Good Value for Money for UK Aid (reviewed as Education For All – Fast Track Initiative) progress assessment summary Has made progress against all reform priorities.
progress includes expanding and improving quality of support to countries; increasing efficiency, transparency and predictability in financial management; and publishing consolidated reporting on results.

Baseline The Global Partnership for Education (GPE) is the only multilateral exclusively focused on supporting low income countries to educate children from pre-primary through secondary school, providing financial and policy support to countries to develop and implement their own education sector plans.
The MAR highlighted several strengths: • • GPE spent 65% of its resources in countries in the top quartile of the index of poor countries.

GPE makes a strong contribution to MDGs 2 and 3.

It incentivises donors to align behind country plans.

Positive reforms to governance, financing and results approved in November 2010, and has an open disclosure policy.
The MAR also highlighted several weaknesses: • The MAR did not find evidence that GPE had shown strong leadership on gender equality nor was it able to provide sufficiently flexible support in fragile contexts.

GPE did not have a global-level results framework, and was not consistently doing enough to help resolve problems in developing countries.

GPE’s progress on disbursement and grant management was weak.

Funding allocations were not transparent or predictable.
DFID’s reform priorities for the MAR Update were: • To provide effective support to fragile contexts, including prioritising implementation funding – assessed under attention to cross-cutting issues (fragile contexts); To provide greater leadership on girls’ education and disaggregating and reporting results by gender – assessed under attention to cross-cutting issues (gender); To have clear, evidenced and compelling results reporting – assessed under contribution to results; To have improved operational effectiveness: a simplified, more flexible Single Fund; stronger leadership and management capacity; addressing delays in disbursement – assessed under strategic and performance management; To provide more, and more sustainable, funding for education from a range of stakeholders – assessed under strategic and performance management; To ensure that GPE can work with a range of institutions appropriate to particular country contexts – assessed under partnership behaviour.
• • • • • • • • summary of overall progress GPE has demonstrated progress against each of the MAR reform priorities.

It now publishes aggregated results reporting as well as an increasing volume of data on its work.

It has adapted its model to make it easier for fragile countries to use its support.

GPE is engaging more consistently with countries to ensure gender is addressed in country plans and is partnering with the UN Girls’ Education Initiative (UNGEI).
It has strengthened financial management including the transparency and predictability of country grants, as well as improving leadership and management through the appointment of a new, senior CEO and an organisational restructure.

The 2011 replenishment raised $1.5 billion and aims to reach $2 billion by the end of 2014, and there are now seven supervising/ managing entities, compared to three at end 2011.
139 DRIVING REFORM TO ACHIEVE MULTILATERAL EFFECTIVENESS Mar reform component progress against reform priorities Mar 2011 score progress rating Mar update score, if any change attention to cross-cutting issues (fragile contexts) GPE has taken practical steps to improve its support to fragile contexts (improved fragility and conflict guidelines, accelerated funding for fragile contexts, increased supervision allocations) and has increased the pool of fragile contexts it funds by 50%.
attention to cross-cutting issues (gender) GPE has revised its appraisal process to promote more constructive challenge to country applications on gender equality, made girls’ education one of its five main objectives, and agreed a new partnership with the UNGEI to promote girls’ education.

contribution to results GPE has addressed a key weakness which was the lack of aggregated results reporting.

It reports on output results for GPE funding and education outcomes in developing countries.
However, reporting is not yet sufficiently clear and compelling, and GPE does not use results systematically to manage performance.

strategic and performance management GPE set clear and stretching objectives in its 2012 strategic plan.

It is planning a major evaluation, has recruited a new CEO, and reports on results, but some management and governance issues remain.

financial resources management GPE has a new aid allocation model which is transparent and predictable.
Accelerated disbursement and proactive portfolio management have sped up delivery to countries.

partnership behaviour An expanded country support team is delivering a tangible improvement in the service GPE offers to countries: better advice and improved ability to support countries in resolving problems.
3 3 2 2 2 3 Reasonable progress Reasonable progress Reasonable progress Some progress Significant progress Reasonable progress 140 MAR UPDATE SUMMARY ASSESSMENTS inter-american development Bank (iadB) Multilateral aid review (Mar) 2013 update progress rating: 2 MAR 2011: Adequate Value for Money for UK Aid progress assessment summary demonstrated strong commitment to implementing reforms.

some progress made against gender, transparency and accountability, and cost and value consciousness.
reasonable progress made on strategic and performance management.

IADB is the largest multilateral lender to Latin America and the Caribbean (LAC).

It works with regional governments and leads on climate change and environmental sustainability, cross-border infrastructure and regional integration.

Baseline The MAR highlighted several strengths: • IADB has a strong focus on wealth creation and expanding poverty and climate change and environmental sustainability operations.
It effectively responds to natural disasters, particularly in Haiti.

It links loan charges to administrative costs to encourage cost savings and has good financial and risk management systems.

The MAR also highlighted several weaknesses: • LAC region has pockets of real poverty but does not match DFID’s focus on regions with higher poverty levels.

Challenges remain on gender equality, especially in the harder areas, such as infrastructure projects.
There is no evidence of emphasis on securing cost effectiveness in the design of development projects.
DFID’s reform priorities for the MAR Update were: • Strengthening the impact of gender policy and activities – assessed under attention to cross- cutting issues (gender); Delivering clear sector strategies and targets for results – assessed under strategic and performance management; Improving cost and value consciousness in administration budgets and in project design – assessed under cost and value consciousness; Demonstrating whether increased transparency has resulted in enhanced accountability to project beneficiaries/borrowing governments/shareholders – assessed under transparency and accountability.
• • • • • • • summary of overall progress DFID MAR reform priorities were included in the Bank’s institutional strategy, which set out reforms (known as Ninth General Capital Increase reforms – GCI-9).

DFID agrees with the conclusions of an independent review commissioned by the Bank’s Board of Governors.

This concluded that the reforms were being implemented fully and effectively.
It found that: • “full implementation” had been met or was in process but progress toward “effective implementation” has been mixed; the focus on results has increased, particularly at the project level; safeguards, including gender, have been strengthened, but implementation challenges remain; the policy on information disclosure needs further clarification; the independent complaints mechanism needs reform.
• • • • In March 2013, the Bank’s governors recommitted themselves to fully implement reforms to ensure that all objectives are achieved by 2015.

141 DRIVING REFORM TO ACHIEVE MULTILATERAL EFFECTIVENESS Mar reform component progress against reform priorities Mar 2011 score progress rating Mar update score, if any change attention to cross-cutting issues (gender) Some progress has been made in meeting targets on gender diversity of Bank staff.
The gender action plan has made a good start, with stronger progress on the proactive (country and sector- specific analytical work, direct investments and mainstreaming) than the preventive (safeguards screening in the design and supervision of lending operations) side.

Making gender concerns an integral part of loan operations especially in harder to reach sectors (e.g.

trade and infrastructure) remains work in progress.
strategic and performance management The Bank made good progress in improving results measurement and monitoring at the project level.

It has delivered new sector strategies on time.

The Bank is updating all of its policies and introducing new sector framework documents by end 2015. cost and value consciousness Information related to the budget and work program has improved, but more progress is needed on a dialogue with partners on programme cost effectiveness.
transparency and accountability Several key initiatives have contributed to increased transparency, but the access to information policy could be further improved and the Bank’s independent consultation and investigation mechanism has been ineffective and needs thorough redesign.
2 2 3 3 Some progress Reasonable progress Some progress Some progress 142 MAR UPDATE SUMMARY ASSESSMENTS international committee of the red cross (icrc) Multilateral aid review (Mar) 2013 update progress rating: 3 MAR 2011: Very Good Value for Money for UK Aid progress assessment summary commitment to address reforms.
progress is underway, in particular, on climate change and environmental sustainability where steps have been taken to understand and reduce the environmental impact of programmes and operations.

increased transparency with donors from headquarters.

ICRC is a private organisation with an exclusively humanitarian mission.

It directs and coordinates the international relief activities conducted by the Red Cross Movement in situations of conflict.
Baseline The MAR highlighted several strengths: • ICRC has unparalleled access to vulnerable populations; it is often the only organisation offering assistance and protection, particularly in remote areas.

Cost efficiency underpins ICRC’s financial management and value for money considerations are taken into account in programming.

It has strong partnerships with governments, donors and the Red Cross movement.
The MAR also highlighted several weaknesses: • ICRC operates on a policy of confidentiality rather than transparency to deliver its mandate.

This makes ICRC less transparent as an organisation.

The degree of ICRC accountability to recipient country stakeholders is not clear.

ICRC’s willingness to engage with humanitarian partners in country is not consistent and depends in large part on the particular ICRC delegates in country.
DFID’s reform priorities for the MAR Update were: • Factoring environmental and climate change and environmental sustainability issues into all its policies and operations – assessed under attention to cross-cutting issues (climate change and environmental sustainability); Improving accountability to beneficiaries, and transparency – assessed under transparency and accountability.
summary of overall progress Evidence indicates that ICRC has taken the reforms seriously and taken forward a number of initiatives, largely at headquarters, to address areas of weakness.

It is too early to see impact at field level, but the ICRC management demonstrate strong commitment to addressing the reform priorities identified.
Key achievements to date include: • • A new strategic framework for sustainable development to guide policies and programmes; Piloting of new approaches to understand and mitigate the impact of ICRC’s activities on the environment; Increased, dedicated resource to focus on environmental issues; Increased transparency – new policy forums with donors and willingness to integrate with the wider international humanitarian system; Continued focus on accountability to stakeholders and increased attention to participatory planning.
• • • • • • • • 143 DRIVING REFORM TO ACHIEVE MULTILATERAL EFFECTIVENESS Mar reform component progress against reform priorities Mar 2011 score progress rating Mar update score, if any change attention to cross-cutting issues (climate change and environmental sustainability) Substantial progress has been made.
A strategic framework is in place to guide policies, additional dedicated staff resources have been appointed and concrete actions are underway to understand the impact of ICRC’s activities and how best to mitigate this.

transparency and accountability Progress is being made.

Steps have been taken to increase transparency, in particular with donors.

ICRC continues to ensure the views of beneficiaries are taken into account in their programmes and activities.
2 2 Significant progress Reasonable progress 144 MAR UPDATE SUMMARY ASSESSMENTS international development association (ida) Multilateral aid review (Mar) 2013 update progress rating: 3 MAR 2011: Very Good Value for Money for UK Aid progress assessment summary taking reform seriously.

reasonable progress made against gender, fragile contexts and contribution to results reform priorities.

some progress made with respect to partnership behaviour.
IDA is part of the World Bank, which aims to reduce poverty by providing loans and grants for programmes that boost economic growth, reduce inequalities, and improve people’s living conditions.

Baseline The MAR highlighted several strengths: • • • • IDA has strong capacity in a range of sectors.

It has good delivery against challenging development objectives.

It has staff of high quality.

Evaluation is a core strength.


The MAR also highlighted a number of weaknesses: • • Presence, timeliness and delivery in fragile contexts are weak.

Adherence to gender policy in core IDA country operations is weak.
DFID’s reform priorities for the MAR Update were: • Improved performance, coordination and resourcing of the Bank – assessed under attention to cross-cutting issues (fragile contexts); Stronger focus on girls and women in country strategies, operations and policy dialogue – assessed under attention to cross-cutting issues (gender); Stronger results framework and more appropriate procedures and instruments – assessed under contribution to results; • • • More flexible instruments and reforms to procedures, which facilitate stronger partnership working – assessed under partnership behaviour.
summary of overall progress The World Bank is taking reform seriously and is making itself a more agile, efficient organisation.

The Bank has made reasonable progress against three reform priorities identified in the MAR – gender, fragile contexts and contribution to results.

Some progress has been made against partnership behaviour reform priority.

It is too early to tell what impact corporate initiatives are having in developing countries.
145 DRIVING REFORM TO ACHIEVE MULTILATERAL EFFECTIVENESS Mar reform component progress against reform priorities Mar 2011 score progress rating Mar update score, if any change attention to cross-cutting issues (gender) A number of corporate initiatives have been introduced to encourage greater focus on gender.
The main achievements include: introduction of a gender data portal; publication of the World Development Report on Gender and Development; and improvements in indicators tracking gender mainstreaming.

It is too early to evaluate the impact of initiatives to improve systems in developing countries.

attention to cross-cutting issues (fragile contexts) The Bank has prioritised working in fragile contexts in recent corporate initiatives and established the Nairobi Hub to catalyse change.
Improved engagement is becoming evident on the ground.

contribution to results Delivering for results was the over-arching theme for IDA16.

The Bank is using its corporate scorecard and IDA 16 results management system to give greater focus on results throughout the organisation.

Approval of programme for results (P4R) instrument is a significant step.
partnership behaviour Modernisation reforms are providing the basis to address concerns raised in the MAR around the use of country systems and the flexibility of Bank procedures.

Reform of the Investment Lending instrument and approval of the P4R instrument is going some way to addressing these issues.
2 2 3 2 Reasonable progress Reasonable progress Reasonable progress Some progress 146 MAR UPDATE SUMMARY ASSESSMENTS international federation of the red cross (ifrc) Multilateral aid review (Mar) 2013 update progress rating: 2 MAR 2011: Good Value for Money for UK Aid progress assessment summary commitment to improve the capacity of national societies, widen partnerships, including with private sector and improve financial management.

Mixed progress in developing countries.


Baseline IFRC is a global humanitarian network that carries out relief operations to assist victims of disasters, and combines this with development work to strengthen the capacities of its 187 National Societies.

The MAR highlighted several strengths: • Scale and reach of the organisation means that it is a critical humanitarian actor and it is often the first to respond to humanitarian emergencies on the ground.
IFRC has a clear gender policy and promotes gender policies within National Societies.

IFRC operates cost effectiveness: striving for cost control in its logistics and decentralising process.

IFRC has a clear mandate, strategy and effective governing body.

The MAR also highlighted several weaknesses: • • The capacity of National Societies is variable.

Despite some improvements, performance management is not yet sufficiently embedded in developing countries.
Strong financial reporting and systems are in place in the secretariat level, but systems in developing countries are more limited.

There is no formal mechanism that allows donors and partner governments to collectively hold IFRC to account.
• • DFID’s reform priorities for the MAR Update were: • leadership and capacity of National Societies and performance and financial management at the country level – assessed under strategic and performance management and financial resources management; partnerships outside the Red Cross Movement – assessed under partnership behaviour; accountability through, for example, establishing a donor support mechanism – assessed under transparency and accountability.
• • • • • summary of overall progress IFRC is addressing the MAR reform priorities to varying degrees.
Key accomplishments to date include: • Steps to improve the performance of national societies by developing strategic plans and increased resources focusing on planning, monitoring and evaluation; Innovative ideas to save money, for example reducing the average monthly cost of renting a vehicle (by NS) by more than 30%; Accountability to donors improved following the establishment of a Donor Advisory Group.
• • Challenges remain and the pace of reform and uptake appears slow: • New initiatives to understand and address the capacity constraints of National Societies (Organisational Capacity Assessment and Certification process) have yet to have traction across the organisation.

Implementation of the Federation-Wide Reporting System is low, which limits IFRC’s ability to measure results and improve performance.
IFRC needs to exert more influence over NS to curb poorly performing projects and to capacity build NS to recognise and address these issues themselves.

• • 147 DRIVING REFORM TO ACHIEVE MULTILATERAL EFFECTIVENESS Mar reform component progress against reform priorities Mar 2011 score progress rating Mar update score, if any change strategic and performance management New policies and systems are in place, with a focus on performance management.
However, the full impact of changes are not yet realised at field level.

financial resources management There have been some positive steps but more is to be done to curb poorly performing projects and to build capacity – a priority in order to deliver results in future years.

partnership behaviour There are established private sector partnerships and clusters at the global level, improved joint working with agencies outside the Red Cross movement.
transparency and accountability Ability to effect change at NS level still has some way to go.

More work is needed on holding NS and IFRC to account by stakeholders.
2 2 3 2 Reasonable progress Some progress Reasonable progress Little or no progress 148 MAR UPDATE SUMMARY ASSESSMENTS international finance corporation (ifc) Multilateral aid review (Mar) 2013 update progress rating: 2 MAR 2011: Good Value for Money for UK Aid progress assessment summary Has taken reform seriously and shown commitment in addressing reform priorities on gender and partnerships.
progress on reform priorities relating to fragile contexts and increased investments into low income countries slower than expected, but moving in right direction.

Baseline The International Finance Corporation (IFC) is part of the World Bank Group.

IFC fosters sustainable economic growth in developing countries by financing private sector investment, mobilising capital in international financial markets and providing advisory services to businesses and governments.
The MAR highlighted several strengths: • IFC is the largest Development Finance Institution (DFI) and the only multilateral DFI with a global reach.

IFC plays a significant coordination role at international level.

Financial management, independent audit and transparency are very strong.

The MAR also highlighted a number of weaknesses were also identified: • • • IFC’s investment portfolio was heavily concentrated in Middle Income Countries.

Strategic corporate targets did not include gender targets.
Collaborative efforts at a country/project level were limited and the IFC did not always engage effectively with the donor community or other DFIs.
DFID’s reform priorities for the MAR Update were: • Better efforts to target investments at girls and women and to report using gender disaggregated data – assessed under attention to cross-cutting issues (gender); Increased presence and more formal policies for prioritising investments in fragile contexts – assessed under attention to cross-cutting issues (fragile contexts); Better targeting of the poor and more geographic diversification of IFC’s portfolio across Low Income and Middle Income Countries – assessed under focus on poor countries; Stronger in-country partnerships and engagement with DFID – assessed under partnership behaviour.
• • • • • summary of overall progress Since the MAR, the IFC has made good progress in partnering and engaging with DFID and other organisations and on targeting investments at women.

Progress has been slower on the IFC’s reform priority to increase the geographic diversification of its portfolio across low income and middle income countries and on investing in fragile contexts.

However, the most recent IFC Roadmap proposes a significant step-up in these.
149 DRIVING REFORM TO ACHIEVE MULTILATERAL EFFECTIVENESS Mar reform component progress against reform priorities Mar 2011 score progress rating Mar update score, if any change attention to cross-cutting issues (gender) There has been improved targeting of investments at women in some areas, as evidenced by the addition of new staff and the introduction of new IFC Development Goals (IDGs) around increasing access to finance for women-owned businesses.
However, the IDGs on improving access to health and education do not include specific targets for access to girls and women.

attention to cross-cutting issues (fragile contexts) Fragile and conflict states was introduced as a corporate priority in 2012 and the IFC’s office presence in this area has increased, although this has not yet translated into an increase in investment commitments.
In the most recent IFC Roadmap under discussion with the board, proposals have been made to significantly step-up investments.

focus on poor countries The IFC has introduced new Development Goals and a strategy to target the poor within MICs, but geographic diversification within IDA countries and between low- and middle-income countries remains weak.

The IFC plans to address this with its latest proposals to increase its investments in challenging IDA countries.
partnership behaviour The IFC has made efforts to increase engagement with partners.

This has resulted in innovative programs such as the global SME Facility and Climate Catalyst Fund.

A number of new collaborations with DFID at country level are in the process of being finalised.
2 2 2 2 Reasonable progress Some progress Some progress Reasonable progress other relevant information The IFC has made a number of positive steps on other issues since 2011, such as updating its Environmental and Social Performance Standards, increasing its transparency through its new Access to Information Policy, introducing new rules on dealing with investments in off-shore financial centres and increasing activities in the Middle East and North Africa in response to the Arab Spring.
In 2012, the IFC also updated its due diligence policy and procedures.

DFID will be monitoring the implementation of these new procedures.

The new President of the World Bank has set out a new vision to “end poverty and pursue shared prosperity”.

Several initiatives are already underway to see IBRD, IDA and IFC collaborating more effectively.
As part of the upcoming IDA-17 replenishment negotiations, private sector development and maximising IFC’s contribution to poverty elimination and inclusive growth will remain key priorities for the UK.

150 MAR UPDATE SUMMARY ASSESSMENTS international fund for agricultural development (ifad) Multilateral aid review (Mar) update 2013 progress rating: 3 MAR 2011: Good Value for Money for UK Aid progress assessment summary ifad is making progress in all areas.
impact evaluation is being introduced as part of a strengthened results framework.

Human resource reform and financial management are being addressed.

improving efficiency remains work in progress.

Baseline IFAD is an international financial institution (IFI) and a United Nations (UN) specialised agency dedicated to eradicating poverty and hunger in rural areas of developing countries.

The MAR highlighted several strengths: • • • IFAD’s unique mandate that is critical to reducing hunger and poverty.
IFAD has a good results framework and a commitment to improving delivery of results.

IFAD is a trusted partner of developing countries.

The MAR also highlighted several weaknesses: • • • The need to improve sustainability of results in developing countries.

Reform needed to enhance efficiency and improve human resource management.

Financial management needs to be strengthened, processes streamlined and disbursement rates improved.
DFID’s reform priorities for the MAR Update were: • Improved delivery of sustainable results in developing countries – assessed under contribution to results; • More emphasis on impact evaluation, mainstreaming gender and climate change – assessed • • • under strategic and performance management; Reduction of IFAD’s admin-programme cost ratio and improved efficiency – assessed under cost and value consciousness; Introduction of greater flexibility and efficiency in human resources – assessed under strategic and performance management; Strengthening financial management and policies, and improving disbursement rates – assessed under financial resources management.
summary of overall progress IFAD is strengthening its work on results and introducing impact evaluation.

Independent evaluation shows that IFAD’s supervision and implementation support is helping to improve IFAD’s results.

Human resource reform is now being addressed in a systematic way.

Financial management has been improved.

Cost savings have achieved zero growth administration budgets while programming has increased.
However, efficiency remains one of the weaker areas of performance and time will be needed to assess implementation and impact of IFAD’s action plan to improve efficiency.

151 DRIVING REFORM TO ACHIEVE MULTILATERAL EFFECTIVENESS Mar reform component progress against reform priorities Mar 2011 score progress rating Mar update score, if any change contribution to results IFAD has reinforced its focus on results and impact.
The Independent Office of Evaluation shows continuing improvements in delivery in developing countries.

Sustainability remains a particularly challenging area.

strategic and performance management IFAD has strengthened its results framework, introducing impact evaluation and greater focus on gender and mainstreaming climate work.

Progress is being made in HR reform.


financial resources management IFAD has created a new Financial Operations Department and introduced new financial and risk management policies.

Disbursement levels are improving, and this needs to continue.

Technology updates to streamline administrative processes for greater efficiency are planned for 2014-15. cost and value consciousness IFAD has maintained the same level of administration budget while increasing its programming.
Time will be needed to assess progress against new indicators to measure efficiency, as well as the implementation and impact of IFAD’s action plan to improve efficiency.

3 3 2 2 Reasonable progress Reasonable progress Reasonable progress Some progress 152 MAR UPDATE SUMMARY ASSESSMENTS international organisation for Migration (ioM) Multilateral aid review (Mar) 2013 update progress rating: 3 MAR 2011: Poor Value for Money for UK Aid progress assessment summary strong commitment to address reforms.
improved financial planning, humanitarian response capacity and project level reporting.

steps taken to strengthen programme management but evaluation remains underfunded and long term financial planning continues to be challenging.

no overall results framework.


IOM is a global organisation which delivers programmes in over 100 countries where it provides transportation, repatriation and humanitarian assistance to forced migrants, internally displaced people, refugees and other vulnerable groups in crisis situations.

Baseline The MAR highlighted several strengths: • IOM is a key flexible UK partner on migration management and contributes to international responses to humanitarian emergencies.
It has a history of working in fragile and conflict states.

It has wide-ranging global partnerships.

The MAR also highlighted several weaknesses: • IOM does not have a development or humanitarian mandate and only fills a marginal gap in the international humanitarian architecture.

It has very limited financial flexibility due to the projectised nature of its work.

Its performance as a lead agency for camp co-ordination (cluster lead) has been uneven as it struggles to scale-up in emergencies.
DFID’s reform priorities for the MAR Update were: • Improved monitoring and evaluation and results based management of projects – assessed under strategic and performance management; Improved financial planning and humanitarian response capacity – assessed under financial resources management; Stronger leadership and management of the camp co-ordination cluster – assessed under partnership behaviour.
• • • • • • summary of overall progress IOM has demonstrated a strong commitment to address reforms, driven by its senior leadership.

Since the MAR, key achievements against reform priorities include: • Strategic and performance management: steps taken to improve programme cycle management and focus on results; steps taken to develop a more strategic approach (regional strategies) and share lessons across the organisation (migration crisis operational framework).
For example, IOM has introduced a new project handbook which establishes a standard project cycle and emphasises a results-based approach to project management through monitoring and evaluation.

Financial resources management: concrete steps taken to improve financial planning and humanitarian response capacity (emergency migration funding mechanism) and delivering results as demonstrated by role in Libya and Syria.
More specifically, IOM has rolled out global training on the new IOM project cycle and results based approach.

It is now mandatory that all IOM projects feature a results matrix with indicators to monitor progress.

Partnership: improved leadership of camp coordination cluster through capacity building efforts.

• • Evaluation remains underfunded and long term financial planning will also continue to be a challenge.
153 DRIVING REFORM TO ACHIEVE MULTILATERAL EFFECTIVENESS Mar reform component progress against reform priorities Mar 2011 score progress rating Mar update score, if any change strategic and performance management There is strong leadership from HQ.

Steps have been taken to develop a more strategic approach across the organisation, improve monitoring and lesson learning.

There has been a shift to results based management.

Concrete steps have been taken to improve programme cycle management.
Evaluation remains weak and under-resourced, but performance monitoring is in place.

financial resources management There has been improved humanitarian response capacity through the establishment of a migration emergency funding mechanism.

A strong response capacity has been demonstrated during the Libya and Syria crises.

There is recognition of the need for core budget stability to improve financial planning.

Financial management is taken seriously – zero nominal growth of the admin budget.
Financial capacity in the field (in particular, procurement) is improving.

partnership behaviour There is continued good collaboration with partners.

There is encouraging evidence from the field and HQ on efforts to incorporate the beneficiary voice and improve coordination and camp management cluster capacity.
2 2 2 Some progress Reasonable progress Reasonable progress 154 MAR UPDATE SUMMARY ASSESSMENTS office of the High commissioner for Human rights (oHcHr) Multilateral aid review (Mar) update 2013 progress rating: 2 MAR 2011: Adequate Value for Money for UK Aid progress assessment summary particular progress on strengthening its focus on results, increasing capacity for results management and improving strategic decision- making in developing countries.
Baseline OHCHR is mandated to promote and protect the enjoyment and full realisation by all people of all rights established in the UN Charter and in international human rights laws and treaties.

The Office also leads efforts to integrate a human rights approach within work carried out by UN agencies.

The MAR highlighted several strengths including: • OHCHR plays a critical role in the context of peace and security and is the only multilateral organisation to focus exclusively on human rights.
There is good evidence of OHCHR contributions to human rights work in fragile states and states recovering from conflict.

A financial monitoring committee, regular cost reviews and clear financial thresholds are in place and trigger examinations by directors.

The MAR also highlighted several weaknesses: • • OHCHR needs to do more to demonstrate results in developing countries.
There is a weak results culture, resulting in a lack of standard methodology and a lack of in- house capacity for lesson learning.

There is insufficient evidence to demonstrate that OHCHR has processes in place to manage poorly performing projects/programmes.
• DFID’s reform priorities for the MAR Update were: • Strengthen results reporting and demonstrating results in developing countries; more clearly establish results above level of activities – assessed under contribution to results; Strengthen results based management, strategic planning in countries and evaluation systems – assessed under strategic and performance management; Strengthen processes for better project portfolio management – assessed under financial resources management; Strengthen cost control measures – assessed under cost and value consciousness.
• • • • • summary of overall progress OHCHR has a greater focus and priority on results with results frameworks introduced across country and regional offices and a new online performance monitoring system (PMS) having been put in place ensuring that staff now plan, monitor and report on results.
OHCHR’s leadership has fostered a stronger results culture within the organisation and is using results management to improve strategic decision making in country, whilst measures are being taken to strengthen the ability of the office to effectively identify and manage poorly performing projects.
However, greater progress needs to be made on establishing an evaluation function and, whilst there has also been some progress in strengthening cost and value consciousness, there is not yet a long term co- ordinated efficiency savings plan or aggregated cost savings.
155 DRIVING REFORM TO ACHIEVE MULTILATERAL EFFECTIVENESS Mar reform component progress against reform priorities Mar 2011 score progress rating Mar update score, if any change contribution to results New country and regional results frameworks, linked to the corporate results framework, and development of the new performance monitoring system (PMS) online tool are ensuring a greater focus and priority on results reporting in country.
Baselines need to be added to the frameworks, and impact-level results more clearly demonstrated.

strategic and performance management Implementing results based management training and rolling-out the new PMS tool have strengthened the results culture and improved results management.

More evidence is needed of OHCHR leadership using results generated by the PMS to inform programmatic decisions and a fully-resourced evaluation function needs to be established.
financial resources management Consolidation of the programme and budget review board, the new PMS tool and a new financial reporting system, is increasing OHCHR’s ability to identify and manage poorly performing projects.

More evidence is needed of this translating into improved portfolio management.

cost and value consciousness Despite limitations, additional cost control and efficiency measures have been taken.
More evidence is needed of the cost savings being achieved and of a co-ordinated efficiency savings plan.

2 2 2 2 Reasonable progress Some progress Some progress Some progress 156 MAR UPDATE SUMMARY ASSESSMENTS private infrastructure development Group (pidG) Multilateral aid review (Mar) 2013 update progress rating: 3 MAR 2011: Very good Value for Money for UK Aid progress assessment summary Has continued to innovate and deliver results over past two years.
all three reform priorities addressed and implementation starting to take effect.

undertaking further strengthening of governance structures.

Baseline The Private Infrastructure Development Group (PIDG) aims to facilitate the private sector’s involvement in infrastructure development in order to foster economic growth and reduce poverty.

This is done through a portfolio of facilities that address market failures and institutional barriers.
The MAR highlighted several strengths: • • A valuable position focused on catalysing private investment in infrastructure.

PIDG-supported projects have attracted $10.5 billion of private investment projected to provide new or improved services to over 50 million people.

It has strong performance, transparent trust arrangements governing funding commitments, annual external audits, funding linked to strategies.
• The MAR also highlighted several weaknesses: • The PIDG has made no effort to date to target investments at girls and women or to report using gender disaggregated data.

No formal policy on prioritising fragile states.

• • Weak partnership with civil society and donor country offices including DFID.

Disclosure policy not fully developed and websites could be more informative.
• DFID’s reform priorities for the MAR Update were: • To introduce formal policies for prioritising investments in fragile states – assessed under attention to cross-cutting issues (fragile contexts); To target investments at girls and women and use gender disaggregated data – assessed under attention to cross-cutting issues (gender); To proactively encourage more transparency, including improving accessibility of information – assessed under transparency and accountability.
• • summary of overall progress The PIDG has taken steps to implement the MAR reforms alongside initiating new facilities and improving governance arrangements.
Specific steps taken since the MAR include: independent reviews of strategy and governance; an audit of development impact; strengthening operating policies and procedures; introducing a results monitoring handbook; establishing a chair’s office to strengthen governance; setting fragile contexts targets for all facilities; and increasing transparency through updated websites and introducing IATI compliant reporting.
157 DRIVING REFORM TO ACHIEVE MULTILATERAL EFFECTIVENESS progress against reform priorities Mar reform component Mar 2011 score progress rating Mar update score, if any change attention to cross-cutting issues (fragile contexts) The PIDG has set a formal policy for Fragile and Conflict Affected States which includes a baseline and minimum targets for its facilities.
Although the PIDG has decided to maintain a focus on frontier markets, the higher risks in fragile contexts mean that attracting private investment remains difficult.

attention to cross-cutting issues (gender) PIDG has developed and implemented new methodologies to allow it to monitor and report gender disaggregated data.

It is already publishing this data and has plans in place to improve further the quality of its gender reporting.
transparency and accountability PIDG has taken steps to improve transparency and accountability over the past two years.

This includes commissioning a governance review leading to a clear code of conduct and operating policies and procedures (including a disclosure policy), improved access to documents via the website, an agreed schedule to implement IATI compliant reporting and introducing standard reporting formats for facilities.
Accountability in developing countries including coordination with DFID country offices has improved but is still limited.

2 1 2 Reasonable progress Reasonable progress Some progress other relevant information The PIDG has further strengthened its results and monitoring framework including climate change and environmental sustainability classification amongst other elements.
In 2012, the PIDG announced a new facility – Green Africa Power (GAP) – to mobilise investments in renewable energy across the continent.

GAP will be launched in 2013.

During the course of 2012, the PIDG carried out a strategic review, the recommendations of which if implemented would be expected to strengthen the PIDG’s direction towards working on difficult sectors and countries.

Recommended actions included that the PIDG expand its provision of early stage patient equity and local currency financing.
It also recommended investigating setting up facilities in water and sanitation, low cost housing and agricultural infrastructure.

158 MAR UPDATE SUMMARY ASSESSMENTS united nations children’s fund (unicef) Multilateral aid review (Mar) update 2013 progress rating: 3 MAR 2011: Very Good Value for Money for UK Aid progress assessment summary taking reform seriously, in particular at headquarters level.

improved humanitarian work and better results framework.
stronger attention to value for money, and greater transparency and accountability.

need to improve results reporting and working with un partners in developing countries.

Baseline UNICEF focuses on young child survival and development, basic education and gender equality, HIV/ AIDS and policy advocacy and partnerships for children’s rights.

The MAR highlighted several strengths: • UNICEF has a strong poverty focus and plays a critical role in delivering the MDGs and working with partners.
It has improved delivery in developing countries and demonstrates results in fragile situations and on gender.

Improving cost control and has good financial disclosure policies.

Struggles to show aggregate results at an organisational level.

The MAR also highlighted several weaknesses: • • Weak at deploying the right staff at the right time to acute humanitarian emergencies.

Inconsistent approach to collaborating with other UN agencies and civil society.
• Does not have a transparency policy and full information on projects is not disclosed.
• DFID’s reform priorities for the MAR Update were: • Improved leadership and delivery in humanitarian emergencies – assessed under contribution to results; Strengthened monitoring and reporting of results at the organisational level – assessed under strategic and performance management; Strengthened cost control and value for money – assessed under cost and value consciousness; Greater leadership and development of strategic partnerships in the UN system – assessed under partnership behaviour; Strengthened transparency by joining the IATI and publishing full project information – assessed under transparency and accountability.
• • • • • • summary of overall progress Overall, UNICEF has improved its humanitarian work including its role in coordinating other actors in emergencies.

The results framework for its next strategic plan for 2014-17 has improved.

UNICEF is also demonstrating greater attention to value for money and has signed up to IATI therefore strengthening its transparency and accountability.

Limited evidence of effective contribution to inter- agency coordination in developing countries.
159 DRIVING REFORM TO ACHIEVE MULTILATERAL EFFECTIVENESS Mar reform component progress against reform priorities Mar 2011 score progress rating Mar update score, if any change contribution to results Improved humanitarian coordination and HR.

Improved policies, procedures and guidance.

But the speed of disbursing funds to partners in emergencies remains a challenge.
strategic and performance management Improved results framework for 2014-17 strategic plan, although causal linkages and attribution of results remain areas for improvement.

cost and value consciousness Adopted the International Public Sector Accounting Standards (IPSAS) and fully implemented its Enterprise Resource Planning system (VISION).

Procurement prices and savings/cost efficiencies are made publicly available.
partnership behaviour Publicly emphasised its commitment to Delivering as One and contributed to the post-2015 development agenda.

Not clear it has progressed, in terms of UN coherence and inter-agency coordination in developing countries.

transparency and accountability Signed up to IATI and committed to full implementation by April 2014.

Initial dataset meeting IATI standards published in June 2013.

Evidence of UNICEF proactively encouraging transparency and accountability in delivery partners needed.
3 2 3 3 2 Reasonable progress Some progress Reasonable progress Some progress Reasonable progress 160 MAR UPDATE SUMMARY ASSESSMENTS united nations development programme (undp) Multilateral aid review (Mar) update 2013 progress rating: 2 MAR 2011: Good Value for Money for UK Aid progress assessment summary some progress in key areas, including improvement in performance in developing countries.

a greater rate of progress required by 2015.


The United Nations Development Programme (UNDP) helps countries build and share solutions to achieve Poverty Reduction and the Millennium Development Goals.

Baseline The MAR highlighted several strengths: • • Important role in achieving a number of MDGs and addressing the UK’s strategic priorities.

Strong partnerships across the UN system and with partner governments.

Able to incorporate beneficiary voice.


Committed to IATI and has good member state accountability.

• The MAR also highlighted several weaknesses: • Delivery results in developing countries can be undermined by staffing issues and bureaucratic processes; and its performance in fragile contexts is mixed.

Broad mandate means its technical resources are spread very thinly and lack strategic direction.

Limited evidence of cost and value consciousness within senior management and in country offices.
DFID’s reform priorities assessed for the MAR Update were: • Demonstrate delivery of results in developing countries, particularly in fragile and conflict- affected countries and leadership in the UN system – assessed under contribution to results; Greater focus on governance and crisis prevention and recovery, and progress on key management reforms – assessed under strategic and performance management; Progress on procurement driven by value for money, cost effectiveness and administrative efficiency – assessed under cost and value consciousness.
• • • • summary of overall progress UNDP has made some progress in improving strategic direction, corporate planning and leading the UN system.

However, stronger evidence of improvements in developing countries is needed.

UNDP is developing a much stronger corporate results framework, has achieved greater value for money in procurement and has reduced recruitment times for key country programme posts.
More progress is needed to improve administrative efficiency, the quality of staffing, particularly in fragile states, results reporting capacity and deliver a more explicit approach to cost control and effectiveness.
161 DRIVING REFORM TO ACHIEVE MULTILATERAL EFFECTIVENESS Mar reform component progress against reform priorities Mar 2011 score progress rating Mar update score, if any change contribution to results Progress in country office performance, particularly leadership and programme effectiveness, though difficult to judge how much.

Good leadership on the resident coordinator system and IATI.
strategic and performance management Improved strategic direction, corporate planning, development of a stronger results framework and improved recruitment times.

Further action required to improve staff performance and recruitment to conflict-affected countries.

Insufficient attention to capacity for results management.

cost and value consciousness Good progress on procurement, but clearer targets required for cost control/effectiveness and administrative efficiency.
2 2 2 Some progress Some progress Some progress 162 MAR UPDATE SUMMARY ASSESSMENTS united nations educational, scientific and cultural organization (unesco) Multilateral aid review (Mar) 2013 update progress rating: 3 MAR 2011: Poor Value for Money for UK Aid progress assessment summary improvements made in work planning, budgeting, Hr policies, transparency and cost control, which have strengthened organisational effectiveness.

will need to keep up pace of change to consolidate work done.
Baseline UNESCO is a UN specialised agency that provides a forum for the negotiation of global agreements, a facility for policy exchanges and capacity building, and a centre for standard setting and monitoring in education, natural sciences, social sciences, culture and communication and information.

The MAR highlighted several strengths: • • • UNESCO has an important role in education policy and reporting.

Accountability is good with systematic involvement of partner countries and NGOs.
UNESCO’s external audits are high quality and it is implementing IPSAS (International Public Sector Accounting Standards).

The MAR also highlighted the following weaknesses: • The programme information system is complicated and inadequate attention is paid to results in programming decisions.

Financial resources management is inadequate, in particular allocation mechanisms and management of poorly performing programmes.
• DFID’s reform priorities for the MAR Update were: • • Improving results-based management – as assessed under contribution to results; Focusing its resources where UNESCO has a comparative advantage – as assessed under strategic and performance management; Strengthening its focus on value for money and cost consciousness and continuing to reduce administrative overhead costs – as assessed under cost and value consciousness; Improving its transparency, particularly in programmes – as assessed under transparency and accountability.
• • summary of overall progress UNESCO is making reasonable progress on both contribution to results and strategic and performance management.

It has improved its results based management, focused more on its comparative advantage and made a step change in implementation of evaluation recommendations.

The confidence of members and other UN organisations in UNESCO is increasing.
163 DRIVING REFORM TO ACHIEVE MULTILATERAL EFFECTIVENESS Mar reform component progress against reform priorities Mar 2011 score progress rating Mar update score, if any change 2 2 Reasonable progress Reasonable progress Reasonable progress Some progress 1 1 2 2 contribution to results Across the board, improvements in results based management include: (a) increased effort to explain how standard setting work achieves change on the ground and to address methodological issues around evaluation of normative work; (b) improvements in its systems to ensure activities in developing countries are results-focused (country planning documents coverage up by 117%); (c) improvements in its results reporting (annual report, first sector report).
In addition, UNESCO is convening a high-panel group which is drafting a successor to the 1996 Delors report on the four pillars of education, setting out new educational goals for the 2015 post MDG agenda, demonstrating significant international partnerships (Education First leadership), and strengthening its political advocacy (stand up for Malala).
strategic and performance management Improved focus has been demonstrated by the significant reduction in ‘small’ activities currently undertaken, together with a reduced overall set of strategic objectives for the organisation going forward.

Through the significant increase in implementation of evaluation and audit recommendations, the organisation is demonstrating an improved, evidence and performance based approach to programme management.
cost and value consciousness UNESCO has achieved continuous savings in its administration overhead.

The ratio of costs of central services overall was able to be improved slightly despite the significant cuts in programmes forced by some members failure to pay contributions.

The reform of administrative processes and procedures has continued and several new time-saving systems are now in use.
The proposed budget for 2014-17 makes the central service reductions permanent whether or not all members resume their contributions.

transparency and accountability The organisation has adopted a financial disclosure policy which supplements already extensive publicly available information.

In addition, the organisation has made a formal commitment to publish IATI compliant programme information by 2015 through a common UN process and format.
Proposed reductions to the operational budgets for the Ethics Office and Internal Oversight Service put at risk their good work.

It is too early to tell whether sufficient progress in transparency and accountability will be made to lead to a score improvement in 2015.
164 MAR UPDATE SUMMARY ASSESSMENTS united nations High commission for refugees (unHcr) Multilateral aid review (Mar) 2013 update progress rating: 2 MAR 2011: Good Value for Money for UK Aid progress assessment summary taking reform seriously.

committed to improving partnerships with humanitarian actors and national authorities but mixed impact at field level.

transparency and accountability improved, good information sharing, including on results.
UNHCR provides international protection and humanitarian assistance to refugees and, in some situations, internally displaced people, while working to find durable solutions to their situation.

Baseline The MAR highlighted several strengths: • UNHCR fulfils a critical role in the international humanitarian architecture, particularly in conflict situations.

Staff are well equipped to work in conflict/fragile contexts.
Policies, structures and incentives promote gender equality and gender is mainstreamed at country level.

Financial accountability is strong, with a range of internal and external oversight processes.

Beneficiary voice is incorporated into policies and programmes.

• • • • The MAR also highlighted several weaknesses: • Despite the fact that UNHCR is a cluster lead, they sometimes fail to provide adequate international leadership.
• While UNHCR has improved considerably, the agency still needs to actively participate in the reformed humanitarian leadership, coordination and financing systems.

Some country offices are not yet thinking about value for money or cost control.
• DFID’s reform priorities for the MAR Update were: • Improved partnership behaviour, in particular cluster leadership, needs assessments, support to the Humanitarian Coordinator (HC) system and effective financial arrangements with implementing partners – assessed under partnership behaviour; • More systematic publishing of programme documentation to improve transparency and accountability; developing a formal disclosure policy – assessed under transparency and accountability.
summary of overall progress Some progress has been made.

Actions at headquarters demonstrate a willingness to change in order to address the weaknesses highlighted in the MAR.

At an operational level, the full impact of these reforms is not evident.

Key achievements to date include: • • The publication of a formal disclosure policy; Increased transparency as a result of a new system to share programme information – ‘Global Focus’; Stronger culture of results based management.
• Key challenges remain: • Steps have been taken to improve partnership behaviour but there is a lack of consistency.

Donors and NGOs continue to highlight this as an area where UNHCR should continue to focus, in particular partnership with implementing partners.
165 DRIVING REFORM TO ACHIEVE MULTILATERAL EFFECTIVENESS Mar reform component progress against reform priorities Mar 2011 score progress rating Mar update score, if any change partnership behaviour Some concrete steps have been made to improve performance.

This should remain a priority.

In future years, we would wish to see increased evidence of UNHCR working well in developing countries to support the international system.
transparency and accountability UNHCR has addressed issues highlighted in the MAR (lack of a disclosure policy and a need to publish documentation).

It has continued to move towards a stronger culture of results based management and corporate reporting with further improvement expected in future years.
2 2 Some progress Reasonable progress 166 MAR UPDATE SUMMARY ASSESSMENTS unitaid Multilateral aid review (Mar) 2013 update progress rating: 2 MAR 2011: Good Value for Money for UK Aid progress assessment summary the pace of progress has been reasonable in most areas though slow in others.

the majority of current uK reform priorities have been partially achieved, albeit slowly.
Baseline UNITAID is a unique organisation seeking to have an impact on markets for drugs, diagnostics and preventive interventions in the field of HIV/AIDS, tuberculosis (TB) and malaria.

It seeks to decrease prices and increase access, and to act as a catalyst for new, innovative treatments and formulations.

The MAR highlighted several strengths: • UNITAID’s focus on malaria and contribution to reproductive, maternal and newborn health (RMNH) means that it has a good fit with DFID’s strategic priorities.
Price reductions have been significant, and should lead to a sustainable benefit for countries, donors and international agencies.

The views of partners and intended beneficiaries are built into UNITAID’s decision making structures.

Financial management has improved with the recruitment of high quality senior personnel.

The MAR also highlighted several weaknesses: • • • There is little evidence that management actively manages for results.
The board has applied value for money criteria unevenly in funding decisions.

UNITAID’s publication of documentation is patchy and often very slow.
DFID’s reform priorities for the MAR Update were: • Improvements in UNITAID’s strategy and business model, a coherent funding strategy, stronger leadership and management capacity delivering an efficient and well-led secretariat – assessed under strategic and performance management; A credible and functional policy on transition of grants away from UNITAID funding and more predictable and reliable donor support – assessed under financial and resources management; Development of internal systems, improved arrangements for appraising and prioritising proposals that offer the greatest value for money – assessed under cost and value consciousness and transparency and accountability.
• • • • • summary of overall progress Overall, progress has been reasonable against important components such as financial and resource management – including risk management – and there has been some progress on cost and value consciousness.

Systems for financial accountability have improved.

Overall systems and policies have improved, though the board itself can still be inconsistent, for example in the way it reaches funding decisions.
The recently approved strategy 2013-16 is an improvement over the previous strategy 2010-12.

The independent Five Year Evaluation (5YE) is positive about many aspects of UNITAID’s performance, while noting that challenges remain.
167 DRIVING REFORM TO ACHIEVE MULTILATERAL EFFECTIVENESS Mar reform component progress against reform priorities Mar 2011 score progress rating Mar update score, if any change strategic and performance management The Five Year Evaluation of UNITAID validates the business model and notes that it is ‘doing the right things in the right way’.

Both the board and secretariat are using more robust tools in appraising projects, and the new strategy will be useful in defining funding priorities.
Leadership has improved.

Despite some challenges highlighted, the pace of reform is reasonable.

financial resources management Good progress has been made in terms of developing and applying internal systems and policies.

Problems of transitioning grants to other sources of funding have been largely resolved, though challenges remain in other areas such as portfolio management.

Tools to support prioritisation have improved, but donor support is not yet predictable.
cost and value consciousness There has been some continued progress from an already good base, as the development and use of internal processes and measurement have improved.

The 5YE was positive about this aspect of UNITAID’s performance, but the board could be stronger on seeking best value for money.
transparency and accountability There has been some progress with an overhaul of the website, and participation of stakeholders remains strong, but the publication of project documentation in a consistent and easily accessible manner could still improve.

The pace of progress in this component has been slow.
2 2 3 2 Reasonable progress Reasonable progress Some progress Little or no progress 168 MAR UPDATE SUMMARY ASSESSMENTS un office for the coordination of Humanitarian affairs (ocHa) Multilateral aid review (Mar) 2013 update progress rating: 2 MAR 2011: Good Value for Money for UK Aid progress assessment summary taking reform seriously.

instigated procedures to improve strategic performance and improve collective results delivered by humanitarian system.

reforms yet to take effect at field level.


further work required to ensure commitment translates into action.

OCHA plays a central role in delivering international humanitarian response.

Baseline The MAR highlighted several strengths: • OCHA has a clear and unique humanitarian mandate, global reach and a pivotal leadership role.

It managed a range of pooled funding mechanisms that supported joined-up working amongst actors and fostered strong partnerships.
It has a strong network of partners at both headquarters and in the field.

The MAR also highlighted several weaknesses: • OCHA needed to become more reliable in fulfilling its leadership, coordination, information management and advocacy role at field level.

It has reasonable levels of transparency, but a weak accountability structure.

It has staffing problems, hampering its ability to deliver.

It needed to focus on value for money as well as cost control.
DFID’s reform priorities for the MAR Update were: • Improving operational coordination through effective inter-cluster coordination and better information management – assessed under contribution to results; Strengthening leadership across the humanitarian system, particularly in relation to humanitarian coordinators – assessed under strategic and performance management; Consistent field capacity, improved through good human resourcing – assessed under strategic and performance management; Improving resource allocation and links between needs and appeals and a well-monitored humanitarian response – assessed under financial resources management.
• • • • • • • • summary of overall progress Evidence gathered since the MAR indicates that OCHA is committed to organisational change and motivated to achieve reform.

However, evidence of the impact of reforms at field level is inconsistent.
Examples of achievements include: • • Strong leadership for reform provided by senior management; Improved staff planning – although at field level gaps remain; better internal surge capacity and a renewed commitment to staff training and career development; A focus on cost-consciousness, value for money and delivering results at all levels; A commitment to increase lesson learning from evaluations and audits and improved accountability and transparency; Expanded resource to manage the Humanitarian Coordinator (HC) pool and better performance management, training and mentoring; Tools and systems to strengthen evidence-based decision making, transparency and accountability within the humanitarian system.
• • • • However, further work is required to ensure OCHA: • • • Improves results (delivery) through better information management and inter-cluster coordination, supported by strong humanitarian leadership; Achieves consistent field capacity – getting the right people in the right place at the right time; Has led the development of more strategic prioritised CAPs, informed by robust needs assessments which act as the basis to evaluate the impact of the international humanitarian response.
169 DRIVING REFORM TO ACHIEVE MULTILATERAL EFFECTIVENESS Mar reform component progress against reform priorities Mar 2011 score progress rating Mar update score, if any change contribution to results There has been progress at the global level but some OCHA country offices continue to face major challenges in delivering results.

There has been mixed progress against the specific reform priority related to improving information management and inter-cluster coordination.
strategic and performance management There has been reasonable progress towards its objectives.

Examples include: an improvement in the timeliness of deployments; reduced vacancy rates; joint assessments for all new emergencies; better use of the humanitarian coordinator pool.
financial resources management OCHA has made some progress, including: the launch of country/regional office standardised performance frameworks; commitment to address poor performing programmes; more realistic and disciplined budgeting and concessions from the UN secretariat.
2 2 2 Some progress Reasonable progress Some progress 170 MAR UPDATE SUMMARY ASSESSMENTS • • • • • united nations peacebuilding fund (pBf) Multilateral aid review (Mar) 2013 update progress rating: 3 MAR 2011: Good Value for Money for UK Aid progress assessment summary taking reform seriously.

instigated procedures to improve performance and focus on results.

taking steps to increase value for money.

some measures have taken effect in developing countries.
Baseline The Peacebuilding Fund (PBF) is a central, system-wide UN fund that fills critical funding gaps and supports catalytic peacebuilding interventions.

The MAR highlighted several strengths: • The fund fills a crucial gap in the international system’s response to conflict and the PBF’s interventions are strongly in line with major UK priorities, including those in the Strategic Defence and Security Review.

The PBF is a flexible, predictable funding source.


There is national government and civil society involvement in developing countries.

The MAR also highlighted several weaknesses: • • • • Conflict analysis and reporting were not systematic.

There was evidence of low performing projects and weak management.

There were difficulties in measuring results.

Value for money and cost control were not as strong as DFID would like.
DFID’s reform priorities for the MAR Update were: • Securing effective results based management and reporting on outputs and outcomes at a country and organisation level – assessed under contribution to results; Reporting demonstrating catalytic contributions to peacebuilding, and improved and consistent delivery in developing countries – assessed under contribution to results; Flexibility and risk management allowing the PBF to deliver against strategic peacebuilding objectives – assessed under cost and value consciousness; Country systems21 and conflict-sensitive review criteria being used and supported consistently – assessed under partnership behaviour.
These reform priorities are largely focused on the ‘contribution to results’ component of the MAR assessment framework but weaknesses also exist across the ‘cost and value consciousness’ and ‘partnership behaviour’ components, and therefore these components have also been assessed.

summary of overall progress Evidence indicates that the PBF is taking the need for change seriously and is working to address the MAR reform priorities.
In the past year, the PBF has instigated a number of initiatives to improve its overall performance in line with DFID’s expectations.

The quality of work undertaken to deliver organisational reforms is high.

In particular: • • Strong steps have been taken to improve reporting at a global level.

New procedures have been designed to improve reporting in developing countries and will be rolled out in the coming year.
New guidelines have been drafted that should require UN organisations and partners to more strongly consider value for money and strategic contributions to peacebuilding.

PBF staff have adopted a more hands-on approach to influencing programme and project design including looking at cost effectiveness.

The PBF has identified the need to urgently strengthen the joint steering committees (upon which so much of the PBF’s mandate depends) but is still deciding how this priority will be taken forward.
• • • 21 Refers to existing mechanisms for coordinating and prioritising international support to peacebuilding in a given country.

171 DRIVING REFORM TO ACHIEVE MULTILATERAL EFFECTIVENESS Mar reform component progress against reform priorities Mar 2011 score progress rating Mar update score, if any change contribution to results PBF initiated reforms to reporting and developed new templates and procedures.
A new performance management plan exists and stronger focus on results based management, with a stronger focus on outcomes.

cost and value consciousness To the extent that it is able, the PBF is maximising its leverage of recipient UN organisations and partners to influence procurement and spending decisions.

They are exercising greater oversight and have instigated new procedures that will be rolled out in the coming year.
partnership behaviour PBF continues to work hard to convene the World Bank, the UN and national stakeholders around peacebuilding.

Success is limited by weak fund management structures at national level.
2 2 3 Reasonable progress Reasonable progress Some progress 172 MAR UPDATE SUMMARY ASSESSMENTS united nations population fund (unfpa) Multilateral aid review (Mar) update 2013 progress rating: 2 MAR 2011: Adequate Value for Money for UK Aid progress assessment summary some progress across the reform areas including a better results framework, updated evaluation function and implementation of iati and international public sector accounting standards (ipsas), as well as evidence of cost reductions.
Baseline UNFPA leads on sexual and reproductive health and rights and on supporting countries use population data for policies and programmes to reduce poverty.

The MAR highlighted several strengths: • • • Has a critical role through global level advocacy and delivery of related MDGs.

Reports comprehensively against its global objectives.

Its procurement strategy considers value for money related principles.
The MAR also highlighted several weaknesses: • • • • High administration costs, procurement savings not systematically tracked or reported.

Evaluation culture and global level results chains are weak.

Audit concerns have been made a priority but progress on this is slow.

Accountability to partner governments is strong, but transparency of programme information is weak.
DFID’s reform priorities for the MAR Update were: • Improved delivery of priority programmes and assisting governments in utilising population data – assessed under contribution to results; Strengthened reporting frameworks and results culture within the organisation – assessed under strategic and performance management; Reduction in the number of outstanding audit recommendations and improved compliance with financial procedures – assessed under financial resources management; Administrative efficiency savings and improved procurement cost control – assessed under cost and value consciousness; Increased transparency of project documentation and data, and implementation of IATI – assessed under transparency and accountability.
• • • • summary of overall progress UNFPA has improved its integrated results framework, implemented the IPSAS and IATI, established a new evaluation function, reduced audit recommendations and continues to focus on its management-to-programme spend ratio.
173 DRIVING REFORM TO ACHIEVE MULTILATERAL EFFECTIVENESS Mar reform component progress against reform priorities Mar 2011 score progress rating Mar update score, if any change contribution to results There are some moves to improve central oversight, evident in the improved global results framework and country programme reviews.

However variable country performance remains a feature of UNFPA’s operations.

More needs to be done in the humanitarian field.


strategic and performance management UNFPA has improved its results and evaluation tools, but more time is needed to see if this results in a strengthened culture.

financial resources management Implemented the international public sector accounting standards and received an unqualified financial audit.

However concerns remain about overall compliance across the organisation.
cost and value consciousness UNFPA has improved its procurement office, and has demonstrated its prioritisation of programme over management costs.

More needed to incentivise country offices to reduce costs.

transparency and accountability UNFPA has started implementation of IATI, and widened its publically disclosed documents to include internal audit.

However its documentation about work in developing countries is still not easily available.
2 2 2 2 2 Some progress Some progress Some progress Some progress Some progress 174 MAR UPDATE SUMMARY ASSESSMENTS Joint united nations programme on Hiv/aids (unaids) Multilateral aid review (Mar) update 2013 progress rating: 3 MAR 2011: Adequate Value for Money for UK Aid progress assessment summary strong progress on administration cost savings, progress on financial resources management and on explaining how they achieve results, but need to ensure such results are shared more widely.
progress was sufficient to improve the overall value for money assessment to “good”.

The Joint United Nations Programme on HIV/AIDS (UNAIDS) is charged with delivering an effective response to the global HIV/AIDS pandemic.

Its joint programme coordinates the efforts of 11 UN agencies (called co-sponsors) that work on HIV/AIDS.

Baseline The MAR highlighted several strengths: • Significant contribution to facilitating progress on HIV/AIDS at the global level, including meeting the HIV/AIDS relevant MDGs.
Strong gender focus including internal policies and use of evidence to inform policy and programme decisions.

Strong partnership behaviour performance.

The MAR 2011 also highlighted several weaknesses: • • Struggles to show consistent results in developing countries.

Inadequate results framework and lack of clarity and authority which weakens leadership in developing countries.

Existence of two financial systems complicated management of poorly performing projects through co-sponsors.
• DFID’s reform priorities for the MAR Update were: • A stronger contribution to better local division of labour and joint planning and implementation – assessed under contribution to results; A strengthened results framework with clearer links between results at output and outcome levels, and clear indicators to measure success – assessed under strategic and performance management; The establishment of one administration system – assessed under financial resources management; Improved cost control and clearer evidence that UNAIDS Secretariat challenges its co-sponsors to achieve value for money at all levels – assessed under cost and value consciousness.
• • • • • summary of overall progress Given the progress made by UNAIDS to address the reforms, in particular the improvement of the score for cost and value consciousness, the overall value for money rating will change from adequate to good.
175 DRIVING REFORM TO ACHIEVE MULTILATERAL EFFECTIVENESS Mar reform component progress against reform priorities Mar 2011 score progress rating Mar update score, if any change contribution to results Improved its results in country by aligning its human resources and job profiles with its strategic priorities and redeploying 80 staff in key countries.

However, UNAIDS has not gone far enough in prioritising resources to the countries with the most cases/ higher rates of HIV.
strategic and performance management UNAIDS improved the results framework which monitors the combined results of the cosponsors and added a clearer results chain UNAIDS has developed a web-based tool, which was used in 2012 to collect the results of the 11 co-sponsors for the first time.

financial resources management UNAIDS has moved to one administration system, which has brought considerable benefits in terms of both staff and financial management.
cost and value consciousness UNAIDS has made 13% efficiency savings on administration costs between 2011 and 2012.

The level of savings exceed our expectations and evidence suggests that UNAIDS has made the changes in a strategic way.

The strategic investment framework and the tool developed by UNAIDS entitled ‘Investing for Results.

Results for People’ help guide investment priorities in countries that are cost-effective, efficient and produce maximum impact.
2 2 2 2 Reasonable progress Reasonable progress Reasonable progress Significant progress 3 176 MAR UPDATE SUMMARY ASSESSMENTS un woMen Multilateral aid review update (Mar) 2013 progress rating: 3 MAR 2011: Not Assessed.

UNIFEM (predecessor) was Poor Value for Money for UK aid progress assessment summary un women has taken reform seriously and has shown commitment to addressing its reform priorities, especially in relation to contribution to results.
Baseline UN Women’s role is to improve the impact of the UN system’s collective response on gender equality and women’s empowerment and to set global norms, policies and standards on gender equality.

The 2011 MAR did not assess UN Women because it did not become operational until January 2011.
The organisation was introduced to overcome the lack of coordinated and coherent support by the UN to member states to advance gender equality and women’s empowerment, and under- resourcing of gender equality in the UN system.

The organisation resulted from the merger of four UN organisations which previously worked on gender.

Of those, UNIFEM was the largest and its staff made up 86% of UN Women’s workforce when the new agency was created.
UNIFEM’s mandate was also very similar to that of the newly created UN Women.

UNIFEM was found to be poor value for money in the MAR 2011.

The challenges identified in the UNIFEM MAR assessment were agreed as areas we wanted to see addressed in the new organisation, UN Women.
DFID’s reform priorities for the MAR Update were: • Evidence of providing technical assistance and policy support to deliver on its strategic priorities and a stronger collective UN country team effort – assessed under contribution to results; Development of a well-defined results framework and monitoring and reporting of results at the country and HQ level – assessed under strategic and performance management; Implementation of IATI and clear disclosure policies – assessed under transparency and accountability; Evidence of a focus on cost control and reporting against efficiency targets – assessed under cost and value consciousness.
• • • summary of overall progress Special consideration has been given to UN Women’s situation as a new organisation facing legacy issues.

Reasonable progress has been made: UN Women’s reform to give more authority to regional and country offices should improve performance in developing countries; and systems are in place to strengthen its results focus and evaluation culture.
It has improved its cost and value consciousness and has set up systems to promote internal and external transparency and accountability.
177 DRIVING REFORM TO ACHIEVE MULTILATERAL EFFECTIVENESS unwoMen Mar update score, if any change 2 Mar reform component progress against reform priorities unifeM Mar 2011 score unwoMen progress rating contribution to results Evidence of better management of country offices leading to better results; evidence of stronger support to strengthening global commitments for gender equality, women’s rights and women’s empowerment and coordination role (including the adoption of the UN system wide action plan on gender).
strategic and performance management Stronger results framework leading to more robust reporting and adoption of an evaluation policy.

cost and value consciousness Use of long term agreements to achieve volume discounts in procurement and systems in place to control cost at HQ and country offices.

transparency and accountability Signatory to IATI.

Disclosure policy in place and evidence it promotes accountability and transparency with its delivery partners.
1 2 2 2 Reasonable progress Reasonable progress Some progress Reasonable progress 178 MAR UPDATE SUMMARY ASSESSMENTS world food programme (wfp) Multilateral aid review (Mar) 2013 update progress rating: 3 MAR 2011: Good Value for Money for UK Aid progress assessment summary reasonable progress.

demonstrates a strong commitment at headquarters to reforms, increased focus on results, and commitment to greater transparency.
needs to consider added value as it shifts from food aid to broader range of programmes.

Baseline WFP is the world’s largest humanitarian organisation with two-thirds of its resources channelled through humanitarian interventions.

It is a voluntarily funded organisation and the scale of its operations is driven by fluctuating humanitarian need and funding.
The MAR highlighted several strengths: • WFP has an ability to deliver emergency food assistance at scale in difficult and often dangerous environments.

It provides essential logistic support to the humanitarian system.

It has strong financial management systems.

There is a strong corporate emphasis on cost efficiencies, coupled with business model improvements.

It drives cost control with delivery partners and seeks value for money through local procurement.
It demonstrates good partnership behaviours.

• • • • The MAR also highlighted several weaknesses: • WFP needs to achieve a results-based culture across the organisation.

• Its programme tools do not always represent value for money in comparison with non-food alternatives.

Accountability to partners and beneficiaries remains weak.

Value for money and cost control are not as strong as they should be.
• • DFID’s reform priorities for the MAR Update were: • Continuing to focus on improving its emergency response capacity – assessed under critical role in meeting development objectives; Strengthening its performance on results and value for money – including improving programme design, appraisal, monitoring and reporting – assessed under contribution to results; Becoming a more strategic player in transition contexts, building capacity at regional and on disaster preparedness and risk management in developing countries – assessed under strategic and performance management; Improving transparency and accountability to beneficiaries – assessed under transparency and accountability.
• • • summary of overall progress WFP is committed to addressing the MAR reforms and has taken a number of positive steps at headquarters, which is in line with expectations.

WFP’s new Executive Director is driving forward many of these reforms, which gives confidence that there will be significant progress in the coming years.
Specific improvements since the MAR include: • Initiatives to improve the impact of WFP’s work in transition and development settings: increased focus on preparedness, resilience and under-nutrition; Improved emergency response capacity: established new mechanisms to decrease response times, streamlined decision making and increased staff capabilities; Improved results reporting and strong commitment from management to embed a results culture across the organisation; A willingness to pilot new and innovative approaches including using geographic information systems (GIS) to provide updates on logistics routes, camps and food security; A commitment to greater transparency through signing up to the IATI; A stronger commitment to risk management: a new corporate risk register in place and new risk management systems piloted in developing countries.
• • • • • 179 DRIVING REFORM TO ACHIEVE MULTILATERAL EFFECTIVENESS Mar component progress against reform priorities Mar 2011 score progress rating Mar update score, if any change critical role in meeting development objectives WFP has continued to play an important role in humanitarian response and begun to improve its work in more stable settings.

Investments have been made in policies and guidance.
contribution to results It has improved reporting and new approaches to improve delivery of assistance, for example the forward purchasing facility.

It is actively working to improve the more challenging parts of its portfolio and address problem areas.

strategic and performance management There is increased focus on risk, results and evaluation.

Whilst the impacts are being felt in some countries, in others these have yet to be realised.
transparency and accountability There has been reasonable progress on transparency.

Accountability to stakeholders remains strong at board level; initiatives are underway to improve accountability in the field.

We expect to see consistent evidence of impact in future.
4 3 2 2 Reasonable progress Reasonable progress Reasonable progress Reasonable progress 180 MAR UPDATE SUMMARY ASSESSMENTS world Health organization (wHo) Multilateral aid review (Mar) update 2013 progress rating: 3 MAR 2011: Adequate Value for Money for UK Aid progress assessment summary progress made in explaining how its work can deliver results, improved financial systems and clear evidence of cost cutting.

More to be done on demonstrating results in developing countries.
Baseline WHO is the directing and co-ordinating authority for health within the United Nations.

Its mission is the attainment of the highest possible level of health for all.

The MAR highlighted several strengths: • Provides global leadership on development and humanitarian health matters and is critical to the delivery of health related MDGs (4, 5 and 6).

Contributes to UK objectives on global health, development and human security.
Partners are well represented in governance mechanisms and policy and guidance are accessible on its website.

• • The MAR also highlighted several weaknesses: • • • There is no clear results chain or formal systems to follow up on evaluation recommendations.

There is no clear and transparent system to allocate aid.

Targets for savings on administration costs are not stretching and there is little attention to cost saving in developing countries.
DFID’s reform priorities for the MAR Update were: • Introduction of a strengthened results chain and framework which should include indicators, baseline and targets at all levels – assessed under contribution to results; • More focused approach to what WHO delivers and improved performance management systems – assessed under strategic and performance management; Strengthened accountability framework providing transparent reporting of allocation and monitoring of finances, particularly in developing countries – assessed under financial resources management; Improved systems to facilitate efficiency savings – assessed under cost and value consciousness.
• • summary of overall progress Progress has been made by WHO to address the reforms, in particular to ensure a clearer results chain and strengthened financial systems and has made significant cost savings.

More still needs to be done to ensure these reforms are rolled out to the regional and country levels of WHO.
181 DRIVING REFORM TO ACHIEVE MULTILATERAL EFFECTIVENESS Mar reform component progress against reform priorities Mar 2011 score progress rating Mar update score, if any change contribution to results New work programme and budget articulate the theory of change and indicate a clear results chain that links WHO’s activities to health and development outcomes nationally, regionally and globally.

Indicators, baselines and targets have also been introduced.
However, evidence remains limited to show progress on results in developing countries.

strategic and performance management Some progress made on focusing activities and clarifying its mandate.

However, progress is slower on improving performance management systems in the organisation, especially staff performance management and culture.

financial resources management The global management system now gives greater access to information about resource use and implementation of programmes.
WHO has also fully implemented International Public Sector Accounting Standards (IPSAS) and strengthened its internal control framework and audit capacity.

cost and value consciousness WHO has improved efficiency and reduced costs by relocating many of its administrative functions to low cost locations, such as Kuala Lumpur, Malaysia.
2 2 2 2 Reasonable progress Some progress Reasonable progress Reasonable progress 182 MAR UPDATE SUMMARY ASSESSMENTS annex 4 Commentary from Lawrence Haddad and Alison Evans, the external reviewers to the MAR and MAR Update As with the original MAR, as external reviewers we were involved in commenting on all phases of the Update: (a) the Update methodology; (b) moderation of scores and (c) the interim and final Update reports.
Here we provide some reflections on the process: Update methodology We noted that the purpose of the Update methodology was to assess progress against reform objectives identified in the original MAR assessments.

The purpose was not to re-assess original MAR scores.

As such we considered the methodology fit for purpose.

We did note, however, some challenges in implementing the methodology, particularly in being crystal clear about the thresholds between progress ratings (i.e.
between some and reasonable progress and between reasonable and significant progress) and the importance of establishing a consistent rule for both aggregating ratings and for justifying any component score changes that resulted.

All these issues were discussed extensively with DFID and we were satisfied with their response and their willingness to take appropriate action as they refined the methodology.

Moderation of scores and follow-up Moderation was undertaken in three separate tranches.
Each tranche considered a cluster of multilateral organisations (MOs).

Extensive internal moderation and consistency checking took place in advance of all three meetings.

Both reviewers read all the assessments in full.

We also read all the moderation notes.

There were no conflicts of interest declared by the reviewers.

The quality of the Update assessments was high overall with considerable efforts made to gather a range of data and to triangulate data wherever possible.
We did note, however, the lack of independent evidence (evaluations, independent reviews) for quite a few MOs and the unevenness with which some of the data from country partners was sought and presented.

We appreciated the DFID assessments of strength of evidence presented for each MO in each reform component.

We discussed the need to distinguish data on country-level impacts from more general data gathered from country partners, something which was followed up on by the MAR Update team.
183 DRIVING REFORM TO ACHIEVE MULTILATERAL EFFECTIVENESS AnnexexTeRnAL ReVIeWeRS' COMMenTARY We were able to ask for clarification on how certain pieces of evidence were used to justify progress ratings, and noted where we felt there were inconsistencies.

In some instances the lack of sufficient evidence was itself an important factor in the assessment.

In all cases our challenge and our input were taken seriously.
In several cases there was so much evidence presented it was difficult to get a sense of its relative importance in driving the progress rating.

Our comment here was taken on board and clarifications promised in the final assessments.

After each tranche discussion, checks were carried out to ensure that any ratings changes arising from the external moderation process were done fairly and consistently.
Of the 146 component progress ratings moderated, 27 were changed at the final stage of moderation, which we were part of.

Seven score changes were made.

While the Update was not as ambitious in scope as the original MAR, it was still a major undertaking.

We note once again the high degree of transparency, accountability and sound judgement with which DFID staff approached the whole process.

All the documentation was received in good time.


Our views as external reviewers were taken seriously and the responses were generally adequate.

Where we felt strongly that additional material needed to be provided to justify a rating, more work was done.

It is not yet fully clear on whether an Update similar to this will be undertaken in the future, but if it were, we felt that the following would need to be addressed: 1.

Have the MAR and the Update exercises provided good value for money?
We feel this is probably the case, but we are close to the process and our views are not based on a systematic review process.

We feel the commissioning of a light touch independent review might promote learning for the next MAR and might give some guidance as to the level of effort allocated to it.

2.

The input of country offices was, we felt, unnecessarily tentative.
We understand the need for anonymity, but we felt that allowing country offices to volunteer information on whichever MO they felt like commenting on (if our understanding is correct) is insufficiently rigorous.

3.

While we were impressed by the thoroughness of the assessments the sheer volume of evidence was sometimes overwhelming and made it difficult to trace the mapping onto the progress rating.
We felt that less might be more here and that a focus on the evidence that really counted would have helped the read across to ratings.

4.

The original MAR clustered MOs around functional type.

This was essential because of the need to establish a set of baselines at roughly the same point in time.
For various reasons the three tranches of MOs in the progress rating sessions mixed up MO functionality and we feel that this made it harder to ensure fairness and equal treatment (which we feel was, nevertheless, achieved).

184 DRIVING REFORM TO ACHIEVE MULTILATERAL EFFECTIVENESS We appreciate the opportunity to provide some public reflections on the process and we commend DFID for the professionalism with which they have undertaken a complex and difficult task.
We were glad to have played a small part in the process.

Dr Alison evans Independent consultant (previously director of the ODI) Prof Lawrence Haddad IDS 185 Front cover images, from top left to bottom right: DFID/Vicki Francis (Pakistan, 2012).

DFID/Pete Lewis (Ethiopia, 2011).

WFP/Dina El Kassaby (Syrian Refugees find safety and food in Iraq, 2013).

DFID/Sam Thompson (Rwanda, 2012).

DFID/Simon Davis (Ethiopia, 2013).

DFID/Ed Hawkesworth (Tanzania, 2013).