Table A- 8: Summary of evaluation ratings for the Niger-Nigeria test case Criterion Sub-criterion Additional Sub-criterion rating Overall criterion rating Attainment of objectives and planned results Achievement of outputs and activities NA MS MS Relevance S Effectiveness MS Efficiency MS Outcomes to impacts MU Sustainability and catalytic role Sustainability Socio-political sustainability L ML Financial resources ML Institutional framework ML Environmental sustainability L Catalytic role Behavioral changes ML Incentives ML Institutional changes ML Policy changes MU Catalytic financing ML Champions ML Procedures affecting attainment of project results Preparation and readiness NA MS S Implementation approach/adaptive mngmnt S Stakeholder participation/public awareness S Country ownership and driven-ness S Financial planning and management S UNEP supervision and backstopping NA Monitoring and evaluation NA Complementarity with UNEP programmes and strategies Linkage to UNEP EAs & POW 2010-11 NA NA NA Alignment with the Bali Strategic Plan NA Gender NA South-South Cooperation NA Rating code for all criteria except Sustainability and catalytic role: Highly Satisfactory (HS); Satisfactory (S); Moderately Satisfactory (MS); Moderately Unsatisfactory (MU); Unsatisfactory (U); Highly Unsatisfactory (HU) Rating code for Sustainability and catalytic role: Highly Likely (HL); Likely (L); Moderately Likely (ML); Moderately Unlikely (MU); Unlikely (U); Highly Unlikely (HU) A. Conclusions The final evaluation ratings for the Niger-Nigeria test case are summarized in Table The major positive achievements of the Niger-Nigeria test case are summarized in the bullets below. The project has provided a good opportunity to explore the magnitude of expected carbon benefits associated with a range of IEM interventions in a trans-boundary context. CBP experiences at the Birnin Konni test site where a 50ha plot zoned off from grazing and other human activities in 2010 has demonstrated of recovery with subsequent re-colonization with grasses such as Pennisetum and Cenchrus, and trees such as Acacia albida, Acacia seyal and Acacia nilotica. Although, there is no evidence of any exploratory testing using the Detailed Assessment or Dynamic Modeling, the Simple Assessment module was adequately used to quantify carbon benefits for all four the four (4) initiatives in both the Birnin Konni and Maradi areas. The major portions of the sub-contractual agreement with CSU dealing with feedback, soil testing, and testing have been successfully implemented, thus
The major positive achievements of the Kenya(A)test case are summarized in the bullets below. Table D- 8: Summary of evaluation ratings for the Kenya(A)test case Criterion Sub-criterion Additional Sub-criterion rating Overall rating Attainment of objectives and planned results Achievement of outputs and activities NA MU MS Relevance S Effectiveness MS Efficiency S Outcomes to impacts MS Sustainability and catalytic role Sustainability Socio-political sustainability ML ML Financial resources ML Institutional framework L Environmental sustainability L Catalytic role Behavioral changes ML Incentives ML Institutional changes ML Policy changes ML Catalytic financing ML Champions ML Procedures affecting attainment of project results Preparation and readiness NA MS MS Implementation approach/adaptive mngmnt MS Stakeholder participation/public awareness MS Country ownership and driven-ness S Financial planning and management S UNEP supervision and backstopping NA Monitoring and evaluation NA Complementarity with UNEP programmes and strategies Linkage to UNEP EAs & POW 2010-11 NA NA NA Alignment with the Bali Strategic Plan NA Gender NA South-South Cooperation NA Rating code for all criteria except Sustainability and catalytic role: Highly Satisfactory (HS); Satisfactory (S); Moderately Satisfactory (MS); Moderately Unsatisfactory (MU); Unsatisfactory (U); Highly Unsatisfactory (HU) Rating code for Sustainability and catalytic role: Highly Likely (HL); Likely (L); Moderately Likely (ML); Moderately Unlikely (MU); Unlikely (U); Highly Unlikely (HU) The CBP achieved its main objective in providing a cost effective, user-friendly, yet scientifically rigorous methodology for measuring carbon and GHG mitigation benefits in GEF projects dealing with natural resources. The CBP has delivered the intended outputs and outcomes and the developed protocol which is now available online. However, the key factors that are likely to contribute to the persistence of the project benefits on the Kenya (A) test case are: Availability of project donor funds; Raising awareness on the CBP tool; Acceptability of the tool for use in carbon trading; Economic benefits the farmers are likely to get by practicing these SLM practices; and Continuity of carbon trading. Several scientists from KARI received training in the CBP model operation. The participation of the KARI scientists in both GEF Soil Organic Carbon (GEFSOC) and CBP projects has enhanced their capacity in modeling and can now measure, predict and monitor carbon stocks and fluxes and mitigate GHG in SLM projects.
Annex F: Terms of Reference - Overall Project TERMS OF REFERENCE Terminal Evaluation of the “Carbon Benefits Project: Modelling, Measurement and Monitoring (CBP: MMM)” PROJECT BACKGROUND AND OVERVIEW Project General Information Table 1. Project summary GEF project ID: 3449 IMIS number: GFL/2648-2713-4A47 Focal Area(s): Land Degradation with relevance to Climate Change and Biodiversity; Sustainable Forest Management (SFM); Forest Conservation as a Means to Protect Carbon Stocks and Avoid CO2 Emissions GEF OP #: GEF Strategic Priority/Objective: SFM-SP3 Sustainable Forest Management (SFM)-SP-3, LD: Forest Conservation as a means to Protect Carbon Stocks and avoid CO² Emissions. Resource efficiency-sust. Consumption/production. GEF approval date: 11 February 2009 Approval date: 30 March 2009 First Disbursement: 27 April 2009 Actual start date: April 2009 Planned duration: 36 months (2yrs Phase 1 + 1 year Phase II Intended completion date: March 2012 Actual or Expected completion date: Dec 2012 Project Type: Full Size Project (FSP) GEF Allocation: US$ 5,526,265 PPG GEF cost: Nil PPG Nil Expected MSP/FSP US$ 3,638,082 Total Cost: US$ 9,164,347 Mid-term /STAP. (planned date): Sept 2012 Terminal Evaluation (actual date): December-April 2012 STAP (In place of MTR) (actual date): 12-16 September 2012 No. of revisions: 1 Date of last Steering Committee meeting: 12-13 October 2011 Date of last Revision*: 13 April 2009 Disbursement as of 30 June 2012: US$ 4,238,903 Total co-financing realized as of 30 US$2,310777 Leveraged financing:
C: The project’s intended outcomes were delivered, but were not designed to feed into a continuing process after project funding C: The measures designed to move towards intermediate states have started, but have not produced results. B: The project’s intended outcomes were delivered, and were designed to feed into a continuing process, but with no prior allocation of responsibilities after project funding B: The measures designed to move towards intermediate states have started and have produced results, which give no indication that they can progress towards the intended long impact. A: The project’s intended outcomes were delivered, and were designed to feed into a continuing process, with specific allocation of responsibilities after project funding. A: The measures designed to move towards intermediate states have started and have produced results, which clearly indicate that they can progress towards the intended long term impact. Thus a project will end up with a two letter rating e.g. AB, CD, BB etc. In addition the rating is given a ‘+’ notation if there is evidence of impacts accruing within the life of the project. The possible rating permutations are then translated onto the usual six point rating scale used in all UNEP project evaluations in the following way. Table 2. Shows how the ratings for ‘achievement of outcomes’ and ‘progress towards intermediate states translate to ratings for the ‘Overall likelihood of impact achievement’ on a six point scale. Highly Likely Likely Moderately Likely Moderately Unlikely Unlikely Highly Unlikely AA AB BA CA BB+ CB+ DA+ DB+ BB CB DA DB AC+ BC+ AC BC CC+ DC+ CC DC AD+ BD+ AD BD CD+ DD+ CD DD In addition, projects that achieve documented changes in environmental status during the project’s lifetime receive a positive impact rating, indicated by a “+”. The overall likelihood of achieving impacts is shown in Table 11 below (a + score above moves the double letter rating up one space in the 6-point scale). The ROtI method provides a basis for comparisons across projects through application of a rating system that can indicate the expected impact. However it should be noted that whilst this will provide a relative scoring for all projects assessed, it does not imply that the results from projects can necessarily be aggregated. Nevertheless, since the approach yields greater clarity in the ‘results metrics’ for a project, opportunities where aggregation of project results might be possible can more readily be identified. Results rating of project entitled: Rating (D – A) Rating (D – A) Rating (+) Overall
Efficiency Are any or time-saving measures proposed to bring the project to a successful conclusion within its programmed budget and timeframe? Does the project intend to make use of / build upon institutions, agreements and partnerships, data sources, synergies and complementarities with other initiatives, programmes and projects etc. to increase project efficiency? Overall rating for Efficiency Sustainability / Replication and Catalytic effects Does the project design present a strategy / approach to sustaining outcomes / benefits? Does the design identify the social or political factors that may influence positively or negatively the sustenance of project results and progress towards impacts? Does the design foresee sufficient activities to promote government and stakeholder awareness, interests, commitment and incentives to execute, enforce and pursue the programmes, plans, agreements, monitoring systems etc. prepared and agreed upon under the project? If funding is required to sustain project outcomes and benefits, does the design propose adequate measures / mechanisms to secure this funding? Are there any financial risks that may jeopardize sustenance of project results and onward progress towards impact? Does the project design adequately describe the institutional frameworks, governance structures and processes, policies, sub-regional agreements, legal and accountability frameworks etc. required to sustain project results? Does the project design identify environmental factors, positive or negative, that can influence the future flow of project benefits? Are there any project outputs or higher level results that are likely to affect the environment, which, in turn, might affect sustainability of project benefits? Does the project design foresee adequate measures to catalyze behavioural changes in terms of use and application by the relevant stakeholders of (e.g.): i) technologies and approaches show-cased by the demonstration projects; ii) strategic programmes and plans developed iii) assessment, monitoring and
project results Project management efficiency Responsiveness Problems/constraints STAP Review recommendations followed Stakeholder Participation and Public Awareness Stakeholder engagement effectiveness Public awareness effectiveness Link of project results with global benefits Country Ownership and Driven-ness Government support/responsibility Adequacy of political/institutional framework Government promotion of project Government responsiveness Financial Planning and Management Application of proper standard Administrative processes Co-financing availability Resources leveraging UNEP and UNDP Supervision and Backstopping Adequacy of supervision plans Emphasis on monitoring Project reporting effectiveness Documentation quality Adequacy of financial/administrative supervision Monitoring and Evaluation Design adequacy Implementation adequacy with UNEP Medium Term Strategy and Work Programme Linkage to UNEP’s EAs and POW NA Alignment with the Bali Strategic Plan (BSP) NA Gender NA South-South Cooperation NA