COMMISSION STAFF WORKING DOCUMENT Accompanying the document Report from the Commission to the European Parliament and the Council on evaluating the implementation of Decision No. 406/2009/EC pursuant to its Article 14
Date: 2016-07-20
Institutions: Directorate-General for Climate Action, European Commission
Subjects: EU Member State, EU emission allowance, EU environmental policy, Kyoto Protocol, adaptation to climate change, climate change, environmental monitoring, greenhouse gas, pollution control, reduction of gas emissions

EN 

   EN 

   EUROPEAN COMMISSION  Brussels, 20.7.2016  SWD(2016) 251 final   COMMISSION STAFF WORKING DOCUMENT Accompanying the document Report from the Commission to the European Parliament and the Council on evaluating the implementation of Decision No. 406/2009/EC pursuant to its Article 14 {COM(2016) 483 final}  
 
 
Table of Contents 

3. 

1. 
2. 

EXECUTIVE SUMMARY .................................................................................................................................... 3 
INTRODUCTION .................................................................................................................................................. 5 
2.1.  Purpose of the evaluation ............................................................................................................................. 5 
2.2.  Scope of the evaluation ................................................................................................................................ 5 
BACKGROUND TO THE INITIATIVE ............................................................................................................... 6 
3.1.  Description of the initiative and its objectives ............................................................................................. 6 
3.2.  Baseline ........................................................................................................................................................ 9 
4. 
EVALUATION QUESTIONS ............................................................................................................................. 10 
5.  METHOD ............................................................................................................................................................. 11 
5.1.  Process and methodology ........................................................................................................................... 11 
5.2.  Limitations – robustness of findings .......................................................................................................... 12 
IMPLEMENTATION STATE OF PLAY ............................................................................................................ 12 
ANSWERS TO THE EVALUATION QUESTIONS ........................................................................................... 13 
7.1.  Relevance ................................................................................................................................................... 13 
7.2.  Effectiveness .............................................................................................................................................. 14 
7.3.  Efficiency ................................................................................................................................................... 21 
7.4.  Coherence................................................................................................................................................... 23 
7.5.  EU added value .......................................................................................................................................... 23 
7.6.  Competition ................................................................................................................................................ 24 
CONCLUSIONS .................................................................................................................................................. 25 
ANNEXES ........................................................................................................................................................... 27 

6. 
7. 

8. 
9. 

Annex 1: Procedural information 
Annex 2: Stakeholder consultation 
Annex 3: Methods and analytical models used in preparing the evaluation 
Annex 4: Provisions for flexibility instruments and monitoring and compliance in the Effort Sharing Decision  

2 

 
 
 
1. 

EXECUTIVE SUMMARY 

Article 14 of Decision No. 406/2009/EC (also called the Effort Sharing Decision, ESD) requires the 
European Commission to draw up a report evaluating the implementation of the ESD by 31 October 
2016. The evaluation was undertaken in 2015. 

Adopted in 2009 as part of the Climate and Energy package, the Effort Sharing Decision establishes 
greenhouse  gas  (GHG)  emission  limits  for  all  Member  States  for  2020.  It  covers  emissions  in  the 
transport,  buildings,  agriculture,  small  industry  and waste  sectors.  These  sectors  accounted  for  more 
than 55% of total EU GHG emissions in 2013. The objective of the ESD is to reduce GHG emissions 
in the EU by 10% by 2020 compared to 2005 and promote reductions of emissions within its scope in 
a fair and cost-effective manner. 

While  the  first year  of  reporting  by  Member  States  under  the  Decision  was  2015,  at  the  time  of  the 
evaluation most of the provisions of the Decision had not yet been applied, making it more difficult to 
draw conclusions about their appropriateness  and to consider any changes based  on lessons learned. 
However,  the  evaluation  could  use  valuable  evidence  deriving  from  the  process  of  legal 
implementation  of  the  Decision  and  the  preparatory  work  undertaken  by  Member  States,  including 
national policies and measures reported by them to the European Commission. 

There  is  strong  and  consistent  evidence  that  the  EU  is  on  track  to  meet  its  2020  GHG  emissions 
reduction  target  in  the  ESD  sectors.  Based  on  Member  State  reports  submitted  in  2015,  GHG 
emissions in 2013 and 2014 in all Member States were below their annual limits for these years. Total 
2013 emissions covered by the ESD at EU level were 9.7% lower than the 2005 emissions. In 2014 
EU emissions covered by the ESD further decreased to a level 12.9% below 2005 levels, which was 
also below the EU-wide ESD target for 2020.  

Most  emission  reductions  since  2009  have  occurred  from  technological  changes  and  policies  which 
have resulted in increased uptake of less carbon-intensive technologies. This effect has been reinforced 
by  the  fact  that  the  ESD  was  launched  together  with  a  number  of  other  EU  climate  and  energy 
initiatives as part of the 2020 package, in particular on energy efficiency and renewable energy.  

The economic recession has also had an effect on GHG emissions in some ESD sectors to date, and 
these effects will continue to resonate until 2020. However, many sectors covered by the ESD are not 
directly impacted by the fluctuations in GDP and are more prone to policy influence. For several of the 
ESD sectors, including buildings, transport, agriculture and waste, part of the emissions reductions to 
date  can  be  attributed  to  factors  that  are  influenced by  policy  interventions  at  EU  and  member  state 
level related to the 2020 package. Whilst it was possible to identify that the ESD has had some effect 
in  stimulating  new  national  policies  in  some  Member  States,  there  was  insufficient  evidence  to 
quantify the overall impact of the ESD on GHG emissions at this stage. 

The Effort Sharing Decision was found to add value through action on EU level. There was a strong 
level of agreement among stakeholders representing Member States that the ESD raised awareness of 
mitigation potential in ESD sectors and contributed to establishing new national institutional and legal 
frameworks.  It  also  improved  coordination  on  GHG  mitigation  across  the  ESD  sectors  and  between 
national and regional or local governments.  

Stakeholders highlighted that an important outcome of the ESD to date has been the improvement in 
the quality of the emissions data and projections relating to ESD sectors at a national level, which has 
helped improving policy preparation. The annual emission limits, combined with annual and biennial 
requirements  for  reporting  emissions,  policies  and  measures,  and  projections,  keep  Member  State 
better informed about the progress not only of GHG emissions but also of climate and energy policies. 
The reporting also gives them and other EU stakeholders a tool to compare their performance with that 
of other Member States. Finally, transparency of reporting helped building trust and confidence among 
Member States that each of them is contributing a fair share to the overall EU climate policy. 

3 

 
 
 
 
 
Evidence  on  the  direct  costs  and  benefits  and  the  efficiency  of  national  policies  implemented  in 
response to the ESD is very limited; it was not possible to assess these costs with confidence. This is 
partly  due  to  the  fact  that  national  policies  and  measures  reported  by  Member  States  so  far  have 
provided insufficient information on expected and actual costs and benefits. 

The  ESD  was  found  to  have  resulted  in  limited  additional  administrative  burden  on  Member  State 
level, although there may be opportunities for reducing administrative costs at EU level, for example 
by simplified or less frequent compliance controls. 

Stakeholders did not present any evidence that national policies resulting from the ESD have unduly 
distorted competition in the EU internal market. 

The objectives of the ESD remain relevant and still correspond strongly with the needs of the EU. The 
European Council Conclusions in October 2014 on the 2030 Climate and Energy Policy Framework 
confirmed the importance of the ESD and of its continuation after 2020 with all its main elements.  

The evaluation found a strong consensus among stakeholders that there was a continued need for an 
instrument such as the ESD also for the period after 2020. Evidence from stakeholder interviews, and 
to a lesser extent the literature review, suggested that the objectives of the ESD are coherent with other 
EU climate and energy policies, such as the ETS, energy efficiency and renewable energy. 

4 

 
 
 
 
 
 
 
 
 
2. 

INTRODUCTION 

Adopted in 2009 as part of the Climate and Energy package1, Decision No. 406/2009/EC2 (also called 
the Effort Sharing Decision, ESD) establishes greenhouse gas (GHG) emission limits for all  Member 
States for 2020. It covers emissions in the transport, buildings, agriculture, small industry and waste 
sectors.  These  sectors  accounted  for  more  than  55%  of  total  EU  GHG  emissions  in  2013.3  The 
objective of the ESD is to reduce GHG emissions in the EU by 10% by 2020 compared to 2005 and 
promote reductions of emissions within its scope in a fair and cost-effective manner. The ESD entered 
into force in June 2009. 

While  the  first year  of  reporting  by  Member  States  under  the  Decision  was  2015,  at  the  time  of  the 
evaluation, most of the provisions of the Decision had not yet been applied, making it more difficult to 
draw conclusions about their appropriateness  and to consider any changes based  on lessons learned. 
However,  the  evaluation  could  use  valuable  evidence  deriving  from  the  process  of  legal 
implementation  of  the  Decision  and  the  preparatory  work  undertaken  by  Member  States,  including 
national policies and measures reported by them to the European Commission. 

2.1. 

Purpose of the evaluation 

This evaluation was performed as required by Article 14 of the Effort Sharing Decision and it covers 
the  period  from  the  adoption  of  the  Decision  in  2009  until  November  2015.4  It  examined  the  actual 
implementation and the achievements of the Effort Sharing Decision at Member State and EU level, 
including to what extent it is contributing to the overall EU 2020 GHG emissions target as expected.  

It provides conclusions on how the Effort Sharing Decision has performed so far, what experience has 
been gained, and what lessons can be learned with respect to policies and measures implemented by 
Member States to reduce greenhouse gas emissions in the sectors covered by the Decision, and their 
effect on their national emissions. 

The results of the evaluation were also used in the impact assessment of the Commission’s legislative 
proposal for continuing effort sharing after 2020 within the 2030 Climate and Energy Framework.5 

2.2. 

Scope of the evaluation 

The  evaluation  explored  the  impacts  of  the  Effort  Sharing  Decision  both  at  EU  and  Member  State 
level.  Impacts  on  third  countries  were  expected  to  be  limited  and  mainly  associated  with  the  use  of 
international project credits as part of the flexibility instruments of the ESD. However, since the use of 
such credits by Member States has, to date, not occurred, the evaluation did not include this point. 

In accordance with Article 14 of the ESD, the scope of the evaluation included all the provisions and 
requirements of the Decision, including how the implementation affected competition. In addition, it 
implicitly  covers  reporting  and  verification  procedures  laid  down  in  Regulation  No  525/2013 
(Monitoring Mechanism Regulation, MMR) as the information needed for the compliance assessment 
under the ESD is submitted by Member States as required in the regulation.  

1 http://ec.europa.eu/clima/policies/strategies/2020/index_en.htm 
2 Decision No. 406/2009/EC of the European Parliament and of the Council of 23 April 2009 on the effort of Member States 
to  reduce  their  greenhouse  gas  emissions  to  meet  the  Community's  greenhouse  gas  reduction  commitment  up  to  2020; 
http://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:32009D0406&from=EN  
3 The current ESD covers the same greenhouse gases as the Kyoto Protocol, with the exception of nitrogen trifluoride NF3. 
Emissions or removals from land use, land-use change and forestry (LULUCF) are not included in the ESD. Emissions from 
international aviation and shipping are not covered by the ESD. 
4 Article 14 of the Effort Sharing Decision states that the Commission shall draw up a report evaluating the implementation 
of the ESD, including how it has affected competition, and present the results to the European Parliament and to the Council 
by 31 October 2016. 
5 SWD(2016) 247 

5 

 
 
 
                                                 
The  general  approach  of  the  evaluation  was  to  focus  on  Member  State  actions  from  when  the  ESD 
entered  into  force  in  June  2009  as  it  is  reasonable  to  assume  that  climate  mitigation  policies  and 
measures adopted in Member States from 2009 and onwards were influenced by the Decision.  

It should be noted that some of the main requirements in the ESD were only due for the first time in 
2015, and others only in 2016.  

3. 

BACKGROUND TO THE INITIATIVE 

3.1. 

Description of the initiative and its objectives 

The Climate and Energy Package sets EU-wide targets for 2020, requiring a 20% reduction in EU total 
greenhouse  gas  emissions  from  1990  levels,  a  20%  share  of  EU  final  energy  consumption  from 
renewable sources, and a 20% improvement in its energy efficiency. 

The  20%  GHG  emission  reduction  target  for  the  European  Union  by  2020  compared  to  1990  is 
equivalent  to  a  14%  reduction  compared  to  2005.  This  effort  has  been  divided  between  the  sectors 
covered by the Emission Trading System (ETS) and sectors under the ESD. In the EU ETS emissions 
are  required  to  decrease  by  21%  relative  to  2005  levels,  while  from  sectors  in  the  ESD  a  10% 
reduction relative to 2005 is required. 

To  share  the  required  EU-wide  effort  of  10  %  GHG  emission  reduction  by  2020  under  the  ESD, 
national targets for 2020 were set for Member States according to economic capacity on the basis of 
their relative wealth (measured by 2005 gross domestic product per capita). This principle for sharing 
the effort ensures fairness. The national targets range from a 20% emissions reduction by 2020 (from 
2005 levels) for the richest Member States to a 20% increase for the least wealthy one. (See Figure 1). 

The  ESD  does  not  set  specific  emission  targets  for  the  individual  sectors  covered  by  the  ESD,  but 
leaves it to Member States to choose where and how to achieve the necessary reductions according to 
their national situation 

To meet their commitments Member States are supposed to limit their greenhouse gas emissions in the 
sectors  covered  by  the  ESD,  and  they  are  expected  to  implement  national  policies  and  measures  to 
fulfil their obligations. Member States can also apply flexibility instruments to meet their obligations 
and enhance cost-effectiveness, if needed.  

The  ESD  also  defines  a  linear  trajectory  of  corresponding  binding  emission  limits  (annual  emission 
allocations,  AEAs)  for  each  year  from  2013  to  2020.  Progress  towards  the  2020  targets  is  ensured 
through  annual  reporting  obligations  and  compliance  checks.  Member  States  are  obliged  to  report  on 
their greenhouse gas emissions and projected progress towards meeting their ESD obligations. 

Member States are responsible for implementing policies and measures to meet their obligations under 
the ESD and are supported by a number of EU  measures6, some  of which also  are expected to help 
achieving  the  EU’s  2020  renewable  energy  and  energy  efficiency  targets.  These  supporting  EU 
policies are important to stimulate EU-wide emission reductions in the sectors under the ESD. But it is 
clear  that  emission  reductions  have  to  be  delivered  through  national  policies  and  measures,  in 
particular in sectors such as transport and buildings.  

In order to provide flexibility for Member States in implementing their commitments and as a  means 
to  enhance  the  overall  cost-effectiveness  of  reaching  the  EU-wide  2020  target,  the  ESD  provides  a 
range  of  flexibility  instruments.  These  concern  Member  States’  possibilities  to  manage  their  own 
AEAs  within  the  compliance  period  and  engage  in  transfers  of  AEAs  among  each  other.  Should  a 
Member State’s GHG emissions exceed its AEAs for a given year, it can borrow 5% of its AEAs from 

6  Examples  of  relevant  EU  legislation  are  the  Regulation  of  CO2  emissions  from  passenger  cars  and  vans,  the  Landfill 
Directive, the Energy Performance of Buildings Directive, the Energy Efficiency Directive, the Renewable Energy Directive, 
the F-gas Regulation and Mobile Air-Conditioning Systems Directive, the eco-design framework, the Nitrates Directive, and 
the greening of the Common Agricultural Policy.  

6 

 
 
 
                                                 
the  next  year,  buy  AEAs  from  other  Member  States,  or  use  international  project  credits  in  order  to 
meet  its  annual  limits.  Should  a  Member  State  reduce  its  emissions  by  more  than  needed  thereby 
overachieving  its  target  for  a  given  year,  it  can  keep  the  surplus  AEAs  for  later  use  within  the 
commitment period or transfer it to other Member States.7  

The ESD has an annual  reporting and compliance cycle consisting of Member  States reporting  their 
GHG  emissions  in  national  inventory  reports  to  the  European  Commission,  which  then  performs  an 
emission inventory review to validate the reported emissions, and a compliance check (i.e., comparing 
the  actual  emission  of  Member  States  with  their  annual  emission  allocations  for  a  given  year).  If  a 
Member State's emissions exceed its annual emission allocation, even after accounting for flexibility 
instruments, they will be subject to certain penalties and have to take corrective measures. 

Figure 1: Member State greenhouse gas emission limits in 2020 under the ESD 

Emission limits in 2020 are set in relation to 2005 emission levels.  Source: Decision No. 406/2009/EC.  

As outlined in the intervention logic diagram that was developed for the evaluation, as a result of the 
ESD  Member  States  are  expected  to  report  on  their  annual  greenhouse  gas  emissions  and  projected 
progress towards  meeting their obligations, and implement  national  policies and measures in  sectors 
covered  by  the  ESD  limiting  their  GHG  emission  by  2020  to  meet  their  national  targets.  They  are 
expected to apply flexibility instruments to enhance cost-effectiveness and compliance, if needed.  

7 For more details of the flexibility instruments and the rules for monitoring and compliance, see Annex 4.  

7 

 
 
 
 
                                                 
Intervention Logic 

In the diagram all actions and consequences are shown as linear. In practice, some steps will occur in parrallel, and there will also be some feedback loops between the different 
steps. For example, depending upon their expected emissions performance at a given point in time relative to their target, Member States may choose to implement more or less 
national policies. 

8 

Expected Impacts (Long-term effects of an intervention (change of state in the target group)•Preparation of national policies and measures to reduce long-term GHG emissions (beyond 2020).•Promotion of low-carbon energy sources, energy efficiency measures and other mitigation actions in Member States.•Additional benefits on Member State level in terms of reduced air pollution and health costs, green jobs and cleaner production in ESD sectors.•Preservation of the integrity of the internal market.External Factors (Factors independent of the regulatory activity's intervention which could partly or entirely be the cause of changes (results or impacts) •Economic crisis.•Fluctuations in energy prices.•Other EU-wide climate and energy policies, including CO2emission standards for light-duty vehicles, F-gas regulations, Renewable energy directive, Energy performance of buildings directive, Eco-design directive•Other EU-wide policies in sectors covered by the ESD, such as transport and agriculture that might affect greenhouse gas emissions. •Other Member States activities in sectors covered by the ESD.Objectives (Ultimate objectives at EU level to which the regulatory activity is supposed to be contributing )•To promote reductions of greenhouse gas emissions (GHG) in a cost-effective and fair manner.Actions(Actions undertaken (linked to operational objectives) to produce the expected results)•Establish national limits for GHG emissions from sectors covered by the ESD for all Member States including the determination of annual emission allocations (AEAs) for each year of the compliance period 2013-2020.•Establish the necessary infrastructure and capacity for annual Union reviews of Member State emission inventories.•Establish the necessary infrastructure for Member State reporting on annual greenhouse emissions and on projected progress towards their obligations under the ESD.•Establish the necessary infrastructure for Member States ESD accounts in the Union registry.Expected Results (Direct effect on target group; attributable short-and mid-term deliverables )•Emission reductions of CO2and other greenhouse gases in the sectors covered by the ESD, in order to meet the ESD target.•Member States assesses if the emission limit can be reached with existing or already planned measures•Member States quantify the abatement potential, and cost of mitigation measures in sectors covered by the ESD.•Member States establish a legal framework to target emissions in sectors covered by the ESD.•Member State establish an institutional framework to coordinate responsibilities for ESD implementation across different national and regional bodies.•Member States implementing national policies and measures in sectors covered by the ESD limiting their GHG emission by 2020.•Member States applying flexibility instruments to enhance cost-effectiveness and compliance, if needed.•Member States reporting on their annual greenhouse gas emissions and projected progress towards meeting their ESD obligations.Needs ( Needs in society, problems, issues to address )•Need to stabilise greenhouse gas concentrations in the atmosphere at a level that would prevent dangerous anthropogenic interference with the climate system.•Market failures , including external costs of climate change, means that the economic costs of climate change are not reflected in market pricesEU Intervention 
 
 
 
In  terms  of  impacts  with  more  long  term  effects  Member  States  are  expected  to  prepare  national 
policies  to  reduce  greenhouse  gas  emissions  beyond  2020  and  also  promote  low-carbon  energy 
resources  and  energy  efficiency.  Other  expected  long-term  impacts  are  reduced  air  pollution  and 
health costs, green jobs and cleaner production. 

3.2. 

Baseline 

In 2008 the Commission undertook an extensive analysis of the GHG emissions trends in EU and 
in  Member  States  up  to  2020  as  part  of  the  impact  assessment  accompanying  the  package  of 
implementation  measures  for  the  EU's  objectives  on  climate  change  and  renewable  energy  for 
20208. According to the impact assessment it was expected that, by 2020, EU-wide ESD emissions 
under a business-as-usual scenario would  increase  by 2.4  % relative to the 2005 emission levels. 
Projections submitted by Member States for a number of selected sectors under the ESD in 2009 
showed  a  similar  trend  in  projected  emissions  to  2020,  with  emissions  from  energy  use  and 
transport in particular expected to increase across the EU. (Figure 2) 

Figure 2: 

Note: WEM = with existing (current) measures; WAM; with additional (planned) measures) 
•Projections  based  on  data  reported  by  Member  States  under  the  Monitoring  Mechanism  Decisions  by  May  2009,  so  represents 
projections from 2008 and 2009. 
•For Industrial processes and waste the WEM scenario is identical to the WAM scenario 
Source: GHG emission trends and projections in Europe 2009, European Environment Agency 2009. 

It was concluded that for the majority of Member States additional national policy measures would 
need to be implemented in order for them to meet their emission limits if they were to be achieved 
with domestic action alone. As mentioned earlier, a wide range of EU-wide policies were expected 
to  help  Member  States  to  meet  their  emission  limits  under  the  ESD.  However,  the  expected 
quantitative  contribution  of  the  individual  EU  policies  towards  the  delivery  of  Member  States’ 
targets was not determined. 

It was also expected that some Member States would use flexibility mechanisms, including the use 
of international project credits, in order to deliver their ESD targets at least cost. 

8  SEC  (2008)  85/3  EC.  Commission  Staff  Working  Document:  Impact  Assessment  accompanying  the  Package  of 
Implementation  measures  for  the  EU's  objectives  on  climate  change  and  renewable  energy  for  2020.  Available  at 
http://ec.europa.eu/clima/policies/strategies/2020/documentation_en.htm  

9 

 
 
 
 
 
 
 
                                                 
4. 

EVALUATION QUESTIONS 

The  main  evaluation  questions  were  agreed  by  the  Commission  steering  group  set  up  for  this 
evaluation and part of the evaluation mandate. The questions relate to the relevance, effectiveness, 
efficiency,  coherence  and  EU  added  value  of  the  ESD  and  follow  the  established  evaluation 
framework  that  is  used  for  Commission  evaluation  methodology.  An  additional  question  was 
included on how the implementation of the ESD affected competition as stated in Article 14 of the 
ESD. 

The main evaluation questions were further elaborated and complemented in the study supporting 
the evaluation.9  

Relevance 

How well do the objectives of the Decision still correspond to the needs within the EU? 

Effectiveness 

How  do  the  observed  effects  correspond  to  the  objectives  of  the  ESD?  To  what  extent  can  the 
effects reasonably be credited to the ESD? 

To what extent is the EU on track to meet its 2020 GHG emission reduction target for the ESD? To 
what extent are Member States on track to meet their 2020 emission limits? 

What drivers and barriers (including actions by Member States and other stakeholders) contributed 
to  or  stood  in  the  way  of  achieving  the  objectives?  How  are  these  factors  addressed  in  the 
Decision? For example: How well are co-operation between central and regional/local authorities 
working in Member States? 

Efficiency 

To  what  extent  are  the  costs  resulting  from  the  implementation  of  the  ESD  proportionate  to  the 
results  and  benefits  that  can  be  linked  to  the  observed  effects  of  the  ESD?  This  should  include 
analysis  of  secondary  impacts  concerning  administrative  and  other  costs,  including  for  SMEs, 
employment etc.? 

What factors influenced the efficiency of the ESD to achieve its objectives? 

Which role did the flexibility instruments play until now? Are Member States using or planning to 
use them? 

Coherence 

How well does the ESD fit with other EU climate and energy policies, including on CO2 emission 
standards  for  light-duty  vehicles,  on  non-CO2  gases,  energy  efficiency  and  renewable  energy 
sources? To what extent are the different requirements under the ESD working coherently as they 
were expected to? 

EU added value 

How is the ESD contributing as expected to the EUs international commitments under the second 
commitment period of the UNFCCC's Kyoto Protocol? To what extent could the changes brought 
by the Decision have been achieved by national or individuals' measures only? 

What would be the most likely consequences of stopping or not continuing the ESD? 

Competition 

To what extent have the policies and measures implemented by Member States in order to achieve 
obligations  under  the  Effort  Sharing  Decision  affected  competition  on  national,  Union  and 
international level? 

9  Supporting  study  for  the  Evaluation  of  Decision  No.  406/2009/EC  (Effort  Sharing  Decision),  Ricardo  Energy  and 
Environment with Trinomic and Vito. 

10 

 
 
                                                 
5. 

5.1. 

METHOD 
Process and Methodology10 

The evaluation of the Effort Sharing Decision was performed in 2015. A steering group consisting 
of  members  from  the  concerned  directorate-generals  in  the  Commission  was  set  up  for  the 
evaluation. The European Environment Agency also participated in this group. A study supporting 
the evaluation was carried out by a team of external consultants for the European Commission from 
April  to  December  2015.11  .  The  steering  group  agreed  on  the  evaluation  mandate  and  on  the 
intervention  logic,  guided  the  work  of  the  consultants  and  made  a  quality  assessment  of  the 
evaluation report produced by the consultants.12 

As  far  as  possible,  evidence  was  collected  from  multiple  sources  to  enable  the  comparison  and 
triangulation  of  key  findings.  The  evaluation  used  evidence  from  emissions  and  emission  trends 
reported  by  Member  States,  adopted  national  policies  and  measures,  policy  literature  and  case 
studies. It used questionnaires and structured interviews with Member State experts and other key 
stakeholders involved in implementing ESD on national level to gather additional evidence.13 The 
evaluation  also  drew  on  the  results  of  a  public  consultation  on  the  preparation  of  a  legislative 
proposal  on  reducing  Member  States’  GHG  emissions  in  sectors  under  the  ESD  from  2021  to 
2030.14 

The evaluation benefited from the well-established EU monitoring framework for GHG emissions, 
projections  and  policies  measures  and  also  previous  research  on  GHG  emission  drivers.15  This 
reduced  the  need  for  extensive  collection  of  new  data  or  modelling.  Member  States  report  GHG 
emissions  data  to  the  European  Commission  and  the  United  Nations  Framework  Convention  on 
Climate  Change  (UNFCCC)  every  year.  In  addition  to  GHG  inventories,  Member  States  have  to 
report every second year on the policies and measures implemented in order to achieve their targets 
under ESD and on their projection. At the moment of the evaluation GHG inventory information 
for all years between 2009 and including 2013 was available. The reports on projections included 
data reported by Member States in 2015. The information in these reports provided important data 
sources for the evaluation. 

Two targeted questionnaire surveys were organised in order to gather the views of stakeholders on 
the  evaluation  questions.  A  general  survey  which  covered  all  of  the  evaluation  questions  was 
shared with all stakeholder groups. A specific survey on administrative burdens associated with the 
implementation of the ESD’s reporting requirements was targeted at Member States. 

The Commission launched a public consultation on the preparation of a legislative proposal on the 
effort  of  Member  States  to  reduce  their  GHG  emissions  to  meet  the  European  Union's  GHG 
emission reduction commitment in a 2030 perspective. The consultation ran from 26 March 2015 
to 18 June 2015 and it included questions relating to the current ESD. Each question relevant for 
the evaluation received between 78 and 100 responses that have been taken into account as part of 
the analysis. 

A number of stakeholder interviews were held in order to allow  the more detailed exploration of 
the evaluation questions and to provide contextual information to the data and indicators collected 
for  the  evaluation.  The  interviews  provided  a  valuable  form  of  evidence,  particularly  for  those 
evaluation  questions  that  cannot  be  tested  empirically.  Furthermore,  the  interviews  helped 
identifying additional sources of research, as well as validating and further explaining the findings 
from the literature review.  

10 The methodology applied for the evaluation is presented in more detail in Annex 3 to this Staff Working Document. 
11  Supporting  study  for  the  Evaluation  of  Decision  No.  406/2009/EC  (Effort  Sharing  Decision),  Ricardo  Energy  and 
Environment with Trinomic and Vito. Available at http://ec.europa.eu/clima/news/articles/news_2015060802_en.htm  
12 See Annex 1 for information on the evaluation procedure.  
13 The results of the stakeholder consultation are presented in Annex 2 of the Staff Working Document. 
14 For more information on this consultation, see http://ec.europa.eu/clima/consultations/articles/0025_en.htm  
15 DG CLIMA (2015) Decomposition analysis of the changes in GHG emissions in the EU and Member States. 

11 

 
 
                                                 
Based on the general survey and drawing on the stakeholders' interviews, three case studies were 
developed as practical examples exploring the wider impacts of the Decision. Each case was used 
to complement answers to evaluation questions together with the other evidence. 

5.2. 

Limitations – robustness of findings 

The  main  limitation  of  this  evaluation  is  linked  to  the  timing.  Because  of  the  requirement  of  the 
ESD to evaluate and report on the implementation of the ESD by October 2016 and the fact that the 
European Council in October 2014 endorsed a continuation of the ESD after 2020, the evaluation 
had to be carried out already in 2015. This was at an early stage of the compliance period and the 
first  year  of  the  compliance  check,  which  was  not  yet  completed  at  the  time  of  the  evaluation.16  
Consequently, at the time of the evaluation, most of the provisions of the Decision had not yet been 
applied, making it more difficult to draw conclusions about their appropriateness and to consider 
changes based on lessons learned.  

Another  limitation  is  related  to  the  fact  that  policies  and  measures  reported  by  Member  States 
provide insufficient information on their expected and actual costs and benefits. 

Further  limitations  are  inherent  to  the  methodology  used  for  the  evaluation.  The  sample  size  for 
both questionnaires and stakeholders interviews was relatively small. This was not surprising given 
the nature of the Decision which mostly concerned a limited number of specialists and experts in 
national  administrations  in  Member  States.  The  risk  of  biased  results  by  unrepresentative  sample 
was mitigated by ensuring the coverage of stakeholder from most groups and types. For example, 
both  the  interviews  and  surveys  included  governmental  representatives  from  large  and  small 
Member States, from Member States who are projected to meet their ESD target easily, and from 
those that may need to implement additional measures. 

Despite the efforts to increase the response rate during the evaluation, for other stakeholder groups 
the  sample  size  cannot  be  considered  representative.  In  particular,  the  views  of  business 
associations  are  potentially  under-represented,  although  the  lack  of  interest  from  this  group  may 
indicate  that  views  are  less  strong  in  general.  This  view  was  supported  by  the  interviews 
themselves,  with  a  generally  lower  level  of  understanding  and  awareness  of  the  ESD  from  this 
group.  This  potentially  reflects  the  lack  of  a  direct  impact  of  the  ESD  on  businesses,  with  any 
impacts arising through other EU and national policies. It should be noted that a public consultation 
preparing a proposal for continuing the ESD after 2020 was organised in parallel to the evaluation. 
The  consultation  received  responses  from  a  number  of  business  associations  and  it  appears  that 
they focused their feedback on this consultation.  

While  validation  of  the  research  conclusions  with  other  sources  was  performed  it  has  not  been 
possible within the scope of the evaluation to critically appraise these studies and their limitations. 

6. 

IMPLEMENTATION STATE OF PLAY 

As  shown  in  the  intervention  logic,  the  Commission  was  supposed  to  undertake  a  number  of 
actions to implement the provisions of the ESD. All these actions have been completed.  

The  Effort  Sharing  Decision  mandates  the  Commission  to  determine  annual  emission  allocations 
for  each  Member  State  for  the  years  2013  to  2020.  Annex  II  of  the  ESD  sets  the  limits  to  be 
achieved  by  each  Member  State  by  2020  expressed  in  percentages  while  Article  3  provides  the 
methodology  for  these  percentages  to  be  translated  into  annual  emissions  allocations  (AEAs) 
expressed in CO2 equivalents in a Commission decision. According to Article 10 of the ESD the 
AEAs set by this decision also have to be adjusted by the Commission in order to account for any 
scope correction in the ESD due to any changes in the scope of the ETS in the period 2013-2020.17  

16  Because  of  technical  difficulties  with  reporting  software  provided  by  the  UNFCCC,  Member  State  submissions  of 
GHG emissions inventories by the Member States, the first compliance check under the ESD was further delayed into 
2016.  
17 Decision 2013/162/EU and Decision 2013/634/2013. 

12 

 
 
 
                                                 
The  Commission  adopted  both  these  measures  in  2013  following  an  initial  review  of  Member 
States  GHG  inventories  for  the  years  2005,  2008,  2009  and  2010  performed  by  the  Commission 
with the assistance of the European Environmental Agency.18 

The Commission has also adapted existing legislation and established the necessary infrastructure 
for Member States ESD accounts in the Union Registry.19. 

With respect  to Member State obligations under the ESD, they must each  year report their  GHG 
emissions.  They  must  also,  every  second  year,  report  on  their  implemented  national  policies  and 
measures  and  submit  updated  projections  in  2015  with  expected  progress  to  their  2020  targets. 
Each  year  the  Commission  compiles  the  information  reported  by  Member  states  and  publishes  a 
progress  report  where  it  analyses  and  presents  Member  States’  progress  towards  their  2020  ESD 
targets.  An  assessment  of  Member  States’  progress  towards  their  targets  is  also  part  of  the 
European Semester country report published by the Commission every spring. Emission trends and 
projections  in  the  EU  and  its  Member  States  are  also  published  each  year  by  the  European 
Environment Agency which assists the Commission with the implementation of the ESD. 

Member  States  have  so  far  fulfilled  their  reporting  obligations  and  the  exchange  of  information 
with  the  Commission  is  working  well.  The  Commission  and  Member  States  expert  meet  several 
times  every  year  in  working  groups  under  the  Climate  Change  Committee  to  follow  up  the 
implementation of the reporting obligations under the ESD.  

So  far,  no  Member  State  has  used  any  of  the  flexibility  instruments  provided  in  the  ESD  as  all 
countries appear to be meeting their annual emission limits for the first two years of the compliance 
period. In the future it can be expected that flexibility instruments, such as trade with other Member 
States, will be used by some Member States that are projected to emit above their limits by 2020. 
Although the flexibility instruments under the ESD are untested they remain widely supported and 
were further endorsed by Member States in the stakeholder consultation. 

7. 

ANSWERS TO THE EVALUATION QUESTIONS 

This section provides a synopsis of the answers to the evaluation questions presented in section 3. 
A  detailed  analysis  of  the  answers  to  all  questions  is  provided  in  chapter  6  of  the  supporting 
evaluation study. 

7.1. 

Relevance 

The ESD is part of the package of measures adopted under the climate and energy package in 2009 
addressing the need to cut emissions to a level that limit the global average temperature increase to 
below  2  degrees  Celsius  compared  to  pre-industrial  levels.  It  was  designed  to  ensure  that  this 
objective is met in a cost effective and fair manner. In order to answer the question on relevance, 
the evaluation assessed how well the objectives of the Decision still correspond to the needs within 
the EU. 

The  relevance  of  the  objectives  of  the  ESD  and  the  need  for  continuing  this  initiative  was 
confirmed  by  the  European  Council  in  its  conclusions  in  October  2014  on  the  2030  Climate  and 
Energy  Policy  Framework  which  agreed  of  its  continuation  to  2030  in  a  cost-efficient  and  fair 
manner with all the elements as applied in the Effort Sharing Decision for 2020.  

The adoption in December 2015 of the Paris Agreement confirmed the commitment at the highest 
level of the EU to implement domestically greenhouse gas emissions reductions also for 2030 and 
beyond. As laid down in its intended nationally determined contribution (INDC), the EU commits 
to cut emissions in its territory by at least 40% below 1990 levels by 2030.  

18 These four years are the reference years for determining the AEAs for the period 2013-2020 as laid down in the ESD. 
19 Regulation No 389/2013 establishing a Union Registry lays down rules in order to insure the accounting of transactions 
under the ESD. Member States accounts in the Union Registry were prepared in several steps and completed in 2016. 

13 

 
 
 
                                                 
The evidence from both literature review and stakeholder consultation suggests that the objectives 
of  the  ESD  of  ensuring  emissions  reductions  in  a  cost  efficient  and  fair  manner  still  correspond 
strongly  with  the  needs  within  the  EU.  Therefore,  while  the  policy  and  economic  context  has 
changed as compared to the time when the Decision was first introduced, the problems that it was 
designed to address remain as important, as do the objectives of the Decision itself.  

Out  of  the  34  responses  received  in  the  stakeholder  consultation  to  the  question  whether  there 
continues  to  be  a  need  for  the  ESD  or  a  similar  instrument  to  promote  reductions  of  GHG 
emissions in the ESD sectors across the EU, in a cost-effective and fair manner, the large majority 
of stakeholder including national governments, NGO and academia agreed (17) or strongly agreed 
(14), with no respondents disagreed with the statement. 

A further need addressed by the Effort Sharing Decision was to overcome the market failure which 
means that prices do not reflect the full social costs of the GHG emissions (i.e. the economic costs 
of  climate  change),  so  fail  to  provide  appropriate  price  signals  to  deliver  the  required  level  of 
abatement  and  stimulate  cost-effective  mitigation.  The  binding  nature  of  the  Decision  helps  to 
address  this  need  by  providing  appropriate  price  signals  for  emissions  abatement,  by  providing 
Member State limits on emissions in the ESD sectors and allowing trading between member States 
as part of the flexibility mechanisms. This need largely remains as strong. Whilst some EU wide 
policies will in part correct for this market failure, for example, by regulating the CO2 performance 
of new vehicles, or by requiring minimum levels of energy taxation, the market corrections are not 
applied equally across all sectors of the economy. 

7.2. 

Effectiveness 

The evaluation questions explored if the EU and its Member States are on track to meet their 2020 
obligations,  to  what  extent  Member  States  have  implemented  national  policies  and  measures  any 
emission reductions to reduce emissions, and whether any emission reductions can reasonably be 
credited  to  the  ESD.  The  questions  also  covered  how  cooperation  between  central  and  regional 
authorities was working with respect to implementing the ESD.  

Based  on  data  reported  by  Member  States  to  the  Commission,  there  is  strong  and  consistent 
evidence  that  the  EU  is  on  track  to  meet  its  2020  GHG  emissions  reduction  target  in  the  ESD 
sectors.20 Total 2013 GHG emissions covered by the ESD at EU level were 9.7% lower than the 
2005  emissions.  In  2014  EU  emissions  covered  by  the  ESD  further  decreased  to  a  level  12.9% 
below 2005 levels, which was below the EU-wide ESD target for 2020.21 

The  achieved  emission  reduction  represents  a  large  improvement  in  performance  relative  to  the 
business-as-usual scenario when the 2020 targets were first agreed. Emission reductions have so far 
been  much  deeper  than  expected  in  2007  when  the  European  Council  agreed  on  the  overall  EU 
climate targets for 2020 and the Commission performed the impact assessment of the Climate and 
Energy Package. According to the business as usual scenario in the impact assessment prepared for 
the  Climate  and  Energy  Package,  EU-wide  ESD  emissions  were  expected  to  increase  by  2.4% 
between 2005 and 2020.22  

In the EU total emission reductions between 2005 and 2013 were achieved in all sectors, ranging 
from  -3  %  in  agriculture  to  -25  %  in  the  waste  sector.  (Figure  3)  In  this  period  there  was  also  a 
convergence of GHG emission intensities across Member States, both per capita and per GDP.  

ESD  emissions  per  Member  State  have  also  decreased  significantly  since  2005.  In  all  Member 
States  ESD  emission  were  below 
in  2013  and  2014.  (Figure  4) 
Overachievements were typically larger in those countries that were allowed to increase emissions 

their  annual 

limits 

20 Greenhouse gas emission trends and projections in Europe 2015. Tracking progress towards Kyoto and 2020 targets, 
European Environment Agency, 2015. 
21 ESD emissions in 2013 based on inventory data and in 2014 based on approximated inventory data reported in 2015 by 
Member States or estimated by the EEA, where needed, on behalf of the Commission. 
22 SEC (2008) 85/3 EC 

14 

 
 
                                                 
compared  to  2005.  Countries  that  experienced  a  particularly  severe  economic  recession  (e.g., 
Greece, Portugal and Spain) also had emissions significantly below their 2014 limits. 

According  to  Member  State  projections  submitted  in  2015,  total  ESD  emissions  are  expected  to 
keep decreasing until 2020, continuously being below target at the EU level. (Figure 5) The EU-
wide ESD emissions are expected to decrease by 13.3% in 2020 relative to 2005 emission levels 
(under  a  scenario  taking  into  consideration  existing  measures)  which  is  overachieving  the  10% 
reduction attributed to the ESD sectors. 

A total of 24 Member States are projected to meet their  national targets domestically, while four 
Member  States  (Austria,  Belgium,  Ireland  and  Luxembourg)  are  expected  to  need  additional 
measures  or  use  flexibility  instruments  within  the  ESD  to  reach  their  targets.23  These  Member 
States  may  exceed  their  emission  limits  for  one  or  more  years  of  the  2013-2020  period  with 
existing measures in place. However, they are expected to comply with their legal obligations for 
each year in 2013-2020, thanks to AEA surplus from earlier years in the 2013-2020 period, which 
Member  States  can  use  to  cover  any  emissions  gap  up  to  2020  or  by  implementing  additional 
measures at home.24  

23 Climate action progress report, COM(2015) 576 final. 
24 For example, Luxembourg is projected to fall short of its 2020 national target and expects to cover this gap by using 
the flexibility instruments provided for in the ESD. Its ESD emissions are mostly from road transport, which represents 
more than two-thirds of its ESD emissions due to low excise duties on  motor fuel as well as a large number of cross-
border commuters. 

15 

 
 
 
 
                                                 
Figure 3: Achieved EU-wide ESD emission reductions 2005-2013 

16 

 
 
 
 
 
 
 
 
 
 
Figure 4 

Gap between ESD emissions and annual limits in 2014 
(expressed in % of 2005 base year). Negative values indicate 
emissions below the annual limit. 

5%

-5%

-15%

-25%

-35%

-16% 

-14% 

-14% 

-14% 

-13% 

-12% 

-12% 

-12% 

-9% 

-9% 

-8% 

-7% 

-7% 

-7% 

-7% 

-7% 

-2% 

-2% 

-2% 

0% 

-4% 

-4% 

-18% 

-19% 

-21% 

-20% 

-27% 

-31% 
Y
C

U
H

L
E

T
P

R
H

L
N

E
S

I

OS
R

E
E

T
I

K
S

G
B

S
E

K
D

R
F

T
M

V
L

K
U

E
I

T
A

E
B

T
L

E
D

L
P

Z
C

U
L

I
F

Gap between 2014 emissions and 2014 emission limits under the ESD

The figure shows that in all Member States ESD emission were below their annual limits in 2014 under the ESD. The 
emissions are based on based on approximated inventory data reported in 2015 by Member States or estimated by the 
EEA on behalf of the Commission, where needed. Adapted from Figure 3 in the Climate Action Progress Report 2015.25  

Figure 5: Actual and projected total ESD emissions 2005-2020 

Source:  Trends  and  projections  in  Europe  2015,  EEA  (2015).  Updated  with  emission  inventory  figures  submitted  by 
March 2016. The black dotted line represents the linear trajectory of the annual emission limits under the ESD for the 
period 2013-2020. 

25 http://ec.europa.eu/clima/policies/strategies/progress/docs/com_2015_576_en.pdf  (For details see Approximated EU 
GHG inventory: Proxy GHG estimates for 2014, report of the European Environment Agency: 
http://www.eea.europa.eu/publications/approximated-eu-ghg-inventory-2014). 

17 

 
 
 
 
 
 
 
 
                                                 
 
Impact of the Effort Sharing Decision on observed GHG emissions reductions 

The Effort Sharing Decision was supposed to limit GHG emissions in the EU and in the Member 
States. While the previous analysis shows a significant drop in GHG emissions at EU and Member 
States  level,  it  is  unclear  from  the  evidence  how  much  of  these  reductions  can  be  attributed  to 
national measures adopted as a direct result of the ESD. It was not possible to quantify the extent to 
which the observed reductions in historical emissions, and the expected future emissions trend, can 
reasonably be credited to specific policies. In particular, it is difficult to isolate the impact of the 
ESD on national climate and energy policies, and from other EU-wide initiatives under the Climate 
and Energy Package, which have a more direct impact on the different drivers of emissions. 

The  economic  recession  has  also  had  an  effect  on  GHG  emissions  in  some  ESD  sectors  to  date 
(especially freight transportation), and these effects are expected to resonate until 2020. However, 
many sectors (such as buildings and agriculture) covered by the ESD are not directly impacted by 
the  fluctuations  in  GDP  and  are  more  prone  to  policy  influence which  gives reasons  to  conclude 
that some of the GHG reductions in the ESD sectors can be attributed to policy interventions at EU 
and Member State level.  

The results from a  study decomposing the historical drivers of GHG emissions in the EU show that 
for the ESD sectors, the economic recession did not play as important a direct role in the observed 
emission  reductions  even  if  for  the  whole  economy  30%  of  the  change  in  emissions  can  be 
attributed to decrease in GDP.26 

While  GDP  is  particularly  relevant  for  the  non-ETS  industry  and  freight  transport,  the 
decomposition analysis revealed that for emissions from buildings, passenger transport, agriculture 
and waste, a significant part of the reductions could be attributed to factors that are influenced by 
policy  intervention.  However,  it  was  not  possible  to  quantify  the  extent  to  which  the  observed 
effects  on  historical  emissions,  and  the  expected  future  emissions  trends,  can  reasonably  be 
credited to the ESD alone. In particular, it is difficult to isolate the impact of the ESD from other 
policies, including other EU policies27. 

Another decomposition analysis was carried out for the 2005-2012 period covering CO2 emissions 
from fossil fuel combustion, which account for about 80 % of total GHG emissions, in both ETS 
and  ESD  sectors.  The  analysis  concluded  that  technological  changes  contributed  most  to  drive 
down  emissions,  by  far  outweighing  the  contribution  of  the  shift  within  and  between  economic 
sectors, and by far overcompensating GDP related emission drivers in times of economic crisis. 28 

The  results  showed  that  CO2  emissions  decreased  overall  by  11.5  %  between  2005  and  2012. 
Technological  changes  had  the  most  significant  effect  on  driving  down  emissions,  leading  to  an 
18.5  %  decrease  and  by  far  outweighing  the  contribution  of  the  shift  between  economic  sectors. 
The policies implemented in the field of climate and energy contributed significantly to the take-up 
of  less  carbon-intensive  technologies,  including  renewable  energy.  Growth  in  economic  activity 
(GDP) caused a 6.8 % increase in emissions. Structural changes in the economy (at constant GDP 
and CO2 z\intensity in every economic sector) caused a small increase in emissions, by 1.7 %.  

Stakeholders  expressed  mixed  views  on  how  the  ESD  contributed  to  the  observed  emission 
reductions.  Respondents  from  all  stakeholders  groups  identified  the  implementation  of  national 
policies (which may have been introduced in response to the ESD) as an important driver in some 
cases,  however,  the  effects  of  the  recession,  and  other  EU  wide  policies  were  also  considered 
important. While some stakeholders considered that the ESD has been an important driver for new 
national policies and measures in certain Member States, others considered that the ESD may have 
had little or no influence on national policy developments so far. The level of influence of the ESD 

26 Why did greenhouse gas emissions decrease in the EU between 1990 and 2012? European Environment Agency, 2014: 
27 For more information on the findings from decomposition analysis please refer to Appendix 4 in Supporting study for 
the Evaluation of Decision No, 406/2009 (Effort Sharing Decision), Ricardo Energy and Environment with Trinomic and 
Vito.   
28 Climate action progress report, COM(2015) 576 final, p.9. 

18 

 
 
                                                 
also appears to vary by Member State, which may reflect the different situation of Member States 
in relation to their ESD targets, i.e., whether they need to take further action or not. 

Based on the findings of the supporting study of the evaluation, Table 1 summarises different that 
have so far contributed to changes in GHG emissions in ESD sectors. 

Table 1: Summarising factors contributing to changes in emission levels in ESD sectors29  

Buildings 

Transport 

Agriculture 

Waste 

GDP 

(0)1 

++  

(only for freight 
transport) 

Other external factors 

+ to +++ 

++  

(0) 

++ 

(0) 

(0) 

(heating demand, 
population size, 
number of 
households) 

(fuel prices, for 
freight transport also 
decreasing share of 
industry)) 

(milk yield) 

Effect of EU and 
national policy 

+++ 

++ 

++ 

+++ 

(e.g. EPBD, 
renewable energy 
directive, energy 
efficiency directive)  

(e.g. renewable 
energy directive, fuel 
quality directive, 
CO2 and cars 
regulation) 

(e.g. CAP, nitrates 
directive) 

(e.g. landfill 
directive, waste 
framework directive) 

Importance is ranked from (0) marginal effect to +++ very significant effect 

Implementation of new national policies in ESD sectors 

The evaluation found an increase in the implementation of national policies in the ESD sectors in 
most  years  starting  from  2007,  when  the  European  Council  agreed  on  the  overall  EU  climate 
targets for 2020. 

All sources of evidence indicated that the ESD has been a driver for new (i.e., planned after 2009) 
national policies and measures (PAMs) in over half of Member States (based  on the EEA PAMs 
database: Belgium, Croatia, Denmark, Estonia, France, Germany, Greece Latvia, Lithuania, Malta, 
Romania  and  Spain,  based  on  the  stakeholder  consultation  also  Austria,  the  Netherlands  and 
Hungary).  The  PAMs  cover  all  sectors:  transport  (39),  energy  consumption  (25),  waste  (12), 
agriculture (7), industrial processes (2), energy supply (6), and cross-cutting (28). 

According  to  the  stakeholder  consultation,  the  ESD  has  been  successful  in  reinforcing  or 
strengthening  existing  PAMs  in  some  Member  States,  as  well  as  raising  awareness  of  the  GHG 
abatement potential and cost of mitigation measures (23 out of 34 of respondents agreed or strongly 
agreed) and supporting the development of new legal frameworks or legislative actions for limiting 
ESD emissions (19 respondents out of 34 agreed or strongly agreed). Specific policy examples that 
were  cited  included  Austria  (Climate  Change  Act  studies),  Hungary  (Domestic  Decarbonisation 
Roadmap),  Lithuania  (Strategy  for  the  National  Climate  Change  Management  Policy  by  2050), 
Latvia (Climate Change Financial Instrument), the Netherlands (various fiscal measures), Romania 
(initial  steps  towards  new  national  legal  framework)  and  Spain  (2020  Roadmap  for  ESD 
emissions). However, some respondents disagreed that the ESD had resulted in new policies (9 out 
of 34 respondents disagreed) or in the strengthening of existing policies (4 out of 34 respondents 
disagreed). 

29 Supporting study for the Evaluation of Decision No, 406/2009 (Effort Sharing Decision), p.46. 

19 

 
 
 
 
 
                                                 
It is therefore apparent that the strength of the ESD as a driver varies between Member States, and 
in  some  of  them  the  ESD  may  have  had  little  or  no  influence  on  national  policy  developments. 
Analysis of the reports submitted by them to the Commission suggests that this difference does not 
necessarily  reflect  the  status  of  Member  States  with  respect  to  the  expected  performance  against 
their  ESD  targets.  However,  this  is  not  born  out  by  the  stakeholder  interviews,  where  this  status 
was considered more important. This indicates an apparent inconsistency in the reporting of linked 
EU policies under the MMR, and on the views expressed by stakeholders. 

The number of PAMs implemented in response to ESD is only a rough indicator of policy action. 
The expected GHG savings from the PAMs would be more informative. However, this information 
is  scarce.  In  the  reporting  of  GHG  savings  by  Member  States,  the  impacts  of  single  PAMs  are 
frequently aggregated and reported for a group of policies. In Figure 6 reported emission savings 
expected  by  2020  have  been  aggregated  per  EU  policy  based  on  an  assessment  made  by  the 
European  Environment  Agency.30  This  shows  that  the  reported  impact  of  the  ESD  is  relatively 
small  compared  to  other  EU  policies,  such  as  the  Renewable  Energy  Directive  or  the  ETS.  The 
result  is  perhaps  not  surprising  given  that  the  ESD  only  sets  an  overall  cap  for  emissions  in 
Member  States,  whereas  most  other  EU-wide  policies  included  in  the  comparison  were  designed 
for  specific  measures  or  sectors.  It  should  also  be  noted  that  this  analysis  only  can  give  a  rough 
indication, considering the incompleteness of reporting of emission savings by the Member States. 

There may be several reasons as to why, according to their reporting, the ESD does not appear to 
have been a major driver of policies in some Member States to date. Firstly, the reporting may not 
represent a complete and accurate reflection of the influence of EU policies on the development of 
national climate policies. In their reporting, Member States might link national policies only to the 
most  obvious  EU  policy  and  not  necessarily  to  all  EU  policies  that  have  had  an  influence. 
Secondly, there may be a lag in policy development in response to the ESD, for instance the case 
studies show that in Belgium and Spain the governments have installed plans to put them on track 
to  achieve  the  ESD  target,  but  this  is  not yet  fully  reflected  in  either  concrete  new  policies  or  in 
their  reporting  of  2015.  Finally,  Member  States  do  not  have  to  achieve  their  ESD  target  with 
domestic action alone, but can instead use the flexibility instruments of the ESD to compensate for 
any shortfalls in emission reductions compared to the target.  

The evaluation produced reasonable evidence that the ESD has contributed to a certain extent to the 
cooperation between national and regional or local authorities. Nevertheless such influence is likely 
to  be  different  between  Member  States.  In  some  Member  States  little  or  no  cooperation  or 
coordination  has  taken  place  between  central  and  regional  or  local  level.  This  is  supported  by 
evidence  from  the  sixth  National  Communications31,  where  16  Member  States  did  not  explicitly 
mention coordination between central and regional or local level. This however does not mean that 
mitigation actions at local or regional level are not taking place. 

30 Synthesis of the data on GHG PAMs reported by Member States under the Monitoring Mechanism Regulation in 2015, 
European Environment Agency, 2015. 
31  Member  States  regularly  submit  national  communications  to  the  UNFCCC  including  information  on 
climate policies and measures to fulfil their international obligations. 

20 

 
 
 
 
 
 
                                                 
Figure 6: Aggregated emission savings from EU policies (source: EEA)32 

7.3. 

Efficiency 

The costs and benefits due to the implementation of measures in ESD sectors are hard to quantify, 
as  a  direct  link  between  many  national  climate  and  energy  policies  and  emission  reductions  in 
sectors under the ESD so far is difficult to establish. This is also due to the fact that policies and 
measures reported by Member States provide insufficient information on their expected and actual 
costs and benefits. 

The relative youth of climate change policies that were introduced as a  result of the ESD and the 
climate  and  energy  package  means  there  are  currently  few  assessments  of  ex-post  cost 
effectiveness.  In  coming  years,  this  should  change  and  a  more  complete  picture  of  the  extent  to 
which the ESD has delivered cost effective abatement across sectors should become more apparent. 

Benefits associated to the Effort Sharing Decision 

The main benefit associated with the ESD relates to the GHG emissions reductions in the sectors 
covered  by  the  Decision.  In  addition  to  any  direct  costs,  the  implementation  of  the  ESD  has 

32 WEM = with existing policies and measures, WAM = with additional measures (not yet implemented) 

21 

 
 
 
                                                 
potentially also delivered a number of economic co-benefits, such as improvements in air quality 
and energy security (i.e., reduced costs of fuel imports).33 

A  further  qualitative  benefit  of  the  ESD  appears  to  be  that  it  helps  Member  States  in  setting  up 
institutional  frameworks,  targets  and  data  collection  and  reporting  procedures  for  reducing 
emissions  in  the  ESD  sectors.  This  benefit  is  more  apparent  in  those  Member  States  where  the 
policies reducing emissions in these sectors are devolved to a regional level. 

Costs  from  the  implementation  of  the  ESD  come  from  two  main  sources  –  implementation  of 
policies and measures in relevant sectors and ESD reporting. 

Costs associated with the implementation of national GHG mitigation policies in the ESD sectors 

Data  gathered  during  the  evaluation  suggests  that  of  four  main  ESD  sectors  (transport,  direct 
emissions  from  buildings,  waste  and  agriculture)  the  most  cost-effective  abatement  is  found  in 
buildings, with measures in the other sectors being more expensive. In particular some measures in 
the agriculture sector have a very high cost per tonne of CO2 reduced.34  

However,  these  findings  have  a  high  level  of  uncertainty  as  the  evidence  comparing  cost 
effectiveness  in  ESD  sectors  is  relatively  scarce  and  where  it  does  exist  has  often  been  done  on 
different timeframes and potentially with differing methodologies making comparison difficult. 

Data reported by Member States includes information on the cost, and cost-effectiveness of GHG 
mitigation policies when available but provide insufficient information on their expected and actual 
costs  of  such  policies.  Of  the  123  individual  policies  reported  by  Member  States  that  were 
identified as being linked with the ESD, cost data was only provided for 9 policies. This severely 
limited the value of this data source in assessing the cost-effectiveness of policies implemented in 
response to the ESD.  

Furthermore, the uncertainty is increased by a number of factors that make comparisons of costs of 
climate  policies  challenging,  including  uncertainty  around  the  extent  to  which  additional  costs 
incurred  by  businesses  are  passed  on  to  consumers,  use  of  different  models  and  differences  in 
definitions of what is included in the term ‘costs’ (e.g. investment costs, operation and maintenance 
costs, subsidy payments, government administrative costs and revenues from energy savings). 

Costs associated with reporting 

The  most  tangible  costs  related  to  the  ESD  are  those  related  to  the  reporting  and  compliance 
requirements  for  national  governments.  The  cost  for  the  Commission  and  the  European 
Environment  Agency  of  the  current  system  for  reviewing  emission  inventories  and  checking 
compliance  is  in  the  order  of  650.000  EUR  per  year,  while  the  annual  costs  for  all  28  Member 
States taken together is estimated to be in the order of 500.000 EUR per year. These costs include 
administration of other reporting obligations under the ESD, which are separate from the emissions 
reporting and compliance check.35. 

The ESD has no direct reporting obligations for business or SMEs. 

Role of the flexibility instruments under the Effort Sharing Decision 

The  flexibility  instruments,  including  transfers  of  AEAs  among  Member  States,  together  with 
legally binding annual targets in the ESD were expected to be an important driver of cost efficiency 
under  the  ESD.  This  was  also  generally  acknowledged  by  stakeholders  although  some  expressed 
concern  that  the  flexibilities  may  undermine  some  of  the  stimulus  that  the  ESD  provides  to  take 
domestic action. 

33 The impact assessment prepared for the climate and energy package that was presented by the Commission in 2008 
estimated  reduced  air  pollution  costs  at  10  to  11  billion  EUR  by  2020  as  a  result  of  the  EU’s  20%  GHG  emission 
reduction target (i.e., including both the ESD and the ETS).  
34 Supporting study for the Evaluation of Decision No, 406/2009 (Effort Sharing Decision), Section 6.3.4.1. 
35 Source: Supporting study for the Evaluation of Decision No. 406/2009/EC (Effort Sharing Decision), Ricardo Energy 
& Environment with Trinomics and Vito, December 2015. 

22 

 
 
                                                 
When  Member  States  reported  in  2014  on  their  intentions  to  engage  in  AEAs  transfers  only  3 
Member  States  indicated  that  they  were  planning  to  purchase  AEAs,  whereas  10  Member  States 
intended to sell.  

One of the reasons for the lack of a market developing under the ESD might be linked to the fact 
that most of the Member States are on track to meet or overachieve their targets. The low demand 
for AEAs might also be due to the fact that the ESD is still in the early years of the implementation 
and  that  there  might  be  other  drivers,  not  investigated  by  the  evaluation,  that  would  prompt 
Member  States  to  prefer  the  implementation  of  domestic  actions  rather  than  buying  AEAs  from 
other Member States.  

The fact that Member States only to a limited extent intend to use these instruments for compliance 
with  their  emissions  limits  might  indicate  that  the  ESD  has  missed  on  its  objective  of  promoting 
mitigation actions in the places where this would be the most cost efficient. The supply indicated 
by  Member  States  in  2014  (around  80  million)  of  AEAs  was  8  times  higher  than  the  indicated 
demand  (around  10  million).  This  reflects  the  expectation  of  most  Member  States  that  they  will 
meet  their  2020  targets  under  the  ESD without  any  need  for  acquiring  AEA  transfers  from  other 
countries in the EU.  

7.4. 

Coherence 

The  objective  of  the  ESD  is  expected  to  be  supported  by  other  EU-wide  climate  and  energy 
policies. As noted earlier, the implementation of the Decision is still at a relatively early stage and 
consequently  there  is  yet  limited  evidence  on  how  coherently  the  different  climate  and  energy 
initiatives have been implemented. There is a lack of studies specifically relating to the ESD and its 
coherence with other PAMs in the published literature. Thus the evaluation to a large extent relied 
on  views  expressed  in  the stakeholder  consultation, which  in  itself  was  a  relatively  small  sample 
size.  

Evidence from stakeholder interviews, and to a lesser extent the literature review, suggests that the 
objectives of the ESD are largely coherent with other EU climate and energy policies, such as the 
ETS, energy efficiency and renewable energy. Stakeholders identified a strong coherence with the 
EU  objectives  relating  to  energy  efficiency  and  renewable  energy,  although  some  stakeholders 
queried the coherence of the targets themselves.  

By providing flexibility as to how Member States meet their emission limits, the objectives of the 
ESD was found to be largely coherent with national policy making.  

There  is  a  potential  lack  of  coherence  between  the  ESD  and  other  interventions  in  relation  to 
agriculture and land use, land use change and forestry (LULUCF). Part of the issue with coherence 
relates to other international commitments, rather than coherence with EU policies; agriculture is 
included in the ESD and Kyoto Protocol, while LULUCF is in the Kyoto Protocol but not the ESD.  

Coherence  with  other  reporting  obligations  was  identified  as  strong,  both  with  EU  internal 
requirements  under  the  MMR  and  with  international reporting  requirements under the UNFCCC. 
Some  streamlining  opportunities  of  reporting  obligations  under  the  ESD  and  under  EU  energy 
policies were identified. One such example is the requirements to report on projections both under 
EU climate and energy legislation, which possibly could be streamlined into one single obligation. 

7.5. 

EU Added Value 

The  evaluation  examined  to  what  extent  the  changes  brought  by  the  Decision  could  have  been 
achieved  by  national  or  individuals'  measures  only,  and  how  it  is  contributing  to  the  EU’s 
international commitments under the Kyoto protocol’s second commitment period (2013-2020). 

There  was  a  strong  indication  in  the  stakeholder  consultation  that  binding  national  targets,  and 
robust  requirements  for  monitoring  and  compliance  in  the  ESD  were  important  features  to 
stimulate national action,  For a large majority of EU Member States there were no or only weak 
domestic policy drivers before the ESD.  

The evaluation indicated that Member States (e.g., Germany, UK, Denmark, Sweden and Finland) 
having  more  ambitious  domestic  targets  than  the  ones  under  the  ESD  enshrined  in  their  own 

23 

 
 
national  legislation  may  have  taken  action  anyway  in  response  to  those  national  laws  and 
associated targets. Although the evaluation did not account for the fact that the EU discussions and 
agreement on the EU 2020 targets in 2007 may have helped support the setting of national targets 
in  the  first  place,  by  providing  certainty  that  other  EU  Member  States  will  be  taking  a  certain 
minimum level of action. However, for a large majority of EU Member States this domestic driver 
did not exist before the ESD, which suggests that without the Decision the actions may not have 
been taken, or may have been taken at a slower pace. Even where other drivers of action were in 
place  within  Member  States,  the  stakeholder  consultation  concluded  that  the  ESD  might  have 
exerted an additional positive influence, even if it was not a primary driver. 

Another important added value of the ESD to date has been the improvement in the quality of the 
emissions  data  and  projections  relating  to  ESD  sectors  at  a  national  level,  which  has  helped 
improving  policy  preparation.  The  annual  reporting  of  emissions,  combined  with  biennial 
requirements  for  reporting  of  policies  and  measures,  and  projections,  keep  Member  State  better 
informed  about the progress not only of GHG emissions but also of climate and energy policies. 
The  reporting  obligations  also  give  them  and  other  EU  stakeholders  a  tool  to  compare  their 
performance with that of other countries in the EU.  

The  ESD  is  important  instrument  to  ensure  that  the  EU  and  its  Member  States  deliver  on  their 
international  obligations  under  the  UNFCCC’s  second  Kyoto  protocol,  which  has  the  same 
commitment period as the ESD (2013-2020).36 As shown in Section 7.2, the EU is well on track to 
meet their Kyoto commitments. 

7.6. Competition 

The evaluation examined whether domestic policies and measures implemented by Member States 
in response to the ESD may have distorted the internal EU market. Due the lack of specific data 
regarding the impact of such national measures on the EU internal market, the evaluation outcome 
is based solely on stakeholder's opinions. 

The majority of stakeholders indicated that the ESD has had no or limited impact on competition 
within the EU. Two respondents indicated that the national mitigation policies induced by the ESD 
may  restrict  the  potential  for  growing  output  from  the  agriculture  sector  because  of  a  perceived 
lower  mitigation  potential  in  that  sector.  However,  no  stakeholders  offered  examples  of  where 
national  policies  and  measures  resulting  from  the  ESD  had  any  adverse  impact  on  specific 
enterprises or parts of a sector. 

36 Member States commitments for reducing or limiting their GHG emission under the second Kyoto protocol is equal 
too their 2020 targets under the ESD but also includes any emissions or removals in LULUCF.  

24 

 
 
 
 
 
                                                 
8. 

CONCLUSIONS 

The  Effort  Sharing  Decision  is  still  in the  early  stages  of  implementation.  Nevertheless,  it  seems 
clear from the evidence gathered so far that the ESD targets together with the other initiatives under 
the  Climate  and  Energy  Package  have  been  effective  in  stimulating  new  national  policies  and 
measures promoting effective reductions of GHG emissions within the scope of the ESD.  

The objectives of the ESD remain relevant and still correspond strongly with the needs of the EU. 
The  European  Council  Conclusions  in  October  2014  on  the  2030  Climate  and  Energy  Policy 
Framework  confirmed  the  importance  of  the  ESD  and  of  its  continuation  after  2020  with  all  its 
main elements.  

There  was  strong  consensus  among  stakeholders  that  the  ESD  as  an  EU  instrument  to  promote 
GHG emission reductions in a cost-effective and fair manner should be continued. There was also a 
strong indication that binding national targets, flexibility instruments, and robust requirements for 
monitoring and compliance were important features to promote action in Member States. 

With respect to effectiveness, the EU is on track to meet its 2020 GHG emissions reduction target 
in the ESD sectors. Based on Member States’ reports submitted in 2015, GHG emissions in 2013 
and 2014 in all Member States were below their annual limits for these years. Total 2013 emissions 
covered by the ESD at EU level were 9.7% lower than the 2005 emissions. In 2014 EU emissions 
covered by the ESD further decreased  to a level 12.9% below 2005 levels, which  was below the 
EU-wide ESD target for 2020. This means that the EU is on track to meet its ESD target in 2020 
and that Member States have contributed to the reduction in GHG emissions.  

Most  emission  reductions  since  2009  have  come  from  technological  changes  and  policies  which 
have  resulted  in  increased  uptake  of  less  carbon-intensive  technology.  This  effect  has  been 
reinforced by the fact that the ESD was launched together with a number of other EU climate and 
energy  initiatives  as  part  of  the  2020  package,  in  particular  on  energy  efficiency  and  renewable 
energy.  

The economic recession has also had an effect on GHG emissions in some ESD sectors to date, and 
these effects will continue to resonate until 2020. However, many sectors covered by the ESD are 
not  directly  impacted  by  the  fluctuations  in  GDP  and  are  more  prone  to  policy  influence.  For 
several  of  the  ESD  sectors,  including  buildings,  transport,  agriculture  and  waste,  part  of  the 
emissions reductions to date can be attributed to factors that are influenced by policy interventions 
at EU and member state level related to the 2020 package.  Whilst it was possible to identify that 
the  ESD  has  had  some  effect  in  stimulating  new  national  policies  in  some  Member  States,  there 
was  insufficient  evidence  to  quantify  the  overall  impact  of  the  ESD  on  GHG  emissions  at  this 
stage. 

The  evaluation  of  the  ESD  showed  an  increase  in  the  implementation  of  national  policies  in  the 
ESD sectors  in most  years starting from 2007, when the European Council  agreed on the overall 
EU climate targets for 2020. Without the ESD, actions to mitigate emissions in the ESD sectors at 
Member States level may not have been taken, or may have been taken at a slower pace.  

Evidence  on  the  direct  costs  and  benefits  and  the  efficiency  of  national  policies  implemented  in 
response to the ESD is very limited; it was not possible to assess these costs with confidence. This 
is partly due to the fact that national policies and measures reported by Member States so far have 
provided insufficient information on expected and actual costs and benefits. 

With  respect  to  reporting costs,  the  ESD  was  found  to  deliver  the  outcomes  efficiently,  although 
there  may  still  be  some  opportunities  for  reducing  administrative  burdens.  Costs  related  to 
reporting  and  compliance  are  modest  and  mostly  fall  on  the  Commission  and  the  European 
Environment  Agency.  There  was  no  major  variation  in  terms  of  administrative  costs  apparent 
between  Member  States.  There  may  be  opportunities  for  reducing  such  costs  at  EU  level,  for 
example by simplified or less frequent compliance controls. 

Evidence  from  stakeholder  interviews,  and  to  a  lesser  extent  the  literature  review,  suggested  that 
the  objectives  of  the  ESD  are  coherent  with  other  EU  climate  and  energy  policies,  such  as  the 
ETS, energy efficiency and renewable energy. 

25 

 
 
By providing flexibility as to how Member States deliver their emission limits, the objectives of the 
ESD are largely coherent with national policy making.  

Coherence with other reporting obligation was also  identified as strong especially with the MMR 
and the reporting requirements under UNFCCC. However, streamlining opportunities of reporting 
obligation under the ESD and under EU energy focused legislation were identified. 

The  ESD  was  found  to  add  value  through  action  on  EU  level.  There  was  a  strong  level  of 
agreement among Member State stakeholders that the ESD raised awareness of mitigation potential 
in ESD sectors and contributed to establishing new national institutional and legal frameworks. It 
also improved coordination on GHG mitigation across the ESD sectors and between national and 
regional or local governments.  

There was strong consensus from stakeholders that there was a continued need for an instrument 
such as the ESD also for the period after 2020. 

Stakeholders did not present any evidence that national policies resulting from the ESD have 
unduly distorted competition in the EU internal market. 

26 

 
 
 
 
9. 

ANNEXES 

ANNEX 1: PROCEDURAL INFORMATION 

The  Directorate-General  (DG)  for  Climate  Action  was  leading  the  evaluation  of  Decision  No. 
406/2009/EC (the Effort Sharing Decision, ESD) within the European Commission.  

An  inter-service  Steering  Group  chaired  by  DG  Climate  Action  was  set  up  in  January  2015  for 
guiding  the  evaluation.  The  following  DGs  and  services  were  invited  to  the  Steering  Group: 
Secretariat  General  (SG),  Legal  Service  (SJ),  Agriculture  and  Rural  Development  (AGRI), 
Competition  (COMP),  Economic  and  Financial  Affairs  (ECFIN),  Energy  (ENER),  Environment 
(ENV),  Internal  Market,  Industry,  Entrepreneurship  and  SMEs  (GROW),  Joint  Research  Centre 
(JRC),  Mobility  and  Transport  (MOVE),  Regional  and  Urban  Policy  (REGIO),  Research  and 
Innovation (RTD), Taxation and Customs Union (TAXUD), Trade (TRADE).  

The  Steering  Group  agreed  on  the  evaluation  mandate  and  the  terms  of  reference,  including  the 
intervention  logic,  for  the  consultants  contracted  for  carrying  out  an  external  evaluation.  The 
European Environmental Agency was also invited to participate in the Steering Group. The group 
met five times in the period from March 2015 to December 2015. 

The evaluation mandate was included in the Commission’s agenda planning and work programme 
for 2015 and was published on the Commission’s Europa web pages.37  

The external evaluation of the implementation of the Effort Sharing Decision was carried out from 
April to December 2015 by a team of external consultants led by Ricardo Energy & Environment 
and contracted by the Commission.38 

The  steering  group  provided  guidance  to  the  external  consultants  at  different  stages  of  the 
evaluation  and  quality  assessed  the  evaluation  report.  The  group’s  quality  assessment  of  the 
evaluation  report  considered  its  relevance,  design,  use  of  data  and  sources,  analysis  and 
conclusions.  

The  external  evaluation  report  and  the  quality  assessment  are  available  on  the  Commission’s 
Europa web pages.39 

37 http://ec.europa.eu/clima/news/articles/news_2015060802_en.htm  
38  Supporting  study  for  the  Evaluation  of  Decision  No.  406/2009/EC  (Effort  Sharing  Decision),  Ricardo  Energy  and 
Environment with Trinomic and Vito. 
39 http://ec.europa.eu/clima/news/articles/news_2015060802_en.htm  

27 

 
 
 
 
 
                                                 
ANNEX 2: STAKEHOLDER CONSULTATION 

The stakeholder consultation was carried out as part of the external study supporting the evaluation. 
A detailed description of how the consultation was organised and its result is provided in Appendix 
1 and Appendix 2 to the study,40  

 Organisation of the consultation 

Stakeholders  have  been  consulted  using  four  mechanisms:  general  questionnaire  survey, 
administrative  burdens  questionnaire,  EU  public  consultation,  stakeholder  interviews.  The  scope 
and audience for each of the mechanisms is described below.  

All  stakeholder  types  were  given  the  opportunity  to  participate  in  the  evaluation.  However,  the 
following stakeholder groups were specifically targeted:  

1.  EU  policy  makers  i.e.  individuals  within  the  Commission  who  may  be  involved  in  the 
development or implementation of the ESD and related policies, and individuals from the 
EEA involved in the monitoring of the ESD. 

2.  National ESD administrators i.e. the individuals involved in the implementation of the ESD 

in Member States. 

3.  National policy makers i.e. the individuals within national, regional and local governments 
who may be involved in the development or implementation of national  GHG  mitigation 
policies.  

4.  National  inventory  compilers  i.e.  individuals  within  Member  States  who  are  involved  in 
compliance activities associated with the monitoring and reporting of emissions data. 

5.  Wider interest groups (Industry associations, academia and civil society). 

The  general  public  was  not  actively  targeted  by  the  evaluation.  However,  this  stakeholder  group 
was able to provide views by responding to the EU public consultation.  

The  evaluation  sought  to  achieve  a  balanced  representation  of  views  from  within  each  of  the 
different stakeholder groups. For example, the representation of Member States aimed to cover the 
following different circumstances: 

  Member  States  with  more  stringent  domestic  emissions  reduction  targets  than  those 

required under ESD. 

  Member  States  that  have,  following  the  introduction  of  the  ESD,  been  projected  to  meet 

their ESD emission limits without the need for further domestic action. 

  Member  States  that  have,  following  the  introduction  of  the  ESD,  been  projected  to  miss 

their ESD emission limits without further domestic action. 

  Member  States  whose  ESD  emissions  are  biased  towards  a  single  source  or  sector  e.g. 

agriculture. 

  Member State where control of GHG emission is a regional or local responsibility.  

In relation to the wider interest group, the evaluation sought  to achieve an equal balance between 
non-governmental  organisations  and  industry  associations.  Likewise,  representation  was  targeted 
from organisation representing each of the main sources/sectors within the scope of the ESD (i.e. 
transport, agriculture, buildings). 

Invitations to participate in the evaluation were sent to a total of 224 individuals, representing each 
of  the  different  stakeholder  types.  To  achieve  the  required  balance  in  representation,  additional 
targeting and encouragement to participate in the evaluation was provided to certain groups. 

Each of the main consultation tools are summarised below. 

40  Supporting  study  for  the  Evaluation  of  Decision  No.  406/2009/EC  (Effort  Sharing  Decision),  Ricardo  Energy  and 
Environment with Trinomic and Vito.  

28 

 
 
                                                 
1.  General questionnaire survey 

Scope:  This  web-based  questionnaire  sought  views  from  stakeholders  on  a  number  of  the 
evaluation questions. The questionnaire therefore provided a structured response, based on closed 
multiple-choice answers. Open-response answers were allowed for certain questions.  

Audience: Invitations to complete the survey were sent to stakeholders in Member States, Industry 
Associations and NGOs. The questionnaire was accessible behind a password protected website.  

2.  Administrative burdens questionnaire 

Scope:  This  was  a  separate  MS  Word  questionnaire  that  sought  views  on  just  the  administrative 
burdens associated with the implementation of the ESD.  

Audience:  The  survey  targeted  those  individuals  within  Member  States  who  are  involved  in  the 
implementation of the ESD at national level, in particular monitoring and reporting. 

3.  EU public consultation41 

Scope: The Commission launched a public consultation on the preparation of a legislative proposal 
on the effort of Member States to reduce their GHG emissions to meet the European Union's GHG 
emission reduction commitment in a 2030 perspective.  

Audience: Open to all stakeholders, including the general public. 

The responses to this consultation, where relevant to the evaluation questions, have been taken into 
account as part of the analysis.  This comprised between 78 and 100 responses (depending upon the 
question).  

4.  Stakeholder interviews 

Scope:  There  is  a  need  to  explore  certain  issues  relevant  to  the  evaluation  in  more  depth.  The 
interviews covered all of the evaluation questions, but were tailored to the audience concerned. 

Audience: This involved a detailed discussion with a limited number of key stakeholders including 
stakeholders in the EC, Member States, Industry Associations and NGOs. The list of stakeholders 
was  informed  by  the  data  and  literature  review,  for  example,  if  an  NGO  had  commissioned 
research  or  prepared  a  position  paper  relating  to  the  ESD  then  this  demonstrated  that  they  had  a 
good  understanding  of  the  ESD  so  were  on  the  priority  list  of  stakeholders.  Likewise,  for  the 
business  associations  prioritisation  was  given  to  organisations  representing  business  within  the 
main  ESD  target  sectors  (e.g.  transport,  agriculture,  buildings,  industry)  as  well  as  organisations 
representing  small  and  medium  sized  enterprises.  However,  ultimately,  interviews  could  only  be 
held with stakeholders who were interested in participation, and it proved challenging to persuade 
some stakeholders, particularly business associations to participate. 

Results from the questionnaire survey and stakeholder interviews 

Overview of respondents to the questionnaire survey 

In  total,  35  complete  responses  were  received  to  the  questionnaire  survey,  including  one  written 
response. In addition 7 partial responses were submitted, but have not been taken into account. In 
almost  all  cases  of  partial  responses  the  respondents  failed  to  complete  more  than  the  initial 
registration  pages  i.e.  they  stopped  once  they  got  to  the  technical  questions,  which  indicates  that 
they may not feel sufficiently knowledgeable to provide views on the questions posed. Given that 
the  questions  were  specific  to  the  ESD,  and  there  is  only  a  relatively  small  community  of 
individuals who are actively involved in ESD implementation, this is an unsurprising outcome. It 
also helps to explain the relatively low response rate overall in comparison to say policies with a 
more clear and direct impact on multiple stakeholder groups e.g. product regulations. 

41 A synopsis of the public consultation is provided in Annex 8.2 to the impact assessment accompanying the proposal for 
continuing effort sharing in the non-ETS sectors in the period 2021-2030, SWD(2016) 247. 

29 

 
 
                                                 
The majority of respondents were from national government, which was the primary target group 
for  the  questionnaire.  The  second  largest  group  was  non-governmental  organisations  with  9 
respondents. A further 3 respondents were from academia or research organisations, and another 3 
organisations  represented  business.  There  were  2  respondents  from  international  administration, 
both from the EC. One further respondent described themselves as ‘Other, public services’. 

On one hand the overall response rate was low compared to the number of invitations to participate 
that  were  sent  out  to  (224).  However,  given  that  only  a  relative  small  cohort  of  individuals  who 
have been directly involved in the implementation of the Decision, the response rate is considered 
reasonable. In particular, responses were received from 18 (including one written response) of the 
28 Member States. 

One  potential  barrier,  which  may  help  to  explain  the  low  response  rate,  was  the  timing  of  the 
questionnaire. The questionnaire was open from May until the end of August, which coincided with 
the holiday season. To address the time constraint the original deadline for the consultation period 
was  extended  (the  original  deadline  was  the  end  of  July).  In  addition,  the  questionnaire  was 
launched  at  the  same  time  that  stakeholders  were  asked  to  provide  responses  to  the  public 
consultation. There was therefore a potential risk that stakeholders may have got confused between 
the  different  consultation  activities.  This  was  mitigated  as  far  as  possible  by  clearly  explaining 
about  the  different  surveys  in  all  communication.  A  related  issues  was  the  risk  of  consultation 
fatigue – whereby respondents may have been less willing to participate in the questionnaire survey 
having  recently  spent  time  responding  to  the public consultation. It  is  not  clear  if  this  was  a  real 
issue, but may explain the low initial response rate.  

Note: does not include one written response from a national government 

In relation to the ESD related experience of the respondents, the experience most frequently stated 
was involvement in the implementation of the ESD, involvement in climate change policy making 
and  involvement  in  ESD  monitoring  and  compliance.  This  response  mirrors  the  stakeholder 
distribution of stakeholder groups described above, since all of these activities are the responsibility 
of the national government actors. 

One  respondent  indicated  that  they  had  no  direct  involvement  in  ESD  implementation  to  date. 
These views are nevertheless still considered valid – since the questionnaire allowed respondents to 
answer don’t know to questions they didn’t feel knowledgeable enough to answer. 

30 

21631291024681012141618International administrationNational government ornational administrationAcademic/researchorganisationA small or medium-sizeenterpriseTrade associationrepresenting businessNon-governmentalorganisationOther (please specify)Number of respondents by stakeholder type (34 responses) 
 
 
Overview of audience for the stakeholder interviews 

In  total  24  interviews  were  held  with  stakeholders.  Of  these,  18  were  follow-up  interviews  with 
selected respondents to the questionnaires survey. The specific stakeholders were selected based on 
their response to the questionnaire e.g. if they made an important point that was worth following 
up, or if they didn’t provide explanation of their views so more detail was required. Consideration 
was also given to the type of stakeholder (to ensure a balance of stakeholder views) in the selection 
process.  The  remaining  6  interviews  were  with  stakeholders  on  the  priority  list  who  had  not 
accepted the invitation to complete the questionnaire survey. The interviews were therefore offered 
as an alternative means by which they could participate in the evaluation. These are highlighted as 
new stakeholders in the table below. 

The interviews were distributed amongst the stakeholder groups as follows: 

Stakeholder Group 

Number of interviews 

EC, and EU decentralised Agencies 

National governments  

Non-governmental organisations 

Business/Industry associations 

Relevance of the Decision 

Are the objectives of the ESD still relevant? 

4 (3 new) 

12 

6 (1 new) 

2 (2 new) 

This question formed part of the stakeholder interview and views from all stakeholder groups.  

There was a strong consensus that the overall objectives of the ESD are still relevant. Stakeholders 
expressed the view that these objectives are consistent with the overarching EU 2020 targets, and 
that an instrument such as the ESD with binding national targets is important in ensuring the EU 
wider  targets  are  met.  Some  respondents,  however,  suggested  that  the  ESD  was  less  relevant 
currently as most Member States can meet their target without any additional measures though it 
was recognised that the ESD will be more important in the future. One stakeholder indicated that 
this is consistent with the European Council Conclusions that call for the ESD to be continued for a 
further 10 years. 

Effectiveness of the Decision 

A series of questions were asked in relation to the effectiveness of the ESD. 

31 

20%41%20%16%3%ESD related experience of respondents (34 responses)Involved in the implementation of the ESDat national, regional or local levelInvolved in climate change policy making atnational, regional or local levelInvolved in ESD monitoring and compliancee.g. through work on emissions inventoriesInvolved in activities or industries affectedby the ESDNo direct involvement in ESDimplementation to date 
 
 
 
 
Do you think the EU is on track to deliver its 2020 GHG emissions reduction target for the non-
traded i.e. ESD sectors? 

There was a strong consensus that the current evidence indicated the EU was on track to meets its 
2020 target for the ESD sectors. Member States latest emissions projections, as well as EU wide 
modelling  results  from  the  PRIMES  and  GAINS  models,  were  cited  as  evidence  to  support  this 
view. However, it was noted that the situation differs between the individual Member States, with 
some Member States being more on track than others. These conclusions also represent the current 
expectations and may change in the future.  

Stakeholders also highlighted that one of the outcomes of the ESD to date is an improvement in the 
quality of the emissions data and projections relating to ESD sectors at a national level. This has 
itself provided more robust data to enable stakeholders to more accurately draw conclusions on 
whether Member States are on track. 

What do you consider are the main factors that have influenced whether the EU is currently on 
track or not? 

Responses were received from all the stakeholder groups, with 12 responses received in total.  

Stakeholders  identified  a  range  of  factors  which  have  influenced  whether  Member  States  are 
currently  on  track  to  deliver  their  emissions  limits  under  the  ESD.  The  introduction  of  national 
policies, which is one of the expected impacts of the ESD, was one of the four main factors cited. 
The  other  main  factors  were  the  economic  recession,  the  impacts  of  EU  wide  policies,  and  the 
overall  stringency  of  the  limits  themselves.  There  was  no  clear  factor  that  was  cited  more 
frequently that the others, and respondents frequently cited more than one factor.  

How effective has the ESD been in promoting reductions of GHG emissions to help deliver the 
target? 

32 

0611132202468101214StronglydisagreeDisagreeNeither agreenor disagreeAgreeStrongly agreeDon’t knowThe Effort Sharing Decision (ESD) has been effective in promoting reductions in greenhouse gas (GHG) emissions, in those sectors and from those emissions sources that are within its scope (34 responses) 
 
 
 
 
 
Has the ESD been more effective in promoting reductions in some sectors than in others? If so 
why? 

In practical terms, what have been the main activities or developments that have been stimulated 
(e.g. within Member States) by the ESD? 

Has the ESD supported national and regional GHG policy developments? 

33 

0510152025303540AgricultureBuildingsIndustry (non-Emissions Trading System (ETS))TransportWasteServicesThe ESD has been effective in promoting reductions in GHG emissions in each of the following sectors (34 responses)Strongly disagreeDisagreeNeither agree nor disagreeAgreeStrongly agreeDon’t know0%10%20%30%40%50%60%70%80%90%100%Established new legal frameworks, or legislative actions forlimiting GHG emissions in sectors covered by the ESDEstablished new institutional frameworks (e.g. carbonbudgets) for the coordination of the delivery of ESD targetsRaised awareness of the GHG abatement potential, andcost of mitigation measures in sectors covered by the ESDThe ESD has stimulated the following activities/developments in Member States (34 responses)Strongly disagreeDisagreeNeither agree nor disagreeAgreeStrongly agreeDon’t know0%10%20%30%40%50%60%70%80%90%100%Implementation of new national policies and measures to reduceemissions in ESD sectorsStrengthening or reinforcement of existing national policies andmeasures to reduce emissions in ESD sectorsImplementation of new regional or local policies and measures toreduce emissions in ESD sectorsStrengthening or reinforcement of existing regional or localpolicies and measures to reduce emissions in ESD sectorsThe ESD has resulted in the following outcomes in Member States (34 responses)Strongly disagreeDisagreeNeither agree nor disagreeAgreeStrongly agreeDon’t know 
 
 
 
 
 
 
 
Do you think the existence of the ESD has made it easier to agree additional mitigation actions 
in the EU and/or Member States? 

This  question  was  asked  as  part  of  the  stakeholder  interviews.  Responses  were  received  from  7 
stakeholders,  including  representatives  from  national  governments,  the  EC,  non-governmental 
organisations and industry associations.  

There was agreement amongst stakeholders that the ESD has had a positive influence on additional 
mitigation  actions  at  Member  State  level,  although  the  strength  of  this  influence  appears  to  vary 
from one Member State to the next. The strength varied between being perceived as an important 
stimulus  for  additional  action,  to  only  stimulating  discussions  amongst  different  government 
department on mitigation actions in ESD sectors.  

Do you think the ESD has had any influence on the cooperation between central and 
regional/local authorities working in Member States on GHG mitigation issues? 

Aside from the impacts on GHG emissions, what other positive impacts have the ESD had at 
Member State level? 

Has the ESD had any negative impacts at Member State level? 

Only  a  few  examples  were  identified,  and  none  more  than  once.  These  were:  potential  for 
competitive  distortions,  negative  air  quality  impacts  from  a  switch  to  biomass,  the  use  of 
flexibilities reducing the motivation for domestic action, and potential conflicts with other policy 
objectives.  One  broader  issue  that  was  highlighted  by  a  stakeholder  was  the  potential  conflict  at 
Member State level between economic development objectives, and the need to limit emissions. It 
was considered that this has not been a major conflict to date, however, it may become more so in 
the future. 

34 

14615350246810121416StronglydisagreeDisagreeNeither agreenor disagreeAgreeStrongly agreeDon’t knowIncreased cooperation between national, regional and local bodieson GHG emissions and/or mitigation options in ESD sectors (34 responses)0%10%20%30%40%50%60%70%80%90%100%Promotion of low-carbon energy sources in Member StatesPromotion of energy efficiency measures in Member StatesPromotion of other GHG mitigation actions in Member StatesImprovement in environmental quality e.g. through reduced air pollutionCreation of new jobs in low carbon sectorsMonetary savings through reduced spending on imported fossil fuelImproved energy securityAs a result of these activities, the ESD has had the following wider (positive) impacts within Member States (in your country or any other EU countries you have experience from) beyond its main aim to reduce GHG emissions (34 responses)Strongly disagreeDisagreeNeither agree nor disagreeAgreeStrongly agreeDon’t know 
 
 
 
Has the ESD affected competition within the EU, for example, by affecting EU internal 
markets? 

What other wider impacts has the ESD had within Member States that are not captured in the 
lists above? 

It was suggested that ESD has given a better awareness of GHG mitigation measures. Also, 
implemented policies such as building regulations were stated as being beneficial to SMEs as they 
carry out most of the installations. One respondent noted that it potentially reduces the impact that 
individuals can have on emission reductions via a change in lifestyle or product choice. 

Has the ESD promoted GHG emissions reductions in a cost-effective and fair manner? 

Efficiency 

As described in the section above, there are a number of benefits associated with the delivery of the 
objectives of the ESD. At the same time, there may be certain costs associated with the 
implementation of the ESD. The following questions refer both to the costs incurred to date, and to 
predicted costs to 2020. 

35 

0%10%20%30%40%50%60%70%80%90%100%Creation of new barriers to the free movement of goods and serviceswithin Member StatesCreation of new barriers to the free movement of goods and servicesbetween Member StatesCreation of new barriers to the free movement of goods and servicesbetween Member States and markets outside the EUReduction in consumer choice, by providing barriers for new suppliersand service providersAs a result of these activities, the ESD has had the following wider (negative) impacts in Member States beyond its main aim to reduce GHG emissions (34 responses)Strongly disagreeDisagreeNeither agree nor disagreeAgreeStrongly agreeDon’t know151114030246810121416StronglydisagreeDisagreeNeither agreenor disagreeAgreeStrongly agreeDon’t knowThe ESD has been effective in promoting reductions in GHG emissions in a cost-effective and fair (in terms of effort distribution among Member States) manner (responses = 34) 
 
 
 
 
 
 
 
What costs have been associated with the implementation of ESD? 

Are  there  any  costs  which  would  have  been  incurred  anyway  due  to  existing  GHG  reporting 
obligations?  

Three respondents indicated that most  of the administrative costs associated with the ESD would 
have  occurred  anyway  in  response  to  other  reporting  requirements.  However,  some  stakeholders 
identified  specific  costs,  including  the  annual  inventory  review,  as  being  additional.  Two 
respondents  stated  that  none  of  the  costs  would  have  been  incurred  anyway,  although  no  further 
details were provided on the specific costs incurred. This would suggest that the ESD is associated 
with  additional  administrative  costs,  but  at  least  a  proportion  of  these  would  have  been  incurred 
anyway. 

Are the costs of ESD implementation proportionate to the benefits from the ESD? 

What opportunities exist to reduce the costs of implementation, whilst also delivering a similar 
level of benefits? 

Several stakeholders suggested options for reducing the costs of implementation of the ESD, whilst 
also  delivering  a  similar  level  of  benefits.  Most  of  these  related  to  the  reporting  and  compliance 
requirements,  although  some  others  related  to  wider  issues.  In  relation  to  the  reporting  and 
compliance  requirements,  several  stakeholders  suggested  that  the  compliance  checks  could  be 
made  less  frequent,  although  one  cautioned  that  this  may  reduce  the  effectiveness.  Other 
respondents  suggested  there  may  be  options  to  develop  tools  for  more  efficient  reporting,  and 
further harmonisation with other international reporting requirements. In relation to the wider costs 
of delivery, one stakeholder suggested that the creation of a more transparent and liquid market for 
AEAs would help reduce the overall costs. Another stakeholder suggested that raising awareness of 
the  ESD  at  EU  level  would  result  in  greater  discussion  at  national  level,  which  would  facilitate 
greater  action.  Another  suggestion  was  to  share  best  practice  more  widely  amongst  national 
administrators.  

What role did the flexibility mechanisms (e.g. banking and borrowing) play until now? 

Stakeholders were in agreement that flexibilities had played a limited role to date, although several 
respondents stated that this role will increase in the  future. The reasons for the limited  role were 
that Member States had not been required to use them for compliance with their emissions limits. 

36 

0%10%20%30%40%50%60%70%80%90%100%Overall, the costs associated with the implementation of theESD are proportionate to the direct and indirect benefits of theESD (e.g. reductions in GHG emissions, improvements inenvironmental quality)Strongly disagreeDisagreeNeither agree nor disagreeAgreeStrongly agreeDon’t know 
 
 
 
 
 
 
 
 
 
The  influence  of  the  economic  crises  on  emissions  was  cited  as  an  important  influence  on  this 
outcome, and this had not been expected at the outset. 

Do you think the use of the flexibility mechanisms has helped (or will help) the ESD to deliver 
emissions reduction in a cost effective and fair manner? 

All  stakeholders  agreed  that  the  flexibility  mechanisms  have  an  important  role  in  delivering 
emissions  reductions  at  the  EU28  level  in  a  cost-effective  manner.  However,  two  stakeholders 
expressed concern that the flexibilities may undermine some of the stimulus that the ESD provides 
to  take  domestic  action,  with  the  use  of  external  credits  reducing  this  pressure  further.  This  is 
consistent with concerns expressed by other stakeholders in response to other evaluation questions. 
The importance of flexibilities in evening out inter-annual fluctuations in GHG emissions (e.g. in 
response to weather) was identified as a key factors requiring the need for such flexibilities. 

Coherence 

The emission sources and sectors within the scope of the ESD are also targeted by other EU wide 
GHG PAMs, which were introduced with the ESD as part of a package of instruments to reach the 
EU's  2020  targets.  A  series  of  questions  were  asked  to  explore  whether  stakeholders  considered 
that the ESD was coherent with this package of policies, and their individual requirements. 

Is  the  ESD,  and  its  associated  activities,  coherent  with  the  current  EU  climate  and  energy 
policies in the 2020 package? 

Is the ESD coherent with national policies and strategies in the target sectors? 

Are the reporting obligations under the ESD coherent with other GHG monitoring and reporting 
obligations under EU legislation? 

37 

0%10%20%30%40%50%60%70%80%90%100%The ESD, and its associated activities, are coherent with thecurrent EU climate and energy policies in the 2020 packageStrongly disagreeDisagreeNeither agree nor disagreeAgreeStrongly agreeDon’t know0%10%20%30%40%50%60%70%80%90%100%The ESD, and its associated activities, are coherent withnational policies in the target sectorsStrongly disagreeDisagreeNeither agree nor disagreeAgreeStrongly agreeDon’t know0%10%20%30%40%50%60%70%80%90%100%Member States obligations under the ESD in relation toreporting are coherent with other GHG monitoring andreporting obligations under EU legislationStrongly disagreeDisagreeNeither agree nor disagreeAgreeStrongly agreeDon’t know 
 
 
 
 
 
 
 
Added value of EU action 

The consultation also explored the added-value of EU action as compared to Member State action 
alone.  A  number  of  the  following  questions  are  related  to  those  explored  in  the  effectiveness 
section, as they concern the overall added value of the Decision. 

Could the specific actions that have been taken, since the ESD came in to force (June 2009), to 
limit EU-wide GHG emissions by 2020, have been achieved by national-level measures alone?  

Does there continue to be a need for an instrument, such as the ESD, to promote reductions in 
GHG emissions in the ESD sectors across the EU? 

What would be the most likely consequences of stopping or not continuing the ESD? 

38 

0%10%20%30%40%50%60%70%80%90%100%The actions that have been taken, since the ESD came in toforce (June 2009), to limit EU-wide GHG emissions by 2020,could not have been achieved by national-level measuresalone without the ESD as overarching EU policyStrongly disagreeDisagreeNeither agree nor disagreeAgreeStrongly agreeDon’t know0%10%20%30%40%50%60%70%80%90%100%There continues to be a need for an instrument, such as theESD, to promote reductions in greenhouse gas (GHG)emissions in the ESD sectors across the EU, in a cost-effectiveand fair mannerStrongly disagreeDisagreeNeither agree nor disagreeAgreeStrongly agreeDon’t know0%20%40%60%80%100%Member States will take less action to reduce GHGemissions in sectors/sources covered by the ESDMember States will take the same level of action to reduceGHG emissions in sectors/sources covered by the ESDMember States will take a greater action to reduce GHGemissions in sectors/sources covered by the ESDGHG emissions reductions in ESD sectors will developmore slowly or with reduced scopeGHG emissions reductions in ESD sectors will develop atthe same pace with the same scopeGHG emissions reductions in ESD sectors will developmore quickly or with greater scopeWhat would be the most likely consequences of stopping or not continuing the ESD? (34 responses) Strongly disagreeDisagreeNeither agree nor disagreeAgreeStrongly agreeDon’t know 
 
 
 
 
 
 
 
ANNEX 3: METHODS AND ANALYTICAL MODELS USE IN PREPARING THE 
EVALUATION 

The ESD came into force in June 2009, and the scope of the evaluation concerns the impacts of the 
Decision  to  November  2015.  While  the  ESD  has  been  in  place  for  some  time,  some  of  the  main 
requirements have only happened for the first time in 2015, and others will not happen until 2016. 
The methodology has been designed to gather evidence on the performance of the ESD to date, but 
the limited evidence on some elements of the ESD needs to be recognised.   

The  following  methods  were  selected  as  most  appropriate  for  the  external  evaluation,  and  were 
subsequently implemented, including analysis of evidence: 

  Policy literature and data review 
  Questionnaire surveys 
  EU public consultation 
  Stakeholder interviews 
  Case studies 
As far as possible, evidence has been gathered from multiple sources, to enable the comparison and 
triangulation of key findings. Each of the methods is described briefly below. 

i.  Policy literature and data review 

This is a key source of evidence. It includes policy documents, data reported by Member States as 
part  of  the  on-going  monitoring  of  the  Decision  and  wider  literature  relevant  to  the  intervention 
logic.  

1. 

Sources of evidence 

A  key  source  of  evidence  that  has  informed  the  evaluation  are  the  reports  submitted  by  Member 
States under the Monitoring Mechanism Decision and, starting from 2015, Monitoring Mechanism 
Regulation (MMR). Member States are required to submit information on GHG inventories (Art 7), 
projections  (Art  14),  PAMs  (Art  13),  projections  and  information  on  the  use  of  flexible 
mechanisms.  This  information  is  aggregated  by  the  EEA  in,  among  others,  the  ‘Trends  and 
projections in Europe’ report and the EEA database on national climate change mitigation policies 
and measures (PAMs) in Europe.42 

Policy  documents  provide  evidence  on  the  EU  climate  policy  context,  which  is  valuable  for 
assessing the relevance and  coherence  of the ESD. For example, the 2008 Impact Assessment of 
the  Climate  and  Energy  Package  provides  important  evidence  on  the  expected  impact  of  the 
Decision prior to its implementation, and its interaction with the other policies in the Climate and 
Energy Package.  

The  literature  review  has  also  drawn  upon  existing  research  into  the  historical  drivers  of  GHG 
emissions across Europe, including modelling studies. This included previous research by the EEA 
on the different drivers of GHG emissions in the EU, as well as the initial results from an on-going 

42 A detailed analysis of national policies and measures is provided in appendix 3 to the study supporting the evaluation, 
Supporting  study  for  the  Evaluation  of  Decision  No.  406/2009/EC  (Effort  Sharing  Decision),  Ricardo  Energy  and 
Environment with Trinomic and Vito. 

39 

 
 
 
                                                 
CLIMA study on the same issue. The existence of this research also removed the need for any new 
modelling work to be performed as part of this evaluation study.43  

Finally, there are number of previous studies that have reviewed and compiled information on the 
implementation of PAMs within Member States. 

The  evaluation  was  therefore  able  to  benefit  from  a  well-established  monitoring  framework  in 
relation to GHG emission trends and policies measures, and also a strong body of previous research 
on  GHG  emission  drivers,  and  national  policy  measures.  This  reduced  the  need  for  extensive 
collection of new data, or modelling. However, it was still necessary to check and synthesise the 
data, and then validate the research conclusions with other sources. It was also necessary to identify 
additional data sources to fill certain gaps in data. 

To fill data gaps a wider review of relevant publications and research reports was performed. This 
included a review of relevant publications, articles and web sites.44 

2. 

Limitations in evidence and key uncertainties 

As far as possible, evidence was gathered for each of the evaluation questions. However, for some 
of the questions, for example, that relating to the wider impacts of the ESD, the evidence from the 
literature was more limited. This in particular related to evidence that was specific to the ESD i.e. 
studies that specifically related to the ESD, rather than climate policies more generally. To address 
this limitation, the analysis drew upon broader literature on the impacts of climate change policies 
more generally, in sectors within the scope of the ESD. This assumes that this broader literature is 
representative of the impacts associated with the ESD, including impacts associated with national 
policies  introduced  in  response  to  the  ESD.  Whilst  this  is  a  plausible  assumption,  given  that  the 
ESD  does  not  specify  how  Member  States  should  meet  their  emission  limits,  it  has  not  been 
possible  to  test  it.  Therefore,  where  evidence  has  drawn  upon  these  more  general  sources  the 
conclusions  should  be  treated  with  greater  uncertainty.  In  particular,  it  is  uncertain  whether  the 
impacts may have arisen as a result of the ESD, or as a result of other drivers.  

The literature review also aimed to gather evidence on the effectiveness of the ESD for the EU as a 
whole,  but  then  to  validate  the  findings  with  evidence  for  individual  Member  States.  This 
validation was possible for some evaluation questions and not all. Likewise, where the analysis has 
drawn upon results for individual Member States the results cannot be assumed to be representative 
for other Member States, or the EU as a whole. Nevertheless, the results from the national studies 
are  generally  supportive  of  the  results  from  the  EU  wide  analysis,  and  where  discrepancies  have 
been  identified  these  have  been  clearly  described.  Where  validation  has  not  been  possible, 
conclusions should be considered more uncertain. 

Some  key  sources  of  evidence  are  reports  and  the  results  from  previous  modelling  work,  which 
have been produced by other organisations. It has not been possible within the scope of the current 
study to critically appraise these studies, and their limitations. 

ii.  Questionnaire surveys 

1. 

Sources of evidence 

To gather the views and opinions of stakeholders in relation to the individual evaluation questions 
two targeted questionnaire surveys have been used. The first was a general survey which covered 

43 The results of the decomposition analyses are discussed in more detail in Appendix 4 to the study supporting 
the  evaluation,  Supporting  study  for  the  Evaluation  of  Decision  No.  406/2009/EC  (Effort  Sharing  Decision),  Ricardo 
Energy and Environment with Trinomic and Vito. 
44  A  full  bibliography  of  all  references  used  for  the  study  supporting  the  evaluation  is  provided  in  Appendix  6  to  that 
study. 

40 

 
 
                                                 
 
all  of  the  evaluation  questions,  and  was  targeted  at  all  stakeholder  groups  (as  listed  below).  The 
second was a specific survey on the administrative burdens associated with the implementation of 
the ESD’s reporting requirements by Member States, such as reporting obligations.  

Based upon previous studies, for example capacity building workshops on GHG mitigation policies 
in ESD sectors that DG CLIMA organised for Member States in 2014, it was recognised that the 
population  of  individuals  directly  involved  in  ESD  implementation  at  a  national  level  was  rather 
limited.  Therefore,  it  was  not  expected  that  the  questionnaire  survey  would  have  a  very  high 
response  rate  (in  comparison  to  consultations  on  policies  with  a  more  direct  impact  on  different 
stakeholder  groups,  such  as  product  regulations,  for  example).  The  priority  for  the  general 
questionnaire  was,  therefore,  to  achieve  a  good  coverage  of  those  individuals  who  have  been 
directly  involved  in  the  ESD  implementation.  At  the  same  time,  the  aim  was  to  ensure  other 
stakeholders  groups,  including  non-governmental  organisation  and  business  association,  also  had 
the  opportunity  to  participate.  It  was  though  recognised,  based  on  experiences  with  the  capacity 
building workshops, that the interest from these groups was likely to be weaker. 

A  priority  list  of  stakeholders  was  developed,  comprising  individual  and  organisations  that  were 
known to have been directly involved in ESD implementation, along with a secondary list of wider 
stakeholders. The stakeholder list drew upon the contacts list for the capacity building workshops, 
and was supplemented through further desk research. 

General survey 

The general survey was aimed at four stakeholder groups: 

  EU  policy  makers  i.e.  individuals  within  the  EC  and  the  EEA  who  may  be  involved  in  the 

development or implementation of the ESD and related policies; 

  National ESD administrators i.e. the individuals involved in the implementation of the ESD in 

Member States; 

  National policy makers i.e. the individuals within national, regional and local governments who 

may be involved in the development or implementation of the policy; and 
  Wider interest groups (industry associations, academia and civil society) 

Administrative burdens survey 

A separate survey was used for gathering data on the administrative costs, in particular with respect 
to Member States’ reporting obligations under the ESD. Since the cost of implementing most of the 
requirements  of  the  ESD  is  only  likely  to  be  known  by  a  small  group  of  individuals  in  each 
Member  State,  a  tailored  questionnaire  survey  was  developed  and  sent  to  national  administrators 
with each of the Member States. 

In  the  assessment  of  the  administrative  burdens  it  was  important  to  take  account  of  the  overlaps 
with  other  Member  State  reporting  obligations.  In  particular,  it  was  necessary  to  determine  what 
activities  were  Member  States  obliged  to  perform  anyway  as  part  of  their  EU  and  International 
reporting obligations, and what requirements were solely arising from the ESD.  

A  total  of  9  Member  States  replied  to  this  survey:  Austria,  Cyprus,  Denmark,  Estonia,  Finland, 
Malta, The Netherlands, Sweden and Slovenia. Estimates were also provided by the EEA and the 
EC of their administrative costs. 

2. 

Limitations in evidence and key uncertainties 

The main limitation with the general questionnaire is that it is based on the views and opinions of 
the  participating  respondents.  This  presents  a  risk  that  the  results  can  be  biased  both  by 
unrepresentative samples, but also by protest responses. In relation to the first point the sample size 
for the general questionnaire was quite low, but this was not surprising given the specialist nature 

41 

 
 
 
of  the  policy.  Reasonable  coverage  was  achieved  from  the  primary  target  group  (national 
governments), with responses from 17 of the 28 EU Member States. This included older and newer 
Member  States,  larger  and  smaller  Member  States,  and  a  good  geographic  spread.  Also  a  good 
response rate was received from the non-governmental organisations active in the climate change 
policy area. A more limited response rate was received from business associations. This means that 
the results are more strongly biased towards the views of national governments, and secondly non-
governmental organisations. The view of business associations may be under represented, although 
these organisation were given equal opportunity to respond which may suggest weaker interest in 
the evaluation from this stakeholder group. 

The general survey was sent initially to a short-list of priority target stakeholders including those 
individuals  involved  in  the  implementation  of  the  ESD  within  national  governments,  individuals 
within  the  EC  and  its  agencies  that  have  been  involved  in  ESD  related  activities,  and  also  non-
governmental  organisations  and  business  associations  that  had  been  involved  in  previous  ESD 
related activities (e.g. workshops). However, the initial response rate to the questionnaire was very 
low.  This  could  be  linked  to  two  factors  1)  the  questionnaire  was  issued  shortly  after  a  related 
public  consultation  (see  below)  so  respondents  were  asked  to  complete  two  surveys  on  similar 
issues 2) the questionnaire was issued during the summer holiday season which is typically a more 
difficult time to achieve high response rates. To address the low response rates the timescales for 
the questionnaire were extended, and the invitation to participate in the questionnaire was issued to 
an extended mailing list (224 invitees in total). In addition, where responses were not received from 
the priority stakeholder list,  a telephone interview was offered to the stakeholders (see below) as 
alternative means of contributing to the evaluation.  

Despite  these  efforts  a  large  number  of  respondents  who  were  invited  to  participate  rejected  the 
offer.  In  relation  to  the  primary  target  audience  some  national  governments  (2  Member  States) 
stated  explicitly  that  they  did  not  wish  to  participate  in  the  survey.  In  the  remaining  cases  no 
response  was  received  despite  numerous  prompts.  In  7  cases  respondents  begun  to  complete  the 
survey,  but  then  did  not  proceed  further  once  they  had  understood  better  the  main  topics  and 
questions  that  were  covered  by  the  survey.  These  incomplete  responses  largely  came  from 
stakeholders  from  the  expanded  list.  This  reinforces  the  point  that  the  subject  matter  was  rather 
specialist in nature, and the cohort of individuals with relevant expertise was relatively small. 

No obvious responses were identified that could be considered a protest response.  

Similar  limitations  exist  with  the  administrative  burdens  questionnaire.  The  overall  response  rate 
was relatively low (9 of the 28 Member States). This may have reflected the timing of the survey. 
The  individuals  within  national  government  whose  input  was  required  were  very  busy  finalising 
their national inventories for submission to the UNFCCC at the time of the consultation. A further 
limitation relates to the potential bias in the responses. One response to the questionnaire included a 
much  larger  estimate  of  the  number  of  days  required  to  perform  the  activities,  than  for  the  other 
responses. To address this potential bias, this estimate was excluded when aggregating the costs at 
an EU level. See Appendix 2. 

iii.  EU public consultation 

1. 

Sources of evidence 

The Commission launched a public consultation on the preparation of a legislative proposal on the 
effort  of  Member  States  to  reduce  their  GHG  emissions  to  meet  the  European  Union's  GHG 
emission reduction commitment in a 2030 perspective. The consultation ran from 26 March 2015 
to 18 June 2015. As part of this consultation there were questions relating to the current ESD. The 
responses  to  this  consultation,  where  relevant  to  the  evaluation  questions,  have  been  taken  into 
account as part of the analysis.  This comprised between 78 and 100 responses (depending upon the 
question)  

42 

 
 
2. 

Limitations in evidence and key uncertainties 

Similar limitations exist to those described for the general survey above, with the potential for bias 
in the results. However, since the results from the public consultation were not focussed on specific 
evaluation questions, the evidence is generally less relevant, so has had a more limited influence on 
the overall conclusions. 

iv.  Stakeholder interviews 

1. 

Sources of evidence 

Stakeholder  interviews  allow  a  more  detailed  exploration  of  the  evaluation  questions.  These 
interviews 
the  wider 
statistics/indicators. The interviews were also used to develop case studies which illustrate specific 
outcomes. 

important  contextual 

therefore  provide 

to  explain 

information 

An  interview  plan  was  developed  with  targeted  representation  from  each  of  the  different 
stakeholder  groups,  as  well  as  stakeholders  in  different  circumstances.  For  example,  interviews 
with  Member  States  included  those  countries  that  had  been  projected  to  meet  their  ESD  targets 
without the need for new policies since its introduction, but also countries which are expected to 
miss their targets without further action. 

In many cases the stakeholder interviews were held with the same individuals that completed the 
general questionnaire. In total 24 interviews were held with stakeholders. Of these, 18 were follow-
up interviews with selected respondents to the questionnaires survey. In these cases the interviews 
allowed  more  detailed  exploration  of  the  issues  raised  in  the  questionnaire.  The  remaining  6 
interviews  were  with  stakeholders  who  had  not  completed  the  questionnaire  survey.  These  are 
highlighted as new stakeholders in the table below. 

2. 

Limitations in evidence and key uncertainties 

The  main  limitation  with  the  evidence  from  the  stakeholder  interviews  is  that  is  based  on  the 
opinions of the stakeholders, and not empirical evidence. Nevertheless, these views still provide a 
valuable  form  of  evidence,  particularly  for  those  evaluation  questions  that  cannot  be  tested 
empirically.  Furthermore,  the  interviews  were  valuable  in  identifying  additional  sources  of 
research, as well as validating and further explaining the findings from the literature review.  

As  with  the  general  questionnaire,  the  results  from  the  stakeholder  interviews  can  suffer  from 
potential  bias.  This  was  mitigated  by  ensuring  that  the  sample  covered  each  of  the  relevant 
stakeholder  groups,  and  also  relevant  stakeholder  types  within  each  stakeholder  group.  For 
example, the interviews with national governments included representatives from large and small 
Member States, from EU-12 and EU16, and from Member States who are projected to meet their 
ESD target easily, and those that need to implement additional measures. This distribution helps to 
ensure that views are representative. However, for other stakeholder groups the sample size cannot 
be considered representative. In particular, the views of business associations are potentially under-
represented, although the lack of interest from this group may indicate that views are less strong in 
general.  This  view  was  supported  by  the  interviews  themselves,  with  a  generally  lower  level  of 
understanding  and  awareness  of  the  ESD  from  this  group.  This  potentially  reflects  the  lack  of  a 
direct impact of the ESD on businesses, with any impacts arising indirectly though national policies 

The overall sample size for the stakeholder interviews was relatively low, so there remains a risk of 
bias towards the views held by certain stakeholders. However, the views from stakeholder did not 
indicate  a  strong  bias  for  most  evaluation  questions.  A  possible  exception  was  that  non-
governmental organisations tended to highlight more strongly the benefits of climate policy and the 
need  for  stronger  mitigation  action.  However,  this  view  was  not  exclusive  to  non-governmental 
organisations, and was balanced out in most cases by views of other stakeholder groups. 

43 

 
 
v.  Case Studies 

1. 

Sources of evidence 

Case  studies  were  used  to  illustrate  interesting  responses  to  some  of  the  issues  covered  in  the 
evaluation. The case studies were identified  based on responses to the general questionnaire, and 
also through the review of wider literature. The evidence to complete the case studies was therefore 
based  on  published  literature,  which  was  then  updated  and  validated  though  further  engagement 
with relevant stakeholders.  

The case studies were used to illustrate practical examples relating to the different aspects of the 
intervention  logic.  In  particular,  the  case  studies  were  used  to  explore  the  wider  impacts  of  the 
Decision – since these impacts are difficult to quantify and may be context specific.  

Three cases studies were developed, as shown below. Each was used to provide answers to some of 
the evaluation questions.  

Table 1: Case studies for ESD evaluation 

Name 

Short description 

Relevant evaluation questions 

1.  The Flemish 
Climate Fund 

2.  The Austrian 

Climate 
Protection Act 

3.  Spain – 2020 
roadmap for 
ESD emissions 

The  Flemish  Climate  Fund  is 
a  response  of  the  Flemish 
government in Belgium to the 
fact 
(and 
that  Belgium 
Flanders)  are  not  on  track  to 
achieve  the  ESD  target  in 
2020.  

To  what  extent  have  Member 
States  implemented  new  policies 
at  a  national 
to  reduce 
emissions in ESD sectors? 

level 

What role did flexibility instruments 
play until now? Are Member States 
using or planning to use them? 

Climate 
Austrian 
The 
the 
Act 
is 
Protection 
for 
legislation 
framework 
reducing  emissions  in  ESD 
sectors.  We  also  look  into 
how  cooperation  between 
regional/local 
national  and 
governments in the context of 
the ESD are organised.  

To  what  extent  have  Member 
States  implemented  new  policies 
at  a  national 
to  reduce 
emissions in ESD sectors? 

level 

and 

How  well  is  cooperation  between 
regional/local 
central 
in  Member 
authorities  working 
States  on  emissions  reductions  in 
ESD sectors?  

to  achieve 

In 2014, Spain was projected 
not 
the  ESD 
target.  In  response  to  this 
some  smaller  PAMs  have 
been 
in 
combination  with  a  2020 
roadmap  for  ESD  emissions, 
which  are  described  in  the 
case study.  

implemented 

To  what  extent  have  Member 
States  implemented  new  policies 
at  a  national 
to  reduce 
emissions in ESD sectors? 

level 

and 

How  well  is  cooperation  between 
regional/local 
central 
authorities  working 
in  Member 
States  on  emissions  reductions  in 
ESD sectors?  

2. 

Limitations in evidence and key uncertainties 

The main limitation with the case studies is that the findings are relevant for the particular case, but 
cannot be consider representative of other Member States, or the EU as a whole. The case studies 
are  therefore  illustrative  of  specific  examples,  but  cannot  be  considered  to  cover  either  the  full 

44 

 
 
range of responses from Member State, or necessarily reflect a representative response. Indeed the 
selection  of  the  case  studies  has  an  inherent  bias  towards  those  Member  States  that  have  taken 
action in response to the ESD. A case study on a Member State that are taking no or limited action 
has not been prepared. Thus the case studies are necessarily biased towards cases where action has 
been  taken,  and  the  intervention  logic  can  be  demonstrated.  Whilst  this  is  valid  –  and  provides 
valuable  insights  into  how  the  intervention  logic  has  operated,  this  potential  bias  should  be 
recognised.  

For certain issues, such as the use of flexibility mechanisms there is insufficient experience to date 
to provide a useful case study. The case studies were therefore selected to cover the aspects of the 
intervention logic where further evidence from case studies was considered to be most valuable and 
sufficient  evidence  available  to  develop  valuable  findings.  Nevertheless,  the  case  study  evidence 
only covers certain issues and is not comprehensive.  

45 

 
 
 
 
 
 
 
ANNEX 4: PROVISIONS FOR FLEXIBILITY INSTRUMENTS AND MONITORING AND 
COMPLIANCE IN THE EFFORT SHARING DECISION 

Flexibility instruments 

The  ESD  allows  a  Member  State  to  carry  over  (“bank”)  any  surplus  annual  emission  allocations 
(AEAs) from a given year to subsequent years within the commitment period (until 2020). It can 
also carry forward (“borrow”) up to 5 % of its annual emission allocation from the following year. 
In this way, Member States can either cover emissions in excess of their AEAs in a given year by 
borrowing AEAs from allocations for future years, or, where the emissions of a Member State are 
below that annual emission allocation, it can bank unused AEAs from one year in order to offset 
future emissions in excess of the AEAs allocated in future years (Article 3.3 in the ESD).  

These  instruments  provide  flexibility  to  Member  States  within  their  own  schedule  of  annual 
allocations  in  2013-2020  and  help  them  managing  the  use  of  their  AEAs  over  the  whole 
commitment  period  to  cover  any  AEA  shortage  in  specific  years,  for  example,  due  to  weather 
fluctuations. A higher borrowing rate is available to Member States for the years 2013-2014 under 
the current ESD in the event of “extreme meteorological conditions” (Article 3.3).  

Member States can also trade with each other in the ESD. A Member State is allowed to transfer up 
to 5 % of its AEAs for a given year to another Member State at any time (Article 3.4). After the 
annual  compliance  check  has  been  concluded  it  can  transfer  its  entire  surplus  AEAs  for  a  given 
year (Article 3.5). A receiving Member State may use these acquired AEAs for complying with its 
obligation  under  ESD  for  the  given  year  or  any  subsequent  years  of  the  compliance  period.  A 
Member State cannot transfer any part of its annual emission allocation if, at the time of transfer, 
that  Member  State  is  not  in  compliance  with  the  requirements  of  the  ESD.  Member  States  are 
obliged  to  report  any  concluded  transfers  to  the  European  Commission,  which  then  publishes  a 
summary of such transfers as part of its annual progress report. 

Finally, under Article 5 of the ESD, Member States can also use international project credits under 
the  Kyoto  Protocol’s  Clean  Development  Mechanism  and  Joint  Implementation  to  meet  their 
commitments under the ESD. The use of such credits is limited on a yearly basis up to 3% of 2005 
non ETS emissions in Member State. Member States that do not use their 3% limit in any specific 
year can transfer their unused part for that year to other Member States or bank it for own future 
use. Member States, which fulfil further criteria (AT, BE, DK, FL, IE, ES, IT, CY, LU, PT, SI, SE) 
may  use  CDM  project  credits  up  to  an  additional  1%  of  their  verified  emissions  in  2005;  these 
credits are not bankable and not transferable.45 

Monitoring of progress and compliance 

Monitoring  of  progress  and  assessing  compliance  is  based  on  a  comprehensive  framework  of 
monitoring,  reporting  and  verification  (MRV)  laid  down  partly  in  the  ESD  itself  (Article  6)  and 
partly  in  the  Monitoring  Mechanism  Regulation46  (MMR)  and  its  implementing  provisions. 
Member  States  are  obliged  to  annually  report  their  greenhouse  gas  emissions  to  the  European 
Commission.  The  MMR  and  ESD  also  require  Member  States  to  report  every  second  year  on 
national policies and measures implemented in order to achieve their targets under the ESD and on 
their emission projections.  

To ensure that the compliance assessment relies on accurate and verified data, the GHG emissions 
inventories  submitted  by  Member  States  are  reviewed  by  the  Commission  with  support  by  the 
European Environment Agency as laid down in the MMR. The review includes quality assurance 
and  quality  control  of  Member  States  greenhouse  gas  emissions  inventories.  A  team  of  technical 

45 Such additional CDM project credits must be from less developed countries and small island states. 
46  Regulation  (EU)  525/2013  of  the  European  Parliament  and  the  Council  of  21  May  2013  on  a  mechanism  for 
monitoring  and  reporting  greenhouse  gas  emissions  and  for  reporting  other  information  at  national  and  Union  level 
relevant to climate change and repealing Decision No 280/2004/EC. 

46 

 
 
                                                 
review  experts  co-ordinated  by  the  Agency  checks  the  transparency,  accuracy,  consistency, 
comparability and completeness of the information submitted.  

The annual compliance assessment is performed by the Commission by comparing the reported and 
reviewed emissions with the annual emission limits under the ESD. Before the compliance check, 
Member  States  can  make  use  of  the  flexibility  instruments  (e.g.,  banking  and  borrowing,  AEA 
transfers from other countries) to close any gap between their actual emission and emission limits 
for a particular year. 

If  a  Member  State  would  still  not  be  in  compliance  after  applying  the  flexibility  instruments,  a 
deduction  from  its  emission  allocation  of  the  following  year  equal  to  the  excess  emissions  (in 
tonnes of carbon dioxide equivalent) multiplied by an abatement factor of 1.08 is applied. 

[

C

a

t

a

l

o

g

u

e

n

u

m

b

e

r

]

47