Towards a harmonised methodology for statistical indicators. Part 3, Relevance of indicators for policy making : 2017 edition.
Date: 2017-07-10
Institutions: Eurostat
Subjects: statistical method, statistics

Towards a harmonised 
methodology for 
statistical indicators 
PART 3 — RELEVANCE OF INDICATORS FOR 

POLICY MAKING  2017 edition 

Printed by the Publications Office in Luxembourg 

Manuscript completed in May 2017 

Neither the European Commission nor any person acting on behalf of the Commission is responsible for the use 
that might be made of the following information. 

Luxembourg: Publications Office of the European Union, 2017 

© European Union, 2017  

Reuse is authorised provided the source is acknowledged.  

The reuse policy of European Commission documents is regulated by Decision 2011/833/EU  
(OJ L 330, 14.12.2011, p. 39). 

Copyright for the photograph of the cover: ©Shutterstock: Rawpixel.com. 

For any use or reproduction of photos or other material that is not under the EU copyright, permission must be 
sought directly from the copyright holders. 

For more information, please consult: http://ec.europa.eu/eurostat/about/policies/copyright 

Print 
PDF 

ISBN 978-92-79-68872-0 
ISBN 978-92-79-68871-3 

ISSN 2363-197X 
ISSN 2315-0815 

doi:10.2785/845509 
doi:10.2785/53076 

KS-GQ-17-007-EN-C 
KS-GQ-17-007-EN-N 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Foreword 

Foreword 

Indicators more and more play a fundamental role in our modern societies and constitute an essential 
resource  for  policymakers,  business  leaders  and  the  general  public.  They  support  evidence-based 
decision making, allow comparisons over time, between policies, countries and regions, social groups 
and industries, and contribute to increased transparency and accountability.   

To  fulfil  this  key  role,  indicators  must  be  'fit for purpose' and  must  be  based  on  statistics  that  meet 
defined quality requirements. European statistics as produced and disseminated by Eurostat and the 
EU national statistical institutes meet the highest quality requirements. Being based on high quality 
statistics  is  a  necessary  but  not  a  sufficient  condition  to  make  indicators  policy  relevant.    Official 
statisticians,  researchers  and  experts  must  be  aware  of  the  implications  of  the  increasing  role  of 
statistical indicators in policy making. They have to engage with decision-makers to understand their 
needs and the expectations, but also to provide guidance for a correct interpretation and use of these 
indicators, and for their limitations.  

This paper is the last in a series of three papers on statistical indicators published by Eurostat. The 
first paper on 'Indicator typologies and terminologies' was published in 2014. The second paper on 
'Communicating through indicators' was released in February 2017. 

This  third  paper,  as  the  whole  series,  aims  to  help  professionals  who  develop,  produce  and 
communicate policy indicators. It analyses how indicators are used at particular stages of the policy 
making  process;  it  considers  the  implications  of  the  increasing  role  of  statistical  indicators  in  policy 
making for official statisticians and indicator developers; finally, it includes recommendations on how 
to develop and maintain policy relevant indicators. And in describing the opportunities and pitfalls of 
indicator use in the policy making process, the paper will also be relevant for policymakers using the 
indicators. 

Mariana Kotzeva, 
Acting Director General, Eurostat

Towards a harmonised methodology for statistical indicators 

3 

 
 
 
 
 
 
 
 
  
 
Acknowledgements 

Acknowledgements 

Chief editor 

Mariana Kotzeva, Acting Director General, Eurostat 

Executive editor 

Barbara  Margarethe  Kurkowiak  (Eurostat,  Unit  E2  —  Environmental  statistics  and  accounts; 
sustainable development) 

Editors 

Lucas Porsch, Terri Kafyeke (Ecologic Institute), Markus Hametner (WU Vienna) 

Co-editors 

Nicola  Massarelli,  Anton  Steurer  (Eurostat,  Unit  E2  —  Environmental  statistics  and  accounts; 
sustainable development) 

The paper also draws from material prepared by the Swiss Federal Statistical Office (FSO) for a course 
on 'Development and Use  of Indicator Systems for Evidence-Based Decision Making' co-organised 
with Eurostat in 2015 and 2016 as part of the European Statistical Training Programme (ESTP).  

Expert advisors 

Marleen  De  Smedt,  John  Verrinder  (Eurostat);  Dorota  Panczyk  (DG  DEVCO);  Ingo  Kuhnert  (DG 
ECFIN);  Barbara  Bacigalupi  (DG  ENV);  Marie  Agnes  Deleglise  (JRC);  Paolo  Bolsi  (DG  MOVE); 
Fabienne Lefebvre (DG SANTE); Vincent Willi, Seraina Pedrini (The Swiss Federal Statistical Office 
— FSO) 

Production 

Renate Ruech, Anna Dimitrova (WU Vienna), Carolyn Avery (ENDS, Haymarket Media Group Ltd) 

Dissemination 

Catherine Coyette (Eurostat) 

For more information please consult 

Eurostat 
Bâtiment Joseph Bech 
5, rue Alphonse Weicker  
2721 Luxembourg  
http://ec.europa.eu/eurostat 
E-mail: estat-user-support@ec.europa.eu 

Disclaimer 

All statements on policies within this publication are given for information purposes only. They do not 
constitute an official policy position of the European Commission and are not legally binding. To know 
more about such policies, please consult the European Commission’s website at: http://ec.europa.eu.

Towards a harmonised methodology for statistical indicators 

4 

 
 
 
 
 
 
 
 
 
 
 
 
Contents 

Contents 

Foreword ........................................................................................................................................ 3 

Acknowledgements ...................................................................................................................... 4 

Contents ......................................................................................................................................... 5 

Introduction ................................................................................................................................... 7 

1. 

Indicators and road signs in the policy making process .................................................. 9 

Indicators as a special statistical product ................................................................................ 9 

Existing definitions of indicators ............................................................................................ 10 

Opinion-based vs. evidence-based policy making ................................................................ 11 

2.  Why are indicators used in policy making? ..................................................................... 13 

Rationale for indicator use .................................................................................................... 13 

From ‘use’ to ‘influence’ and ‘impact’ .................................................................................... 15 

Indicator development for different rationales ....................................................................... 16 

3.  When and how are indicators used in policy making? ................................................... 17 

Concept of the policy cycle ................................................................................................... 17 

Using indicators for agenda setting ....................................................................................... 22 

Using indicators for policy assessment ................................................................................. 23 

Using indicators at different levels of policy making .............................................................. 24 

4.  Requirements for policy relevant indicators .................................................................... 27 

Key quality criteria for policy relevant indicators ................................................................... 28 

Achieving both policy relevance and scientific accuracy ....................................................... 30 

Key challenges of developing and maintaining policy relevant indicators ............................. 32 

5.  Recommendations .............................................................................................................. 37 

Bibliography ................................................................................................................................ 39 

Towards a harmonised methodology for statistical indicators 

5 

 
 
 
 
 
 
 
Contents 

List of tables 

Table 1: Practical examples of the use of indicators at each stage of the policy cycle ...................... 19 

List of boxes 

Box 1: The ‘Bellagio Principles’ .......................................................................................................... 29 

List of figures 

Figure 1: Statistical information infrastructure .................................................................................... 10 

Figure 2: The link between the use, influence and impact of indicators ............................................. 15 

Figure 3: The policy cycle .................................................................................................................. 17 

Figure 4: Strategic intent of statistics: improving the usability of evidence ........................................ 31 

Towards a harmonised methodology for statistical indicators 

6 

 
 
 
 
 
 
Indicators as road signs in the policy-making process 

1 

Introduction 

Indicators are important tools in the policy making process. Like road signs, they condense complex 
information into a simple pointer that can help us assess where we are headed — towards or away 
from  our  objectives.  However,  when  information  is  consolidated  in  such  a  way,  care  is  needed  to 
interpret  the  indicators  correctly  and  to  understand  what  they  do  and  do  not say. While  part of  this 
responsibility lies in the indicator user’s hands, those developing indicators also need to understand 
how and why indicators are being used, so the producers can design them in a way that best suits 
policy making.  

This paper is the third and last in Eurostat’s series ‘Towards a harmonised methodology for statistical 
indicators’.  It  complements  two  previous  publications  on  ‘Indicator  typologies  and  terminologies’ (1) 

and  ‘Communicating  through  indicators’ (2).  The  former  introduces  the  reader  to  the  concept  of 

statistical indicators by setting out their different characteristics and typologies. The latter focuses on 
the  theoretical  and  practical  aspects  of  indicator-based  communication.    This  third  paper  aims  at 
helping researchers and experts who develop indicators make these more policy relevant and provides 
a more detailed view on policy use of indicators in general.  

The  paper  analyses  how  indicators  are used  at  particular  stages  of  the  policy  making  process  and 
discusses  their  purpose  in  the  political  process,  referring  to  theoretical  and  practical  examples.  It 
considers the implications of the increasing role of statistical indicators in policy making, which official 
statisticians and indicator developers should be aware of. Most important, it includes recommendations 
on how to develop and maintain politically relevant indicators. And in describing the opportunities and 
pitfalls of indicator use in the policy making process, the paper will also be relevant for policy makers 
who use indicators.  

This paper is structured as follows: Chapter 1 provides an overview of the concept of indicators and 
introduces the aspect of their use in policy making. Chapter 2 looks at the rationale for the indicator 
use in the policy making process. Chapter 3 explores the use of indicators in relation to different stages 
of the policy cycle as well as to different geographical levels of policy making. Chapter 4 discusses 
requirements for policy relevant indicators and key challenges of their development and maintenance. 
Finally, Chapter 5 suggests potential solutions to these challenges. 

(1)  Eurostat, Towards a harmonised methodology for statistical indicators — Part 1: Indicator typologies and terminologies, 2014. 

(2)  Eurostat, Towards a harmonised methodology for statistical indicators — Part 2: Communicating through indicators, 2017. 

Towards a harmonised methodology for statistical indicators 

7 

 
 
 
 
                                                 
 
 
 
Indicators as road signs in the policy-making process 

1 

1 Indicators and road signs 

in the policy making 

process

Indicators can be considered as the road signs of policy making. Road signs help drivers to see if they 
follow  the  right  direction  on  the  way  to  their  destination.  Similarly,  indicators  help  people  to  quickly 
understand complex realities, allowing them to assess where processes are heading and if goals are 
likely to be reached. Road signs condense a map’s complex information to a simple indication ‘This 
way’ to a specific location.  

Just  like  road  signs,  indicators  are  a  simplification  of  reality;  essential information  is extracted from 
complex data sets to provide easily understandable message. This simplification helps users to absorb 
the information quickly, but it also increases the amount of context  needed for them to interpret the 
information in the correct way. 

Indicators  can  be  of  a  qualitative  or  a  quantitative  nature.  Similar,  road  signs  have  in  fact  mostly 
qualitative (e.g. a stop sign) and sometimes quantitative character (e.g. distance to a city). Whereas 
both - qualitative and quantitative indicators — are needed in the policy process, this paper deals only 
with the quantitative indicators. 

Indicators as a special statistical product  

Official statistics are produced through data collection conducted by national statistical institutes and 
presented in a clear format, often based on accounting frameworks. Data is information which can be 
collected via traditional statistical activities (for example, sample surveys and censuses) or from other 
statistical  sources  and can be  differently  used  to serve  different purposes.  Accounting  systems  are 
coherent  and  integrated  sets  of  accounts,  balance sheets  and  tables  that  conform  to  agreed  rules. 
They  allow  data  to  be  compiled  and  presented  in  a  consistent  and  comparable  way,  suitable  for 
analysis and policy making (3). These agreed processes and standards such as appropriate codes of 

practice, defined international standards, as well as the respect of impartiality and objectivity provide 
quality  assurance  to  the  data  users.  However,  these  tabulated  data  do  not  provide  the  context 
necessary to understand their meaning for specific policy questions or issues. Indicators, unlike the 
data  from  which  they  are  derived,  are  measures  designed  for specific  purposes  determined  by  the 
context (see Figure 1). The same data can serve as a source to derive different indicators providing 
answers  to  different  questions.  Indicators  thus  provide  condensed  information,  encompassing  a 
quantitative measure and a contextual element. 

(3)  See Annex to Regulation (EU) No 99/2013 of the European Parliament and of the Council of 15 January 2013 on the European 

Statistical Programme 2013-2017, Official Journal of the European Union, L 39, 9.2.2013. 

Towards a harmonised methodology for statistical indicators 

9 

 
 
 
                                                 
 
 
 
Indicators as road signs in the policy-making process 

1 

Figure 1: Statistical information infrastructure 

Source:  Adapted from the Annex to Regulation (EU) No 99/2013 of the European Parliament and of the Council of 15 January 2013 on 

the European Statistical Programme 2013–2017, Official Journal of the European Union, L 39, 9.2.2013. 

Developers of indicators consequently need to solve a trade-off between the amount of information the 
intended audience can handle and the amount they need to understand the problem, as shown in the 
pyramid above.  

Existing definitions of indicators 

Indicators are measures designed for a specific purpose and there are several ways in which they can 
be defined. The term itself has its roots in the Latin words indicare (to point out, to show, to indicate) 
and index, literally meaning ‘anything used for pointing’ and ‘the finger used for pointing’.  

The  European  Statistical  System  Committee  (ESSC)  defines  indicators  as  ‘a  particular  subset  of 
statistical  information,  directly  related  to  a  special  purpose  such  as  monitoring  specific  policy 
objectives’.  In  the  Regulation  No  99/2013  on  the  European  Statistical  Programme,  indicators  are 
described as ‘a summary measure related to a key issue or phenomenon and derived from a series of 
observed  facts.  Indicators  can  be  used  to  reveal  relative  positions  or  show  positive  or  negative 
change’ (4). 

Several  other  national  and  international  organisations  provide  similar  definitions.  The  recurring 
message  is  that  indicators  are  measures  that  condense  information  on  policy  relevant  issues  to 
facilitate  decision  making.  This facilitation  element  is  essential  and  will  be  explored in  the following 
chapters. While this paper focuses on the use of indicators for policy making, it should also be noted 
that indicators are also used for other purposes such as administrative purposes.   

It  is  also  important  to  clarify  the  concepts  of  ‘indicator  set’  and  ‘indicator  system’,  as  often  several 
indicators are presented and used together. An indicator set can be defined as a list of indicators based 
on a policy reference, as opposed to an indicator system, in  which the indicators rely on a specific 
conceptual framework.  At European level, there are several  examples of indicator sets such as the 
Europe 2020 indicators (5) used to monitor the European Union (EU) growth strategy (6) and the EU 

(4)  Regulation (EU) No 99/2013 of the European Parliament and the Council of 15.01.2013. 
(5)  For more details see: http://ec.europa.eu/eurostat/web/europe-2020-indicators/europe-2020-strategy  

(6)  For more details see: https://ec.europa.eu/info/strategy/european-semester/framework/europe-2020-strategy_en 

Towards a harmonised methodology for statistical indicators 

10 

 
 
 
 
                                                 
Indicators as road signs in the policy-making process 

1 

Sustainable  Development  Indicators  (SDI) (7)  used  until  2015  and  related  to  the  EU  Sustainable 

Development Strategy (EU SDS) (8). As example of an indicator system serve the indicators derived 

from Eurostat’s framework of national accounts. However, it is worth noting that in the literature, these 
definitions  of  indicator  sets  and  systems  are  not  uniformly  used  and  indicators  systems  are  often 
referred to as indicator sets. 

Opinion-based vs. evidence-based policy 
making  

Indicators should provide ‘evidence’ or should facilitate decision making by providing the information 
needed for a good decision. It is therefore important to specify further the concept of evidence in order 
to better understand how it relates to indicators. Evidence 'concerns facts (actual or asserted) intended 
for use in support of a conclusion’ (9). Evidence should thus not be confused with a conclusion; it can 

support a conclusion, but it is not a conclusion in itself (i.e. it is a means to an end, not the end) (10). 

Furthermore, not all evidence is of the same quality and some research results are more convincing 
than others, based on factors such as the quality of the methodology (11). To sum up, evidence should 

fulfil two criteria: it should be relevant to the decision and have the required quality or accuracy for that 
decision.  

This is essential in regard to the often-stated objective of achieving ‘evidence-based' policy making 
in which decisions are based on careful consideration of the best available evidence. Central banks 
tend to use this approach when formulating monetary policy to achieve macroeconomic stability. They 
rely  on  different  macroeconomic  indicators  (for  example  employment  rate,  inflation  rate,  growth 
forecasts)  to  make  decisions.  Value  judgements  are  also  used,  especially  to  define  objectives.  But 
when it comes to finding the best way to reach these objectives, central banks  refer to quantitative 
evidence in the form of indicators.  

On  the  other  hand,  in  the  'opinion-based'  policy  making,  decisions  are  driven  by  opinions  and 
judgements that are not based on or tested against robust evidence. In this case, they are shaped by 
a selective, anecdotal or improper use of evidence (12). Opinion-based policy making could also be 

described as a qualitative assessment based on an agreed and legitimate theory. Non-evidence-based 
factors such as ‘institutional constraints, interests, ideas (including  values), and external factors like 
recessions’ (13) can also influence opinions and, with that, opinion-based decision making.  

In practice, there is always a mixture of evidence-based and opinion-based decision making which 
can be specified as 'evidence-influenced' policy making. While monetary policy might lean heavily 
towards  an  evidence-based  approach,  policy  announcements  and  policy  decisions  in  pre-election 
times might lean more towards opinion-based decision making. Politicians competing for election will 
form their decisions not only on the basis of what they are convinced will work from evidence, but also 

(7)  http://ec.europa.eu/eurostat/web/sdi/indicators 

(8)  http://ec.europa.eu/environment/sustainable-development/strategy/index_en.htm 

(9)  Lomas et al., 2005. 

(10) Oxman et al., 2009. 

(11) Id. 

(12) Segone and Pron, 2008. 

(13) Oxman et al., 2009. 

Towards a harmonised methodology for statistical indicators 

11 

 
 
                                                 
Indicators as road signs in the policy-making process 

1 

on their perception of the electorate’s opinion on what works. This helps to explain why, for example, 
many  macroeconomic  decisions  have  been  removed  from  the  election  cycle  by  assigning  them  to 
independent central banks. Monetary policy might have been very different after the introduction of the 
single currency Euro, if it had been based on the perception of inflation, which at that time exceeded 
the measured inflation by a considerable margin (14). 

The use of evidence and indicators can improve policies by: 

  helping to recognise a policy issue; 

  providing information on policy design and choice; 

  assisting with forecasting and modelling; 

  monitoring policy implementation; 

  evaluating policy impact (15). 

These five areas reflect the structure of the policy cycle, which will be discussed in Chapter 3. In the 
meantime, it is essential to note that evidence and indicators can reinforce each of these stages by 
providing  information  into  policy  making  processes;  improving  the  relevance,  efficiency  and 
effectiveness of policies or policy reforms; and legitimising decisions.  

To help EU statistics provide the information needed to support the EU policy making, the European 
Statistical Programme 2013–2017 was established by Regulation (EU) No. 99/2013 to regulate data 
collection. Its main purpose is ‘ensuring that European statistics are focused on the information needed 
to design, implement, monitor and evaluate Union policies’ (16). Providing evidence for policy making 

at  all  levels  is  therefore  one  of  the  most  important  reasons  for  collecting  information.  Of  course 
European institutions use not only indicators produced by Eurostat/European Statistical System (ESS) 
but also statistics collected by other European Commission Directorates-General (DGs) and by other 
specialised EU agencies. 

(14)  Döhring et al., 2007, p. 5.  

(15)  Segone et al., 2008. 

(16)  See Article 2 of Regulation (EU) No 99/2013 of the European Parliament and of the Council of 15.01.2013. The European Statistical 
Programme 2013-2017 is extended to 2018-2020 as indicated in the proposal for a Regulation of the European Parliament and of 
the Council COM (2016)557 from 07.09.2016 for amendment of the Regulation (EU) No 99/2013. 

Towards a harmonised methodology for statistical indicators 

12 

 
 
                                                 
Why are indicators used in policy making? 

2 

2 Why are indicators 

used in policy 

making? 

Indicators can be used for different purposes in the policy making process; for example to either arrive 
at a decision or to explain a decision to the outside world. Understanding the different purposes and 
rationales for the use of indicators in the policy making process is the first step to understand why and 
how indicators become and remain relevant in political processes.  

Rationale for indicator use  

The purposes and rationale  for indicator use in the policy process  have been analysed by different 
authors.  The  literature  defines  three broad  categories  of  rationales  by  referring  to  the  instrumental, 
conceptual and political functions of the indicators (17).  

Instrumental use refers to the use of indicators ‘as direct input to specific decisions’ (18). In this case, 

policymakers use them to steer policy along the different stages of the policy cycle. Indicators play a 
crucial role in monitoring and assessment of the impacts of existing policies on political decisions, such 
as keeping, broadening or abandoning the policy measure in question.  For instance, indicators  can 
help in the decision making stage to choose among several policy options, or in the policy assessment 
stage  to  identify  strengths  and  weaknesses  and  provide  recommendations  for  improvement.  The 
instrumental  use  can  be  illustrated  by  the  example  of  policymakers  using  CO2  emissions  observed 
over time in different sectors to set priorities for the emission-reduction policy.  

Conceptual  use  (enlightenment)  is  the  use  of  indicators  to  shape  conceptual  frameworks  for 
assessments, ways of thinking and mental models. It relates to the ‘percolation of new information, 
ideas  and  perspectives  into  the  arenas  in  which  decisions  are  made’ (19).  In  particular,  the  use  of 

indicators may help decision-makers to define a problem and to provide new perspectives and insights. 
The  conceptual  use  differs  from  the  instrumental  use  in  that  sense  that  indicators  do  not  directly 
influence a decision but generally improve the user’s knowledge and information base.  

Political use is the use of indicators as accountability tools or supporting arguments in the political 
discussion to promote specific ideas, such as sustainable development, transparency, improvement in 
social security level or trade liberalisation. For instance, indicators on the ‘working poor’ (20) showing a 

rise in poverty levels among households of employed persons might affect the plans for welfare policy. 

(17)  Lehtonen, 2008. 

(18)  Id. 

(19)  Weiss, 1999, p.471. 
(20)  Such as 'in-work at-risk-of-poverty rate' which refers to the percentage of persons in the total population declaring to be at work 
(employed or self-employed) who are at-risk-of-poverty (i.e. with an equivalised disposable income below the risk-of-poverty 
threshold. 

Towards a harmonised methodology for statistical indicators 

13 

 
 
 
                                                 
 
 
Why are indicators used in policy making? 

2 

Political  use  can  be  further  subdivided  into  three  categories.  Firstly,  legitimisation  is  the  use  of 
indicators to justify decisions that have already been taken or policies that are already in place; often 
seen in a negative light but also essential to secure acceptance of policy in a democracy. In the case 
of tactical use, an evaluation is commissioned in order to postpone decision making. This can be also 
a type of indicator misuse. Finally, symbolic use can be identified when an indicator is primarily used 
for motivating existing policy positions or when there are clear instances of non-use or misuse.  (21).  

The  rationale  behind  the  use  of  an  indicator  might  change  during  the  policy  making  process.  For 
example, the above mentioned indicator on the 'working poor' might first only be used conceptually to 
provide insight into the development of work patterns and wage structures in the labour market. If this 
leads to the identification of an existing problem, the same indicator might be used instrumentally to 
arrive at decisions on how to best influence wages and work patterns of the working poor. Finally, once 
the decisions are taken, the indicator might also be used for political reasons to defend and legitimise 
the decision.  

There are also other categorisations of rationale for indicator use which provide different perspectives 
and nuances, although many of these definitions overlap.  

Process use can be observed when people involved in the research or evaluation process change 
their behaviour or understanding based on the indicators (22). This means the indicators feed into the 

research or evaluation conducted.  

Ritual use occurs when information is collected to maintain the appearance of compliance, but that 
information  serves  no  practical  purpose,  which  leads  to  production  of  useless  reports  and 
indicators (23).  Very  often  such  ritual  use  is  caused  by  legal  obligations  and  is  sometimes  called 

imposed use (24). 

Intended  use  is  observed  in  case  when  an  indicator  is  used  for  the  specific  purpose  that  was 
considered when it was developed. Over time, however, people may start using it in different ways and 
interpreting it differently from  its initial meaning, which results in an unintended use. For example, 
gross domestic product (GDP) was designed to measure and ultimately help steer the production of 
an economy (intended use), but over time it has started being used and communicated as an indicator 
of  economic  and  even  societal  well-being  (unintended  use).  While  it  remains  a  useful  indicator  to 
measure an  economy’s  level  of  production,  it  is  generally  seen  as  unfit  and  even  misleading  when 
employed as an indicator of overall societal welfare.  

Indicators can also be misused or abused when incorrect conclusions are deliberately reached from 
the data. For instance, indicators showing a cost shift can sometimes be wilfully misinterpreted as a 
cost  savings.  ‘Programme  exits’  is  a  common  indicator  of  the  success  of  welfare  programmes. 
However, few systematic attempts are made to discover why people leave welfare programmes, for 
how  long  and  where  they  go.  In  some  cases,  individuals  are  merely  transferred  from  one  form  of 
income support programme to another’ (25). Such misuse of indicators can happen more easily if no 

additional indicators and information are available to identify this misinterpretation. This behaviour is 

(21)  Patton, 1997. 

(22)  Rosenström, 2009. 

(23)  Id. 

(24)  Id. 

(25)  Perrin, 1998. 

Towards a harmonised methodology for statistical indicators 

14 

 
 
                                                 
Why are indicators used in policy making? 

2 

also sometimes called deliberate manipulation, which often appears in the form of ‘underreported 
negative changes in performance and disproportionally reported positive changes’ (26).  

Illegitimate  suppression  is  the  most  extreme  form  of  a  deliberate  manipulation  and  occurs  when 
relevant information is not only underreported but is deliberately not published or made available.  

From ‘use’ to ‘influence’ and ‘impact’ 

Although the terms 'use', 'influence' and 'impact' tend to be applied interchangeably, they are distinct 
concepts. The use of an indicator is only the first step in a chain of reactions (influence) that ultimately 
lead to its impact, which can be desired or undesired.  

Figure 2: The link between the use, influence and impact of indicators 

Source:  Own illustration. 

Use relates to the handling of indicators in the policy context, whereas influence relates to the effects 
indicators  potentially  have  on  policies (27)  and  can  be  defined  as  ‘the  subset  of  evaluation 

consequences that could plausibly lead toward or away from social betterment’ (28). In other words, it 

encompasses the effects of indicator use which are likely to have an impact on policy (see Figure 2). 

There  are  three  levels  of  influence:  individual,  interpersonal  and  collective.  While  individual 
influence  describes  how  an indicator’s  results  may  lead  a  person  to  shape,  change or reinforce  an 
opinion, the interpersonal influence refers to the ability of one person to convince another person on 
the basis of the message conveyed by an indicator. Further to this, the collective influence is defined 

(26)  Kalgin et al., 2015. 

(27)  Gudmundsson et al., 2009; Henry et al., 2003; POINT final project report. 
(28)  Henry et al., 2003, p. 295. 

Towards a harmonised methodology for statistical indicators 

15 

 
 
 
 
 
 
                                                 
Why are indicators used in policy making? 

2 

as the ability to shape a whole political discussion in a group of people or in a society. Quantifying the 
influence of an indicator is very challenging as the opinion of most people is shaped by many factors 
that  normally  cannot  be  disentangled.  Hence,  ‘indicator  influence  cannot  be  measured  on  a  one-
dimensional scale’ (29). 

While influence relates to the process of changing or confirming the direction of policies, impact relates 
to the potential results stemming from the use of indicators and can be considered in two categories: 
‘direct intentional’ and ‘indirect’ (30). Indicators can be intentionally developed to change policies (for 

example, collection of poverty data to force political action against poverty). However, in many cases 
indicators that were developed a long time ago are still being used for recent political problems. One 
reason for this is that it takes long to develop indicators, making timely provision of new ones for the 
political  process  challenging.  Therefore,  the  observed  impacts  of  indicators  are  ‘typically  indirect, 
unanticipated and systemic’ (31). The impact of indicators should not be confused with the impact of 

policies, which indicators can help measure. This section refers to the impact of the use of an indicator 
in the policy making process (which could be for example assessing the impact of a policy; see Chapter 
3). 

The use of an indicator does not automatically imply that it has an influence on decision making, or 
that an indicator can only be influential if it is used by the decision makers. Even when not used actively 
by any policy actor, an indicator can have an impact on policies and society through various indirect 
pathways, like for example the shaping of public opinion which in turn influences policy making. The 
influence of an indicator can have different forms e.g. it can trigger changes in the targeted policy, as 
most often happens, or even changes to administrative structures or to the operations of democratic 
institutions.  That indirect influence can also be caused by indicators used in the decision process. It 
happens when indicators ‘open up’ discussions instead of being used to provide an answer to important 
questions or to ‘close’ discussions (32).  

Indicator development for different 
rationales  

The various rationales and motivations for using indicators pose a challenge for indicator developers 
to provide indicators that at least support all the ‘legitimate’ rationales. To influence decision making, 
indicators need to be clear in their message and both the data and the metadata must be accessible 
to allow robust conclusions, both for the intended use and to prevent the unintended use.  

The same is true for the political use of indicators, which requires the indicator developers to gain and 
retain a reputation of a trusted source. Indicators also need to be developed in cooperation with the 
potential users to keep up with the changing political discussion. Provision of accessible information 
to help robust interpretation is also needed for the political use, as it helps to identify misuse and wilful 
misinterpretation of indicators in the political discussion.  

(29)  POINT synthesis report. 

(30)  Gudmundsson et al., 2009. 

(31)  Lehtonen et al., 2016. 

(32)  Eurostat, 2017b. 

Towards a harmonised methodology for statistical indicators 

16 

 
 
 
                                                 
When and how are indicators used in policy making? 

3 

3 When and how are 

indicators used in 
policy making? 

Indicators can be used at different stages of the policy making process and at different governance 
levels.  Based  on  the  timing  and  scale  of  their  use  and  the  needs  of  policy  makers,  the  quality 
requirements for indicators can vary to a significant extent. Information needs may differ at each stage 
of the policy cycle; therefore it is essential to understand this process and to accommodate the ‘fit for 
purpose’ approach for the defined indicators. 

Concept of the policy cycle  

The policy cycle is a well-known concept that helps visualise the different stages of the policy making. 
It describes a process that starts with the identification of a problem leading to the development of a 
policy  measure,  which  is  then  monitored  to  gauge  its  success.  That  assessment  might  identify 
unresolved problems, which opens a new round of the cycle.  

The literature provides different descriptions of the policy cycle consisting of a varying number of identified 
phases.  One  possible  clear  and  simple  description includes  five  stages  (33):  agenda  setting,  policy 

formulation, decision making, policy implementation and policy assessment (34) (see Figure 3).  

Figure 3: The policy cycle 

Agenda 
setting

Policy 
assessment

Policy 
formulation

Policy 
implementation

Decision 
making

Source:  Own illustration 

(33)  Howlett et al., 1995. 

(34)  ‘Policy evaluation’ in Howlett et al., 1995.  

Towards a harmonised methodology for statistical indicators 

17 

 
 
 
 
 
                                                 
 
When and how are indicators used in policy making? 

3 

The most important aspect of the policy cycle is that ‘at all stages, there is an information need where 
information  can  serve  general  audiences  and  individual  public  decision-makers’ (35).  Indicators  can 

help  fulfil  this  need  —  from  agenda  setting  to  policy  assessment (36).  The  role  of  the  statistical 

community is to provide high  quality data, in respect of the appropriate codes of practice, including 
defining  international  standards,  ensuring  adequacy  of  resources,  quality  commitment,  statistical 
confidentiality, impartiality and objectivity. 

The policymaker's information needs can differ depending on the stage of the policy  cycle and this 
needs to be reflected in the process of indicator design. These changing needs are discussed in the 
section below. 

In the phase of agenda setting policymakers need to understand and clearly define the problem they 
want to solve and the policy objectives they want to reach. Indicators can facilitate this by quantitatively 
capturing  the  problem  that  the  agenda  should  address,  and/or  by  quantitatively  illustrating  the 
objectives that should be reached. Indicators available in time series and for many different geographic 
units (such as countries, regions, sub-regions) help in agenda setting as they provide benchmarks and 
can be used for time and country comparisons.  

Once objectives have been identified and quantified, policymakers need to identify the political options 
in order to reach them. This happens in the stage of policy formulation. Indicators can facilitate this 
task by providing clear information on the policy measures that worked in the past and the context in 
which they were used.  

Decision making is the next step during which the political options available and their impacts are 
compared in order to opt for the best way forward. Indicators can add value in impact assessments by 
helping to understand the causality chains between measures and the impacts of the policy measure.  

Once the decision on specific policies has been reached, they need to be implemented by the relevant 
authorities.  Indicators  can  provide  information  on  the  administrative  costs  and  the  effectiveness  of 
different policy implementation options.  

Finally, in the stage of policy assessment indicators are used in the monitoring process, including the 
assessment of the effectiveness of existing policies. Indicators that help policymakers to understand 
trends and their drivers will enable them to see which policies work and which next policy actions might 
be necessary. The policy assessment can  lead to a new policy cycle, starting with another agenda 
setting stage.  

Some practical examples of indicators or indicator sets used in a specific stage of the policy cycle are 
presented in Table 1. 

(35)  Vestman et al., 2008. 

(36)  Jann et al., 2007. 

Towards a harmonised methodology for statistical indicators 

18 

 
 
                                                 
When and how are indicators used in policy making? 

3 

Table 1: Practical examples of the use of indicators at each stage of the policy cycle 

Stage of policy 
cycle 

Practical example 

Agenda setting 

  The  Macroeconomic  Imbalance  Procedure  (MIP)  Scoreboard  (37)  is  an 
indicator set used by the European Union (EU) to detect economic imbalances in 
the  Member  States.  The  set  consists  of  a  scoreboard  including  14  headline 
indicators selected to ‘capture both shorter term rapid deteriorations as well as the 
longer  term  gradual  accumulation  of imbalances’ (38).  Each  headline indicator is 
accompanied  by  alert  thresholds.  For  instance,  the  indicator  '3-year  average  of 
unemployment  rate'  has  an  indicative  threshold  of  10%, meaning  that  when  the 
value of the indicator surpasses 10%, an alert mechanism might be triggered. The 
scoreboard is not intended to be used mechanically and the headline indicators 
are complemented by additional indicators and other relevant information that help 
analyse, detect issues and draw informed conclusions. 

  At the national level, central banks in developed countries set inflation target rates. 
At the European level, the European Central Bank (ECB) aims at inflation rates of 
below,  but  close  to,  2%  over  the  medium  term  to  avoid  unexpected  and  high 
inflation creating perverse incentives and unwanted redistribution of wealth. As an 
indicator of inflation and price stability, the ECB uses the Harmonised Index of 
Consumer Prices (HICP) which is a consumer price index compiled according to 
a methodology harmonised across EU countries. 

(37)  See http://ec.europa.eu/eurostat/web/macroeconomic-imbalances-procedure/indicators 

(38)  European Economy, Scoreboard for the surveillance of macroeconomic imbalances, 2012, p. 4. 

Towards a harmonised methodology for statistical indicators 

19 

 
 
 
 
 
 
                                                 
When and how are indicators used in policy making? 

3 

Table 1 (cont.) 

Stage of policy 
cycle 

Policy formulation 

Decision making 

Practical example 

  Eurostat  publishes  a  set  of  indicators  measuring  Quality  of  Life (39),  which  is 
based  on  a  theoretical  framework  according  to  which  quality  of  life  can  be 
disaggregated  into  'eight  plus  one'  dimensions/domains  such  as  material  living 
conditions,  productive  or  main  activity,  health,  education,  leisure  and  social 
interactions, economic and physical safety, governance and basic rights, natural 
and living environment and overall experience of life (40). For each of these areas, 

relevant indicators for which data are available have been selected. For example, 
the  education  dimension  is  currently  measured  by  five  indicators:  population’s 
educational  attainment,  number  of  early  school  leavers,  self-assessed  and 
assessed skills and participation in life-long learning. The indicators aim at helping 
policymakers  to  focus  on  policy  objectives  which  directly  affect  the  quality  of 
people's life (41). 

  The European Environmental Agency (EEA) collects data in Member States for 
the Council Directive 2003/96/EC on the taxation of energy products and electricity 
(42). The indicator transport fuel prices and taxes considers the price of fuel in the 
EU, including cost price, excise duty and VAT and is used to advise European and 
national policymakers about energy tax levels and their structures (43).  

  Many  European  countries  and  the  EU as  a  whole  (44)  maintain  frameworks  for 
impact  assessments  that  support  decision  making  and  use  specific  indicators. 
Impact  assessments  are  carried  out  on  initiatives  expected  to  have  significant 
economic, social or environmental impacts. These can be legislative proposals; 
non-legislative  initiatives  (e.g.  financial  programmes,  recommendations  for  the 
negotiations  of  international  agreements)  or  implementing  and  delegated  acts. 
The  type  of  indicators  used  depends  on  the  impact  assessment  in  question, 
including e.g. estimates of potential savings. 

(39)  The indicator set is available here: http://ec.europa.eu/eurostat/statistics-explained/index.php/Quality_of_life_indicators 

(40)  For a detailed description of the 'eight plus one' dimensions see http://ec.europa.eu/eurostat/statistics-

explained/index.php/Quality_of_life_indicators_-_measuring_quality_of_life 

(41)  Eurostat, 2015b. 

(42)  See http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=OJ:L:2003:283:0051:0070:EN:PDF  

(43)  See https://www.eea.europa.eu/policy-documents/council-directive-2003-96-ec 

(44)  See https://ec.europa.eu/info/law-making-process/planning-and-proposing-law/impact-assessments_en 

Towards a harmonised methodology for statistical indicators 

20 

 
 
 
 
 
 
                                                 
When and how are indicators used in policy making? 

3 

Table 1 (cont.) 

Stage of policy 
cycle 

Practical example 

  The Principal European Economic Indicators (PEEIs) are ‘used as a statistical 
support  for  the  implementation  of  economic  and  monetary  policies’  (45).  They 
provide  information  on  labour,  housing,  consumer  prices  and  many  other 
the  policy 
economic  variables,  which  help  monitor  the  economy  during 
implementation process.  

Policy 
implementation 

  The EU Sustainable Development Indicator (SDIs) set was used until 2015 to 
monitor  the  implementation  of  the  EU  Sustainable  Development  Strategy.  It 
included  more  than  130  indicators,  grouped  in  10  themes,  each  with  selected 
headline indicator (46). 

 

In March 2017, the United Nations Statistical Commission (UNSC) agreed on a list 
of  244  global  indicators  to  measure  the  progress  towards  Sustainable 
Development Goals (SDGs) (47), which are at the heart of a global agenda for the 
years  2015–2030.  At  the  same  time,  Eurostat  has  developed  the  EU  SDG 
indicator set monitoring the SDGs in an EU context. 

  At the European level, for the monitoring of the Europe 2020 strategy, which aims 
to create smart, sustainable and inclusive growth, the EU has set eight quantitative 
objectives (‘Europe 2020 targets’) measured by nine indicators (48) in five thematic 

areas: employment, education, poverty and social exclusion, climate change and 
energy  and  research  and  development.  For  each  of  these  topics,  headline 
indicators were selected to help measure countries’ progress toward these targets. 
Eurostat  analyses  these  headline  indicators  together  with  other  contextual 
indicators in an annually released report on the Europe 2020 strategy (49). 

Policy assessment 

The use of indicators has a particular meaning in the agenda setting and policy assessment, being the 
first and the last stage of the policy cycle. The following sections will explore these two stages in greater 
detail.  Indicators  play  an  important  role  also  in  the  other  three  stages  (policy  formulation,  decision 
making and policy implementation), but examples of indicator use can be more clearly illustrated on 
the  example  of  the  agenda  setting  and  policy  assessment  stages,  which  explains  the  particular 
emphasis put on these aspects in the following section. 

(45)  Eurostat, 2011; for the detailed list of indicators see also http://ec.europa.eu/eurostat/web/euro-indicators/peeis.  

(46)  For the detailed list of indicators see http://ec.europa.eu/eurostat/web/sdi/indicators 

(47)  See https://sustainabledevelopment.un.org/?menu=1300 

(48)  http://ec.europa.eu/eurostat/web/europe-2020-indicators/europe-2020-strategy/headline-indicators-scoreboard 

(49)  For the recent report on the Europe 2020 strategy see http://ec.europa.eu/eurostat/web/europe-2020-indicators/europe-2020-

strategy/publications 

Towards a harmonised methodology for statistical indicators 

21 

 
 
 
 
                                                 
When and how are indicators used in policy making? 

3 

Using indicators for agenda setting 

Indicators can be useful in different ways in the agenda setting phase, helping to reflect on medium- 
and long-term goals during the planning stage (50). They can be used to determine budget allocations 

by indicating where funds are most needed. For example, specific indicators are analysed in the reports 
published by the Organisation for Economic Co-operation and Development (OECD) to illustrate the 
state of different sectors, which during the planning stage can contribute to steering policy in a given 
direction.  

Indicators can be used to clarify and quantify policy objectives. This provides a quantitative basis 
that can be clearly understood and implemented by all parties. In the European environmental policy, 
this is often done with a set of indicators that are used to determine good environmental status. The 
Ambient Air Quality Directive (2008/50/EC) (51), for example, uses nitrogen dioxide as an indicator of 

air quality and provides a threshold level that should not be exceeded in the EU Member States. 

Indicators can provide the quantitative basis for comparing several issues and help to set priorities 
determining which one(s) should be addressed first within the decision making process. For example, 
the EU Strategic Engagement for Gender Equality 2016–2019 (52) is a direct response to challenges 

and gaps within this area that were analysed with quantitative indicators on the current status of gender 
equality such as employment levels, the percentage of women as board members of listed companies, 
the proportion of women in national parliaments and governments, the proportion of female heads in 
institutions,  the  percentage  of  women  entrepreneurs  and  several  more (53).  The  analysis  of  these 

indicators provided directly informative input into the priorities and key actions for 2016–2019 that are 
described in the strategy.  

Indicators can also be used to explore ambiguities and uncertainties about the future in order to 
help  setting  priorities.  The  recent  development  process  of  new  indicators  for  societal  progress  and 
well-being serves here as a useful example. Indicators used to monitor the trends on the labour market 
enable an analysis of potentially very disruptive trends in labour demand (like robotisation) and enable 
the users to explore trade-offs, uncertainties and ambiguities (54). 

It  is  worth  noting  that  indicators  are  only  one  source  of  information  for  evidence-based  policy,  and 
should be interpreted in their context. Indicators can be used for continuous assessment but as they 
cannot track causal relationships or explain political context, they cannot replace evaluation. Research 
and evaluation is therefore needed in addition to continuous monitoring through indicators. 

(50)  Lehtonen, 2008. 

(51)  http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=OJ:L:2008:152:0001:0044:en:PDF 
(52)  See http://ec.europa.eu/anti-trafficking/eu-policy/strategic-engagement-gender-equality-2016-2019 

(53)  European Union, Strategic Engagement for Gender Equality 2016–2019, 2016. 

(54)  Eurostat, 2017b. 

Towards a harmonised methodology for statistical indicators 

22 

 
 
 
 
                                                 
When and how are indicators used in policy making? 

3 

Using indicators for policy assessment 

Indicators  can  also  support  performance  analysis  carried  out  by  international  organisations  and 
governments  at  all  levels.  A  performance  analysis  can  be  done  for  three  purposes:  monitoring, 
evaluating and benchmarking. 

According  to  the  European  Commission’s  Better  Regulation  guidelines,  ‘monitoring  generates 
evidence on an intervention’s activities and impacts over time in a continuous and systematic way’ and 
is  ‘a  necessary  and  integral part of  Better Regulation’ (55).  It  is  used for tracking  whether a  policy’s 

general  objectives  are  reached  without  analysing  the  reasons  for  which  they  were  or  were  not 
achieved. The indicators can be used as monitoring tools; in this case, the results of indicators are 
observed to determine the direction they are moving in and whether progress towards predetermined 
goals can be reported (56). An example of this type of analysis could be the monitoring of the Europe 
2020 strategy targets conducted by Eurostat.  

Monitoring should not be confused with evaluation, which the Better Regulation guidelines define as 
‘an evidence-based judgement of the extent to which an intervention has been:  

  effective and efficient; 

 

relevant given the needs and its objectives; 

  coherent both internally and with other EU policy interventions, and 

  achieved EU added-value’ (57). 

Evaluation goes beyond observing changes in indicators and aims to determine the contribution of the 
policy to these changes (as part of the change can always be attributed to other factors), which is also 
simply called the impact of the policy (58). Evaluation is conducted at the policy assessment stage, but 

it can also be used in future policy formulations and decision making stages. Currently, the European 
Commission focuses on evaluation, in which in the next years all policy programmes will be evaluated 
to create an improved evidence base for future policy development. Indicators that provide evidence 
on the effectiveness of a policy can thereby be used in evaluation and impact assessment.  

Finally,  benchmarking  is  carried  out,  for  example,  by  international  organisations  charged  with 
overseeing binding international agreements or by national organisations responsible for overseeing 
the  regional  implementation  of  national  policy  initiatives.  Benchmarking  normally  contains  some 
elements  of  monitoring  and  evaluation,  but  additionally  a  formal  benchmark  against  which  the 
development  of  an  indicator can  be  measured is  agreed.  For  instance,  monitoring  greenhouse  gas 
(GHG)  emissions  becomes  a  benchmarking  if  an  international  organisation  uses  the  countries' 
emission reduction targets as a benchmark for progress.   

(55)  European Commission, Better Regulation Guidelines, 2015b.  

(56)  European Union, Guidance document on monitoring and evaluation – Concepts and Recommendations, 2015. 

(57)  European Commission, Better Regulation Guidelines, 2015b.   

(58)  European Union, 2015. 

Towards a harmonised methodology for statistical indicators 

23 

 
 
                                                 
When and how are indicators used in policy making? 

3 

Using indicators at different levels of policy 
making 

Indicators can be used at all political levels, from local to international,  which implies differences in 
their characteristics and the requirements for the underlying data. Political decision-makers often need 
the  statistics  available  at  the  level  over  which  they  have  jurisdiction,  but  they  can  also  look  data 
available for other levels. For instance, the focus of the European Commission lies on data at the EU 
and Member State level. Local policymakers very often need data at the local level, while international 
policymakers  may  need  information  on  an  international  or national  level.  If  national policies  have  a 
local or regional focus, then national policymakers need local or regional data. However, policies rarely 
only affect one single governance level; their effects and implementation usually trickle down (or up) 
several levels. For this reason, an increasing number of policy initiatives aims to foster collaboration 
between  multiple  governance  levels.  The  United  Nations  (UN)  2030  Agenda  on  Sustainable 
Development is a good example of a multi-level collaboration. 

At  the  international  (global)  level,  indicators  become  increasingly  important  for  monitoring,  for 
example,  the  implementation  of  conventions  and  international  agreements.  The  global  list  of  244 
indicators to monitor the SDGs provides a clear example of the need and actual use of indicators to 
track the progress of different countries towards common objectives.  

Global phenomena, in particular environmental ones, are  also often monitored using indicators. For 
example, for monitoring climate change, indicators such as global average temperature or Arctic sea 
ice extent (59) are used. The United States Environmental Protection Agency uses a set of 30 Climate 

Change Indicators (60) to monitor Earth’s changing climate. 

Indicators  are  also  used  at  the  European  level.  The  euro  convergence  criteria,  also  referred  to  as 
‘Maastricht criteria’ (61), were designed to determine whether a Member State’s economy is prepared 

for  the  adoption  of  the  single  currency.  For  this  purpose  indicators  for  measuring  price  stability, 
soundness and sustainability of public finances, exchange-rate stability and long-term interest rates 
are used. 

Until 2015, Eurostat published biannually a report on the EU Sustainable Development Strategy (62), 

based on the EU Sustainable Development Indicators (SDIs). Eurostat produces also regular annual 
reports on the progress towards the targets of the  Europe 2020 strategy using the related indicator 
set (63).  The  EU  Resource  Efficiency  Scoreboard  (64)  published  by  Eurostat  presents  indicators 

covering themes and subthemes of the Roadmap to a Resource Efficient Europe (65). The scoreboard 

aims to monitor the implementation of the roadmap, to communicate the link between resources and 
economy and to engage stakeholders. Indicators are arranged in three groups - lead, dashboard and 
theme-specific indicators.  

(59)  See https://nsidc.org/arcticseaicenews/ 

(60)  The indicator set is available here: https://www3.epa.gov/climatechange/science/indicators/#learnmore 

(61)  European Commission, Who can join and when?, 2015c. 

(62)  Eurostat, 2015. 

(63)  http://ec.europa.eu/eurostat/web/europe-2020-indicators/europe-2020-strategy/headline-indicators-scoreboard 
(64)  http://ec.europa.eu/eurostat/web/europe-2020-indicators/resource-efficient-europe 
(65)  http://ec.europa.eu/environment/resource_efficiency/about/roadmap/index_en.htm 

Towards a harmonised methodology for statistical indicators 

24 

 
 
                                                 
When and how are indicators used in policy making? 

3 

At the national level, the EU Member States can use indicator sets to adapt goals and targets from 
EU strategies into national priorities and monitor their implementation. For instance, the government 
of the Czech Republic uses 27 of the 30 indicators proposed by the OECD in ‘Towards Green Growth 
—  Monitoring  Progress:  OECD  Indicators’  to  monitor  its  Strategic  Framework  for  Sustainable 
Development’ (66).  Belgium’s  Federal  Planning  Bureau  established  a  Task  Force  on  Sustainable 

Development (TFSD), which will publish a set of sustainable development indicators in line with the 
SDGs in 2017 (67). 

Indicators can also be used to monitor the national economy as for example in Germany, where budget 
figures submitted by regional authorities are used for redistribution of the tax income.  

The Swiss Federal Statistical Office (FSO) uses a system of 73 indicators called MONET (68) to monitor 

the national progress towards sustainable development objectives. The indicators are focused around 
four questions (How well do we live? How well are resources distributed? What are we leaving behind 
for our children? How efficiently are we using our natural resources?).  

There are also several examples of indicators sets used at the sub-national, regional or local level. 
In  Switzerland,  a  set  of  sustainable  development  indicators  called  ‘Cercle  Indicateurs’  is  used  to 
compare cities and cantons with each other and for their own monitoring. There is a total of 35 ‘themes’: 
11 for the environment, 12 for the economy and 12 for society. Each theme is measured with a different 
indicator,  in  some  cases  with  two  indicators (69).  For  more  than  one  third  of  the  themes,  there  is  a 

different  indicator  used  for  cities  and  cantons.  For  instance,  in  cantons  the  biodiversity  theme  is 
measured with the indicator ‘diversity of vegetal species’, while in cities the indicator ‘city nesting bird 
index’ is used instead. Further, for the European Cohesion Funds and European Rural Development 
Funds, many economic indicators are made available on a regional level. Several examples of such 
indicators can be found on the Eurostat website related to regional statistics (70).  

The  tension  between  local,  regional,  national  and  EU  levels  remains  a  major  challenge  for  many 
indicator sets used at an international level. The relevance of indicators will vary at different levels and 
also for different regions or countries on the same level. The need for international comparability is 
often an obstacle to maximise the relevance of indicators for policy actors at the national and even 
more so at the sub-national level. An individual indicator can never be fully relevant at all governance 
scales and across different contexts observed in a country. Therefore, indicator systems should have 
a certain degree of flexibility, allowing different indicators to be employed, depending on the needs of 
the country or governance level.  

(66)  See http://www.oecd.org/greengrowth/greengrowthinactionczechrepublic.htm 

(67)  Indicators.be, Indicatoren van duurzame ontwikkeling, 2017.     

(68)  See https://www.bfs.admin.ch/bfs/en/home/statistics/sustainable-development/monet.html 

(69)  See https://www.bfs.admin.ch/bfs/de/home/statistiken/nachhaltige-entwicklung/cercle-indicateurs.html  

(70)  Eurostat, Regions and Cities illustrated (RCI), 2013.  

Towards a harmonised methodology for statistical indicators 

25 

 
 
 
                                                 
Requirements for policy relevant indicators 

4 

4 Requirements for 

policy relevant 
indicators 

After clarifying when and why indicators are used, it is important to determine how to develop indicators 
suited to their specific roles. From a statistical perspective, some key criteria need to be respected to 
create or select the right indicator or indicator set. 

In the European Union, the quality standards for statistics are set in the European Statistics Code of 
Practice (71), which was adopted by the European Statistical System Committee in 2005 and revised 

in  2011.  The  Code  of  Practice  contains  15  principles (72)  that  must  be  followed  by  the  statistical 

authorities  (73).  The  principles  are  grouped  in  three  areas:  Institutional  Environment,  Statistical 

Processes and Statistical Output. It is worth noting that the fulfilment of statistical quality criteria alone 
is  necessary  but  not  sufficient  for  indicators.  As  important  as  those  criteria  are,  any  indicator  must 
clearly and correctly communicate its information to its target audience. Using the road sign analogy, 
a road sign that does not point in the right direction - for example, to the destination indicated on it - is 
obviously misleading. But even a road sign that points in the right direction can be unfit for purpose if 
it is misunderstood by users. Thus a road sign, as any indicator, must not only be accurate but also be 
correctly understood by users.  

Users expect a high level of quality from official statistics and therefore from indicators derived from 
them. They want indicators to be relevant to their individual needs, measured accurately, provided in 
time  and  comprehensive  yet  simple,  as  evidenced  by  the  progress  in  timeliness  of  the  Principal 
European  Economic  Indicators  (PEEIs) (74).  Furthermore,  the  quality  of  indicators  comes  under 

scrutiny when important decisions are being based on them. Clear quality assurance procedures for 
indicators  should  be  defined.  Performance  management  indicators  are  increasingly  being  ‘used  for 
control and management, often as part of mandatory  monitoring, reporting, assessment, evaluation 
and performance measurement frameworks’ (75).  

(71)  European Statistical System, European Statistics Code of Practice, 2011. 

(72)  These include professional independence, mandate for data collection, adequacy of resources, commitment to quality, statistical 
confidentiality, impartiality and objectivity, sound methodology, appropriate statistical procedures, non-excessive burden on 
respondents, cost effectiveness, relevance, accuracy and reliability, timeliness and punctuality, coherence and comparability, 
accessibility and clarity. 

(73)  Eurostat, national statistical institutes and other national authorities responsible for the development, production and dissemination 

of European Statistics.   

(74)  Eurostat, Principal European Economic Indicators, 2009.  

(75)  Lehtonen, 2008. 

Towards a harmonised methodology for statistical indicators 

27 

 
 
 
                                                 
 
 
Requirements for policy relevant indicators 

4 

Regular exchanges between producers of statistics and their political stakeholders is essential for a 
quick  response  to  new  policy  demands  and  increases  the  likelihood  of  relevance  of  indicators  for 
decision making and usage. Additionally, the development of relevant indicator sets for policy making 
also  requires  close  cooperation  between  statistical  offices  and  other  indicator  experts  to  establish 
common standards and improve methodologies.  

Key quality criteria for policy relevant 
indicators  

According  to  the  Quality  Assurance  Framework  of  the  European  Statistical  System (76),  five  criteria 

must be followed to ensure the quality of statistical output being in such way relevant for indicators:  

  Relevance  

Indicators must ‘meet the users' needs';  if they do not measure anything useful to policymakers, 
the  public  or  researchers,  they  will  probably  not  be  widely  used.  Indicators  should  also  be 
unambiguous in showing which direction is 'desirable' (77).  

  Accuracy and reliability  

Indicators  must  ‘accurately  and  reliably  portray  reality’;  an  inaccurate  indicator  can  lead  to 
erroneous  conclusions,  steer  the  policy  making  process  in  the  wrong  direction  or  let  negative 
effects go undetected.  

  Timeliness and punctuality 

Indicators  must  be  released  at  a  time  that  not  only  complies  with  European  and  international 
release standards, but is also relevant to the end user.  

  Coherence and comparability 

Indicators  should  be  ‘consistent  internally,  over  time  and  comparable  between  regions  and 
countries. This is particularly relevant for indicators used for policy monitoring and assessment.  

  Accessibility and clarity 

Indicators should be ‘presented in a clear and understandable form, released in a suitable and 
convenient manner, available and accessible on an impartial basis with supporting metadata and 
guidance.’  For  indicators,  this  will  be  interpreted  differently  depending  on  the  target  audience. 
What  is  clear  to  a  scientist  may  not  be  clear  for  a  policymaker,  and  what  is  accessible  to  a 
policymaker might not be accessible for the general public.  

Further, three another key criteria can be specified to summarise the user-friendliness of indicators (78). 

First, they refer to credibility which relates not only to the scientific quality of the indicator, but also to 
the  reputation  of  its  author as  indicators  are believed  or disbelieved  on  the  basis  of  who  has  been 
developing  and  calculating  them (79).  Further,  salience  refers  to  the  fitness  of  the  indicator  for  its 

(76)  See http://ec.europa.eu/eurostat/documents/64157/4392716/ESS-QAF-V1-2final.pdf/bbf5970c-1adf-46c8-afc3-58ce177a0646 

(77)  Eurostat, 2016b. 

(78)  Lehtonen, 2008. 

(79)  Lehtonen, 2016. 

Towards a harmonised methodology for statistical indicators 

28 

 
 
 
                                                 
Requirements for policy relevant indicators 

4 

intended  purpose,  i.e.  whether  it  addresses  the  intended  issue (80).    Finally,  the  legitimacy  is 

determined by the political acceptability of the indicator and relates to the political objectives that the 
indicator measures.  

Not all of these criteria can always be met simultaneously — as ‘there are obvious trade-offs between 
the three criteria’ — and the importance accorded to each can vary depending on the situation and 
user (81). Therefore, at least there may be some tensions observed between them. Large indicator sets 

will  always  limit  the  communicability  and  prevent  the  general  picture.  Some  topics  —  such  as  for 
example  sustainable  development  —  are  however  broad  and  encompass  different  areas.  A  proper 
coverage  of  all  the  dimensions  requires  a  relatively  large  indicator  set.  The  EU  Sustainable 
Development Indicator set, for instance, included about 130 indicators. Communication of such broad 
indicator  set  is  obviously  a  challenge.  A  clear,  hierarchical  structure  of  the  indicator  set,  including 
headline  indicators,  themes  and  sub-themes  play  a  key  role  in  this  respect (82).  To  deal  with  these 

challenges, a group of international experts developed the ‘Bellagio Principles’ (83), which as guidelines 

originally conceived to establish sustainable development indicators are also applicable more widely 
in the practice (see Box 1 (84) (85)).  

Box 1: The ‘Bellagio Principles’ 

The  Bellagio  Principles  guide  the  measurement  of  progress  towards  sustainable  development, 
including  the  choice  and  design  of  indicators,  their  interpretation  and  communication  of  results. 
While  there  were  originally  10  principles,  the  2009  revised  version  —  called  the  Bellagio 
Sustainability Assessment and Measurement Principles (Bellagio STAMP) — contains the following 
eight principles: 

1.  Guiding vision 
2.  Essential considerations 
3.  Adequate scope 
4.  Framework and indicators 
5.  Transparency 
6.  Effective communications 
7.  Broad participation 
8.  Continuity and capacity 

All of the criteria for indicators listed above show that the source and its credibility are important factors 
in developing indicators that are actually used. If potential users suspect the data is produced to foster 
a hidden agenda, then the data will not be trusted and the indicator’s influence will diminish.   

For indicators used to monitor a policy implementation, the European Commission’s Better Regulation 
Toolbox (86) notes that indicators should be ‘RACER’, which stands for:  

(80)  Id. 

(81)  Lehtonen, 2008. 

(82)  See also Eurostat, Towards a harmonised methodology for statistical indicators — Part 2: Communicating through indicators, 2017 

(83)  The revised principles can be consulted here: https://www.iisd.org/pdf/2009/brochure_bellagiostamp.pdf 

(84)  The experts met in response to a call of the UN World Commission on Environment and Development (Brundtland Commission) to 

assess progress toward sustainable development.  

(85)  Pintér et al., 2012. 

(86)  European Commission, Better Regulation 'Toolbox”, 2015d. 

Towards a harmonised methodology for statistical indicators 

29 

 
 
 
                                                 
Requirements for policy relevant indicators 

4 

  Relevant,  

  Accepted,  

  Credible for non-experts, unambiguous and easy to interpret, 

  Easy to monitor, 

  Robust against manipulation. 

These criteria are broadly similar to the Bellagio Principles. 

A practical example of principles and criteria used for the selection of the indicators can be derived 
from the establishment process of the Eurostat EU SDG indicator set. In line with the Better Regulation 
Toolbox of the Commission, the EU SDG indicator sets had to fulfil the RACER criteria to the largest 
possible  extent.  To  achieve  these  requirements  in  a  transparent  and  efficient  manner,  a  series  of 
specific quality criteria was applied to indicator proposals. The selection criteria were structured in a 
hierarchical order.  The following three step approach was applied to indicators considered for the EU 
SDG indicator set: 

  Requirements  on  policy  relevance:  indicators  had  to  be  either  part  of  existing  indicator  sets 
monitoring EU policies or designed to monitor a policy or initiative as reported in the Staff Working 
Document 
the  Sustainable 
Development’ (87)  accompanying  the  Communication  COM  (2016)  739  ‘Next  steps  for  a 

‘Key  European  action  supporting 

the  2030  Agenda  and 

sustainable European future (88)’.  

  Admissibility  requirements:  indicators  must  be  produced  and  disseminated  in  line  with  the 
principles stipulated in the European Statistics Code of Practice. The admissibility requirements 
could only be relaxed in areas where there is a genuine lack of indicators that meet these criteria. 
In  these  cases,  users  should  be  appropriately  informed  about  the  limitations  of  the  affected 
indicator.  

  Requirements on  data quality: indicators must achieve a minimum quality score according to a 
quality rating on frequency, timeliness, geographical coverage and comparability as well as length 
of time series and comparability over time.  

This process aimed at maximising the relevance and quality of the EU SDG indicator set, thus allowing 
a solid and reliable measuring of progress  towards the SDGs in an EU context.  

Achieving both policy relevance and 
scientific accuracy  

In addition to ensuring that the road sign is pointing in the right direction and is easily understood, it is 
also crucial for the sign to point to a destination that users want to travel to. If an indicator is accurate 
and  well  understood  but  the  information  it  provides  is  of  no  interest  for  the  current  policy  making 
process,  it  will  have  no  policy  relevance.  Figure  4  illustrates  the  factors  that  come  into  play  when 

balancing scientific accuracy and policy relevance. 

(87)  http://ec.europa.eu/europeaid/sites/devco/files/swd-key-european-actions-2030-agenda-sdgs-390-20161122_en.pdf 

(88)  https://ec.europa.eu/europeaid/sites/devco/files/communication-next-steps-sustainable-europe-20161122_en.pdf 

Towards a harmonised methodology for statistical indicators 

30 

 
 
                                                 
Requirements for policy relevant indicators 

4 

Figure 4: Strategic intent of statistics: improving the usability of evidence 

Source: Segone and Pron (2008, p. 3) 

It is essential to establish an ongoing interaction between data providers (indicator developers) and 
data  users  (mainly  policymakers) (89).  Similarly,  ‘close  cooperation  between  policy  making  and 

statistical  spheres  needs  to  be  ensured,  with  policymakers  assessing  the  relevance  of  proposed 
indicators  for  a  given  policy  and  statisticians  assessing  the  measurability  of  policy  targets’,  in  an 
‘iterative  process  with  statisticians  being  involved  from  an  early  stage’ (90).  Improving  the  dialogue 

between  the  two  groups  is  seen  as  the  key  to  producing  high  quality  statistics  that  are  also  policy 
relevant (91). 

An iterative approach can also be suggested to allow policy goals serving as a ‘normative basis for the 
development of indicators’, while also allowing indicators to help define ‘more specific policy objectives 
[...]  for  specific  policy  areas’ (92).  This  iterative  approach  is  especially  useful  when  dealing  with 

emerging issues, as they often lack clear data sources and precise targets. This dynamic process of 
review  and  update  is  essential,  because  indicators  ‘may  change  as  scientific  knowledge,  policy 
concerns and data availability progress’ (93).  

(89)  Segone et al., 2008.  

(90)  European Statistical System Committee (ESSC), Lisbon Memorandum, 2015; see p.1. 

(91)  Eurostat, Towards a harmonised methodology for statistical indicators — Part 2: Communicating through indicators, 2017. 

(92)  Hildén et al., 2008. 

(93)  OECD, 2003, p.5. 

Towards a harmonised methodology for statistical indicators 

31 

 
 
 
                                                 
Requirements for policy relevant indicators 

4 

The  iterative  process  corresponds  to  the  learning  process.  The  Macroeconomic  Imbalance 
Procedure (MIP) Scoreboard (94) provides a good example of an iterative approach as it has been 
enlarged  twice.  Similarly,  the  Principal  European  Economic  Indicators  (PEEIs)  list  has  been 
enlarged once. It is nonetheless important to stress that the set does not change completely and that 
most of the original indicators are retained to allow for uninterrupted monitoring of the related policy. 

To maximise policy relevance of indicators a broad participation of stakeholders and the public in this 
iterative process is necessary. Participation is vital in particular in the early stages during the discussion 
on the framing and definition of the indicators. It might be also essential at the more technical stages 
of establishment of indicators dealing with decisions on quantifications and the underlying framework, 
enhancing the use and influence of indicators.  

Dependent on the subject, participation can engage a broad group of citizens, or a restricted group of 
experts, citizens and stakeholders. The importance of civil society organisations is now more important 
than ever and should be emphasised in this context, as they can play a significant role in mediating 
between policymakers, experts and the citizens.   

A good example of participation of the governmental and non-governmental actors is the establishment 
process of the SDG global list of indicators agreed in March 2017. 

Key challenges of developing and 
maintaining policy relevant indicators  

When developing indicators for policy making, official statisticians or any other data scientist should 
take into account a number of challenges.  

A first one is understanding statistical indicators as a transformation of quantitative information into 
knowledge,  and  to  see  them  as  one  element  in  the  ‘information  package’  together  with  qualitative 
information  (i.e.  underlying  framework,  contextualisation)  and  expert  analysis (95).Communication  is 

more than just dissemination; it should ideally be a two-way exchange. Therefore, fostering exchange 
with policymakers and media, as well as educating them, is an essential task of official statistics.  

Another potential  challenge  for  official statistics  is  the  relation  with  policymakers.  In  particular, in 
case of important policy initiatives policymakers tend to control the development of related indicator 
sets. Often policy targets are expressed in numerical terms and can be directly measured by indicators. 
A widespread — but often not true — idea is that only what can be measured is important and that 
what cannot be measured may become irrelevant. A frequent expectation of policymakers is also that 
suitable indicators exist or can be easily produced for any policy initiative or phenomenon potentially 
interesting  from  a  policy  perspective.  These  expectations  can  result  in  challenging  discussions 
between  policymakers,  who  are publicly  accountable for their  choices  and  official statisticians,  who 
can face significant pressure during the creation of indicator sets and the dissemination of the results 
of  a  statistical  monitoring.  The  opposite  challenge  may  also  exists:  politicians  make  'post-factual' 
statements and take decisions disregarding factual evidence and sometimes deliberately against it. 
They prosper on a larger and larger share of public opinion that sees 'numbers' as tools of institutions 

(94)  See http://ec.europa.eu/eurostat/web/macroeconomic-imbalances-procedure/indicators 

(95)  European Statistical System Committee (ESSC), Lisbon Memorandum, 2015. 

Towards a harmonised methodology for statistical indicators 

32 

 
 
                                                 
Requirements for policy relevant indicators 

4 

they do not trust. In that sense, the public debate is shaped rather by appeals to emotion and personal 
beliefs than by objective facts. 

There are numerous additional challenges in the process of developing indicators for policy making, 
which  can  be  clustered  into  six  groups:  comparability,  aggregation,  decontextualisation,  goal 
displacement, misuse and maintenance.  

Geographical comparability and consistency over time 

The  ability  to  compare  data  is  vital  for  European  policy  and  essential  for  building  meaningful  EU 
aggregates and comparing country performances (between each other but more importantly against 
benchmarks, such as the Europe 2020 targets). The value of such rankings or comparisons clearly 
depends on the comparability of the data within the different jurisdictions and over time (96). Ensuring 

the comparability of statistics between geographical areas, population groups or business sectors and 
over time is difficult due to differing priorities and data availability across countries, differently defined 
economic  sectors  and  changing  conditions.  However,  for  issues  that  are  of  global  or  regional 
importance,  mechanisms  to  define  standards  are  normally  coordinated  between  international 
organisations, which over time have led to a significant increase in geographical comparability in many 
statistical domains.  

Metadata  play  a  crucial  role  in  documenting  concepts,  methods  and  data  sources  behind  the 
calculation  of  statistical  indicators,  because  they  allow  users  to  identify  potential  limitations  for  the 
comparison  across  countries.  The  analysis  of  that  comparability  is  also  an  important  aspect  of  the 
continuous improvement of indicators.  

It should also be noted that policymakers use both European and other statistics to derive indicators 
for policy making. There are two main reasons for this. Firstly, the European Statistical System (ESS) 
is not always competent for some statistical areas, such as biodiversity or climate change. Other data 
providers  such  as  the  European  Environment  Agency  (EEA)  and  the  International  Union  for 
Conservation of Nature (IUCN) can provide data for those areas. Additionally, the resources in national 
statistical institutes are increasingly limited, which makes it difficult to cope with new data needs.  

Disaggregation  

The level of disaggregation should reflect policy needs. For decision-makers, having information at the 
level of aggregation that reflects their governance level is crucial to helping them understand both the 
challenges they are facing and the potential impacts of their decisions. Hence, the disaggregation for 
many indicators along territorial, demographic and socio-economic characteristics is essential (97).  

However, disaggregation comes at a cost: the smaller the unit of disaggregation, the more extensive 
the data coverage needs to be as many statistical methods of completing datasets depend on sufficient 
data  points  for  robust  extrapolation.    That  leads  to  higher  financial  resources  needed  to  collect 
information.  For  instance,  collecting  data  on  the  number  of  underemployed  people (98)  by  gender, 

education  attainment,  single-year  age  group,  in  small  regions  at  NUTS  3  level  on  a  monthly  basis 
would be an extremely resource-intensive task. 

Modelling techniques such as small area estimation  may help in getting highly disaggregated data. 
However, model-based estimates may not be sufficiently reliable to be used for policy making.  

(96)  Hildén et al., 2008. 

(97)  See Lisbon Memorandum. 

(98)  People, who would like to work more hours than they can.  

Towards a harmonised methodology for statistical indicators 

33 

 
 
                                                 
Requirements for policy relevant indicators 

4 

Decontextualisation  

Decontextualisation is one of the most common and yet complex risks in the use of indicators. It occurs 
when indicator information is taken out of context and interpreted in an incorrect  way. In such case 
other information that may be important for understanding the problem at hand might be ignored (99). 

One  commonly  used  example  for  decontextualisation  is  the  use  of  energy  efficiency  indicators 
developed to show whether a country or an industry is progressing towards creating more economic 
prosperity with less environmental impact. In some cases energy efficiency can increase without the 
industry  or  the  country  truly  becoming  more  efficient  in  their  energy  use,  e.g.  if  a  country  stops 
producing energy-intensive products and imports them instead.  

One way to ensure that indicators are interpreted in a correct context is to present them in scoreboards, 
dashboards or indicator sets. As individual indicators are more prone to being taken out of context, 
having  a  set  of  indicators  that  complement  each  other  in  a  logical  structure  is  one  way  to  avoid 
decontextualisation.  For  instance,  the  EU  Resource  Efficiency  Scoreboard  is  a  scoreboard  which 
features  several  indicators  relating  both  to  resources  and  the  economy,  and  thus  making  the  user 
consider individual indicators  in  a  broader context.  Other  similar  examples  are the  indicator sets  to 
monitor  the  Europe  2020  strategy  or  the  global  list  of  the  UN  Sustainable  Development  Goals 
indicators.  

The metadata that accompany an indicator or indicator set can also help avoid decontextualisation, 
provided  it  clearly  explains what  the  indicator (set) measures  and  describes  common  interpretation 
mistakes and possible limitations. 

If the strengths and limitations of indicators are clearly understood, the limitations can be overcome 
with additional, supplementary indicators. Measuring Progress (100), a green economy indicator search 

engine,  helps  prevent  decontextualisation  by  providing  fact  sheets  in  which  the  research  team 
identifies blind spots or potential misinterpretations of each indicator. Complementary indicators are 
provided that could be used to fill the identified blind spots. Using the example of energy efficiency, 
Measuring Progress provides a link to indicators on trade and energy efficiency in different sectors to 
explain the drivers of energy efficiency. For instance, it suggests using indicators based on raw material 
equivalents (RME) to complement the domestic material consumption indicator. 

The method of adding explanatory indicators (101) can in some cases lead to the creation of indicator 

clusters (102) where the information of some indicators can provide context for the others.  

Misinterpretation and misuse  

Selection  and  correct  interpretation  of  an  indicator  is,  for  all  the  reasons  mentioned  above,  a 
challenging task. Some indicators are better suited for some purposes than others. For example, the 

(99)  TA-SWISS, 2010, p. 5. 

(100) See http://www.measuring-progress.org/ 

(101) It is crucial to note that explanatory indicators should be added only if needed. The Bellagio principles call for ‘a limited number of 

indicators or indicator combinations to provide a clearer signal of progress’. Point 2 of the Lisbon Memorandum states that ‘Indicator 
sets should include a limited number of indicators, a few of which are identified as key indicators, to make the set relevant, of 
analytical value and practically manageable and to ensure clear and concise communication.’ Thus indicators should only be added 
if they contribute to the key message and do not conflict with parsimony and effective communication. 

(102) See for further reference: The Swiss Federal Statistical Office (FSO), Creation of clusters to supplement legislature indicators. 

Methodology report, 2015. 

Towards a harmonised methodology for statistical indicators 

34 

 
 
                                                 
Requirements for policy relevant indicators 

4 

ecological footprint (103) may be useful for communicating with the public but not for taking concrete 

policy decisions on measures to improve resource efficiency.  

For  users  not  familiar  with  indicators  or  rarely  using  them,  there  is  always  a  risk  of  ‘simple’ 
misinterpretation.  Some  might  compare  values  without  knowing  in  which  direction  the  indicator 
should be interpreted. For instance, it might be difficult to assess if an increase in the Genuine Progress 
Index (104) for a country means a positive or negative development or what level of improvement would 

be sufficient or desirable. 

Obviously, misinterpretation is not always unintentional. Indicators used in the political process may 
be intentionally misinterpreted to influence public opinion. For instance, reporting apparently alarmingly 
significant  growth  rates  of  crimes  based  on  very  low  overall  numbers  is  just  one  way  of  using  the 
correct indicators in a misleading way. Another example would be choosing only those indicators that 
support certain pre-established policy objectives, rather than taking the entire set of indicators in order 
to provide a ‘balanced’ view.  

Providing knowledge on the interpretation of indicator results and potential pitfalls in that process is 
therefore an important strategy against possible misinterpretation, both intentional and unintentional. 

Goal displacement  

Goal  displacement  is  another  problem  associated  with  the  use  of  indicators (105).  Goodhart’s  law, 

named after the British economist Charles Goodhart, states that ‘when a measure becomes a target, 
it ceases to be a good measure’ (106). When a goal is represented by a specific measure, stakeholders 

will  often  start striving  towards  that  measure and  lose  sight  of  the  goal  behind  it.  For  example,  if  a 
country focuses on the ‘number of physicians per thousand people’ as an indicator of quality of health 
services, the target might be shifted towards increasing that number while neglecting the quality of the 
service provided by these physicians, the state of medical facilities and other essential elements of 
healthcare. 

This  challenge  has  already  been  explicitly  addressed  in  the  Macroeconomic  Imbalance  Procedure 
(MIP)  Regulation (107)  according  to  which  ‘the  indicators  should  not  be  understood  as  goals  for 

economic  policy  in  themselves  but  as  tools  to  take  account  of  the  evolving  nature  of  the 
macroeconomic imbalances within the Union. Further, it specifies that ‘conclusions shall not be drawn 
from a mechanical reading of the scoreboard indicators’ (108). 

That shows once again that providing context is the most important method to avoid goal displacement. 
Target  setting  should  always  involve  an  analysis  of  the  potential  harmful  incentives  the  target  may 
cause  and  include  a  selection  of  appropriate  indicators  to  measure  whether  negative  impacts  are 
occurring. Metadata can also help to avoid goal displacement if it clearly presents what an indicator 
can and cannot provide. 

(103) See http://ecologicalfootprint.com/ 

(104) The Genuine Progress Index (GPI) measures the economic growth of a country, like GDP, but includes both the positive and 
negative effects of economic growth in the calculation. An increase in GPI indicates an increase of economic welfare, while a 
decrease of the value is considered undesirable. Find more information on the GPI here: http://measuring-progress.eu/genuine-
progress-index-gpi  

(105) Perrin, 1998. 

(106) Goodhart's original formulation is: 'As soon as the government attempts to regulate any particular set of financial assets, these 

become unreliable as indicators of economic trends.' See Goodhart,1975.  

(107) Regulation (EU) No 1176/2011 on the prevention and correction of macroeconomic imbalances. 

(108) Chapter II, Art. 3, par. 2. 

Towards a harmonised methodology for statistical indicators 

35 

 
 
                                                 
Requirements for policy relevant indicators 

4 

Irrelevance due to lack of maintenance or changing goalposts  

As  shown  above,  the  usefulness  of  an  indicator increases  if  it  is  reported  over  longer  time  frames. 
Many indicators are created as a result of political initiatives, and statistical offices have to maintain 
them  for  years  after  they  were  launched.  To  keep  the  indicator  relevant  for  users,  data  sets  and 
sometimes  the  indicator itself,  need  to  be constantly  updated,  maintained,  evaluated,  and  regularly 
reviewed  with  involvement  of  stakeholders,  concerned  policymakers,  academics,  citizens,  social 
partners etc. Both the updates and potential reviews compete for scarce resources in the statistical 
offices, so often priorities needs to be set by determining which indicators are most likely to be relevant 
in the future and where resources should be spent. New indicators (sometimes established from new 
sources) become available over time, adding to the pressure to set priorities. Therefore, the scarcity 
of  the  resources  available  to  statistical  offices  will  also  affect  the  availability  of  relevant  indicators. 
Using existing data sources is therefore an effective way of compiling relevant new indicators.  

Over time the relevance of indicators can change if the associated targets become obsolete, either 
because they are reached ahead of time or because they become unattainable. Targets are always 
based  on  some  prediction  of  the  future  and  over  time  they  can  change  significantly,  making  the 
connected indicator more or less relevant.  

Towards a harmonised methodology for statistical indicators 

36 

 
 
Recommendations 

5 Recommendations 

Indicators are used throughout the entire policy cycle for numerous reasons. In the context of policy 
making, the development of statistics needs to foster the creation of indicators that provide information 
to  policymakers  for  important  political  decisions.  Hence,  indicators  need  to  simultaneously  provide 
correct  information  and  be  understood  and  interpreted  correctly  (as  described  in  Chapter  4).  This 
communication process which uses quantitative indicators has many challenges (see Chapter 4 and 
Part 2 of this series), but developers and providers of statistical information can help tackle these if 
they keep the communication process in mind. The section below outlines the most important aspects 
that can help meet these challenges.  

International standardisation of indicators  

Standardising indicators across countries and over time is a prerequisite for tackling problems that can 
only  be  solved  through  international  cooperation.  Ensuring  that  robust  comparisons  between 
jurisdictions and over time can be made is important for both national and international policy making. 
Even if full standardisation cannot be achieved, it is important to provide the right metadata to help 
users understand to what extent data can be compared. This includes also identifying poor data quality. 
The Lisbon Memorandum (109) formulates a commitment to ‘further developing an indicator terminology 

and methodology, capturing both the statistical aspects and the guidelines for adequate dialogue with 
users’. Particularly important is the close cooperation with other partners such as the United Nations 
(UN) and the Organisation for Economic Co-operation and Development (OECD). Indicators that have 
been standardised using the common terminology developed at the global scale can be used in the 
context  of  international  policies  or  endeavours  conducted,  for  instance,  by  the  UN.  Indicators  that 
cannot achieve this level of standardisation will always be restricted to use only at the national or even 
local levels. Standardisation is important not only for the creation and definition of indicators but also 
for the definitions and methods of the underlying statistics.  

Compliance with the quality criteria of official statistics 

Only indicators that are trusted will be relevant for policy making, therefore gaining and keeping the 
trust of users by providing high quality statistical output based on stringent criteria and impartiality is 
crucial to making the produced indicators policy relevant. While official statisticians have always been 
meticulous in observing statistical quality criteria, a clear and sound set of quality criteria such as the 
ones specified in the European Statistical System Code of Practice or the Bellagio Principles provide 
guidance to ensure that indicators are policy relevant and can be interpreted correctly. Principles for 
indicator  development  help  to  improve  the  communicative  value  of  indicators  and  applying  those 
principles should be a common standard for indicator developers.  

(109) European Statistical System Committee (ESSC), 2015 

Towards a harmonised methodology for statistical indicators 

37 

 
 
 
 
                                                 
 
 
 
Recommendations 

5 

Quality control of third-party statistics 

The limited resources of official statistics and the increasing availability of 'non-official' data (e.g. big 
data) suggest that official statistics will not be the only source of information. It is worth debating if and 
in which cases statistics can use their reputation of probity and independence to provide quality control 
for third-party data. Such a quality control role could result in development of more trusted data.  

Presenting indicators in the right context 

Each  high  quality  indicator  provides  some  insight  and  has  some  limitations  which  are  important  to 
acknowledge for a correct interpretation. Understanding those limitations and presenting indicators in 
the right context by providing accessible metadata and complementary indicators helps reducing the 
misinterpretation and misuse of indicators.  

Providing information on interpretation and on statistical quality 

All providers of statistics need to find a balance between preserving their neutrality and reputation and 
helping the users understand what an indicator means and what it does not mean. Providing not only 
information on statistical features but also on the potential pitfalls of interpretation (that are sometimes 
the effect of statistical features) will help increasing indicator literacy and limiting the misinterpretation.  

Organised cooperation between statisticians, data scientists and policymakers 

Increased  support  for  the  interaction  between  these  three  groups  is  necessary  for  all  the  other 
aforementioned recommendations to work. This could be done through research networks, research 
projects  or  permanent  committees  of  potential  users.  Regardless  of  the  form  it  would  take,  it  is 
important  to  organise  this  cooperation  and  ensure  proper  roles  for  each  group  in  the  process.  
Policymakers  using  indicators  should  also  interact  with  statisticians  and  data  scientists  in  order  to 
explain what type of indicators they need for taking informed decisions. While statisticians work with 
data and generate indicator sets, data scientists interpret them. These two groups should work closely 
together to ensure the intended meaning is correctly portrayed by the indicators. 

Towards a harmonised methodology for statistical indicators 

38 

 
 
Bibliography 

Bibliography 

Advisory Committee on Official Statistics, Good practice guidelines for the development and reporting 
of indicators, Statistics New Zealand, Wellington, July 2009 

Bartelmus, P., Indicators of sustainable development, The Encyclopedia of Earth (EOE), 2013  

Bassi, S., Mazza, L., ten Brink, P., Medarova, K., Gantioler, S., Polakova, J., Lutchman, I., Fedrigo-
Fazio, D., Hjerp, P., Baroni, L. and Portale, E., Opportunities for a better use of indicators in policy-
making: emerging needs and policy recommendations, Deliverable D7.2 of the IN-STREAM project, 
Institute for Environmental European Policy, 2011 

Bell, S., Eason, K., Frederiksen, P. (ed.), A Synthesis of the Findings of the POINT Project, European 
Commission, 2011 

Best, A., Giljum, S., et al., Potential of the Ecological Footprint for monitoring environmental impacts 
from natural resource use: Analysis of the potential of the Ecological Footprint and related assessment 
tools for use in the EU‘s Thematic Strategy on the Sustainable Use of Natural Resources, Report to 
the European Commission, DG Environment, 2008 

Boswell, C., Yearley, S., Fleming, C., Rodrigues, E., Spinardi, G., ‘The effects of targets and indicators 
on policy formulation: narrowing down, crowding out and locking in’, in Jordan, J., Turnpenny, J. R. 
(eds.), The Tools of Policy Formulation Actors, Capacities, Venues and Effects, pp. 225-244. Edward 
Elgar Publishing Limited, Cheltenham, 2015   

Braat,  L.,  ‘The  predictive  meaning  of  sustainability  indicators’  in  Kuik,  O.,  Verbruggen,  H.  (eds.), 
Research of indicators of sustainable development, pp. 59-60. Kluwer Academic Publisher, Dordrecht, 
1991 

Brown, D., Good practice guidelines for indicator development and reporting. Paper presented at the 
OECD  3rd World  Forum  on  ‘Statistics,  Knowledge  and  Policy’:  Charting  Progress,  Building  Visions, 
Improving Life, Busan, South Korea, October 2009 

Carraro  C.,  Ciampalini,  F.,  Cruciani,  C.,  Giove,  S.,  Lanzi,  E.,  Aggregation  and  Projection  of 
Sustainability  Indicators:  a  New  Approach.  Paper  prepared  for  the  OECD  3rd  World  Forum  on 
‘Statistics, Knowledge and Policy’, Busan, South Korea, 27-30 October 2009 

Davis, K. E., et al., Governance by indicators — Global power through Quantification and rankings, 
Oxford University Press, Oxford, 2012 

Desroisières,  A.,  The  Politics  of  Large  Numbers  —  A  History  of  Statistical  Reasoning,  Cambridge 
Massachusetts, Harvard University Press, 1998 

Diaz-Bone, R., Didier, E. (eds.), Conventions and Quantification — Transdisciplinary Perspectives on 
Statistics and Classifications, Historical Social Research (HSR) no. 156, 2016 

Towards a harmonised methodology for statistical indicators 

39 

 
 
 
 
 
 
 
Bibliography 

Dietz, S., Neumayer, E., Weak and strong sustainability in the SEEA: Concepts and measurement, 
Ecological Economics 61: pp. 617-626, 2007 

Dilks,  D.,  Measuring  Urban  Sustainability:  Canadian  Indicators  Workshop,  workshop  proceedings. 
Prepared for State of the Environment Directorate, Environment Canada and Centre for Future Studies 
in Housing and Living Environments, Canada Mortgage and Housing Corporation, June 19-21 1995 

Döhring,  B.,  Mordonu,  A.,  European  Economic  Papers,  No.  284,  DG  for  Economic  and  Financial 
Affairs, European Commission, 2007 

European Commission, Impact assessments, 2015 

European Commission, Better Regulation Guidelines, 2015b  

European Commission, Who can join and when?, 2015c 

European Commission, Better Regulation ‘Toolbox’, 2015d 

European Commission, European Semester 2016, 2016 

European Commission, Making it happen: the European Semester, 2016b 

European Commission, Impact assessments, 2016c  

European  Economy,  Scoreboard  for  the  surveillance  of  macroeconomic  imbalances,  Occasional 
Papers 92, February 2012  

European Environmental Agency (EEA), EN 32 Energy Taxes, 2015  

European Statistical System (ESS), European Statistics Code of Practice, 2011 

European Statistical System (ESS), Quality Assurance Framework of the European Statistical System 
– Version 1.2, 2015    

European Statistical System Committee (ESSC), Lisbon Memorandum, 2015 

European  Union,  Council  Directive  2003/96/EC  of  27  October  2003  restructuring  the  Community 
framework for the taxation of energy products and electricity,  Official Journal of the European Union, 
L 283/51, 31.10.2013 

European Union, Directive 2008/50/EC of the European Parliament and the Council of 21 May 2008 
on ambient air quality and cleaner air for Europe,  Official Journal of the European Union, L 152/1, 
11.06.2008 

European Union, Regulation (EU) No 1176/2011 on the prevention and correction of macroeconomic 
imbalances, 2011 

European Union, Regulation (EU) No 99/2013 of the European Parliament and of the Council of 15 
January  2013  on  the  European  Statistical  Programme  2013-2017,  Official  Journal  of  the  European 
Union, L 39, 9.2.2013 

European  Union,  Guidance  document  on  monitoring  and  evaluation  —  Concepts  and 
Recommendations, 2015  

European Union, Strategic Engagement for Gender Equality 2016–2019, 2016 

Eurostat, Principal European Economic Indicators, 2009 

Eurostat, Principal European economic indicators, 2011 

Towards a harmonised methodology for statistical indicators 

40 

 
 
 
Bibliography 

Eurostat, Regions and Cities illustrated (RCI), 2013 

Eurostat, Towards a harmonised methodology for statistical indicators — Part 1: Indicator typologies 
and terminologies, 2014 

Eurostat,  Sustainable  Development  in  the  European  Union:  2015  monitoring  report  of  the  EU 
Sustainable Development Strategy, 2015  

Eurostat, Quality of life indicators – measuring quality of life, 2015b  

Eurostat, Macroeconomic Imbalances Procedure, 2016 

Eurostat, Principles, Criteria and Selection Process for Developing an EU-SDG Indicator Framework, 
2016b 

Eurostat,  Towards  a  harmonised  methodology  for  statistical  indicators  —  Part  2:  Communicating 
through indicators, 2017 

Eurostat, Review of the policy relevance, formulation, and communication of the Eurostat Sustainable 
Development Indicators and the European Wheel of Competitiveness, unpublished manuscript, 2017b 

Frederiksen P., et al., Policy influence of indicators – POINT  

Fukuda-Parr, S., et al., The  Power of Numbers: A Critical Review of Millennium Development Goal 
Targets for Human Development and Human Rights, Journal of Human Development and Capabilities: 
A Multi-Disciplinary Journal for People-Centered Development 15: 1–13, 2014 

Gallopín, G. C., ‘Indicators and Their Use: Information for Decision Making. Part One-Introduction’, in 
Moldan,  B.,  Bilharz,  S.  (eds.)  Sustainability  Indicators.  A  Report  on  the  Project  on  Indicators  of 
Sustainable Development, pp. 13–27. SCOPE 58. Wiley, Chichester, 1997 

Giovannini, E., Hall, J., D'Ercole, M. M., Measuring Well-being and Societal Progress, Organisation for 
Economic  Co-operation  and  Development.  Background  paper  for  the  conference  'Beyond  GDP', 
Brussels, 19-20 November 2007 

Goodhart, C. A. E., Problems of Monetary Management: The U.K. Experience. Papers in Monetary 
Economics. Reserve Bank of Australia. I., 1975 

Gudmundssson  H.,  Morse  S.,  Bauler  T.,  Lehtonen  M.,  The  use  and  influence  of  indicators  —  a 
conceptual framework for research. Nordic Environmental Social Sciences, London, 9-12 June 2009 

Hardi P., Zdan T., Assessing Sustainable Development: Principles in Practice. International Institute 
for Sustainable Development, Manitoba, 1997 

Henry, Gary T., Mark, Melvin M., Beyond Use: Understanding Evaluation’s Influence on Attitudes and 
Actions, American Journal of Evaluation, Vol. 24, No. 3, pp. 293–314, 2003 

Hildén,  M.,  Rosenström,  U.,  The  Use  of  Indicators  for  Sustainable  Development,  Sustainable 
Development 16, pp. 237–240, 2008 

Howlett M., Ramesh M., Studying Public Policy: Policy Cycles and Policy Subsystems, Toronto, Oxford 
University Press, 1995 

IISD  (International  Institute  for  Sustainable  Development),  Sustainable  Development  Indicators: 
Proposals for the Way Forward, 2005 

Indicators.be, Indicatoren van duurzame ontwikkeling, 2017 

Towards a harmonised methodology for statistical indicators 

41 

 
 
 
Bibliography 

IN-STREAM, Linking Sustainability Indicators with Policy Making. Deliverable 8.1, 2011  

Jann, W., Wegrich, K., ‘Theories of the Policy Cycle’, in: Fischer, Frank, J. Miller, Gerald, Sidney, Mara 
S., (eds.), Handbook of Public Policy Analysis. Theory, Politics and Methods. Boca Raton, London, 
New York: CRC Press, pp. 43-62, 2007 

Jordan, A. J., Turnpenny J. R. (eds.), The Tools of Policy Formulation: Actors, Capacities, Venues and 
Effects, pp. 225–244, New Horizons in Public Policy, Cheltenham, Edward Elgar Publishing, 2015 

Kalgin,  A.,  Eliseenko,  V.,  Why  Manipulate?  Performance  Measurement  and  Data  Manipulation  in 
Regional and Local Government (working paper WP8/2015/05), 2015  

Lehtonen,  M.,  Mainstreaming  sustainable  development  in  the  OECD  through  indicators  and  peer 
reviews, Sustainable Development 16(4), pp. 241–250, 2008  

Lehtonen, M., ‘Indicators: tools for informing, monitoring, or controlling?’ in Jordan A. J., Turpenny, J. 
R., (eds.) The Tools of Policy Formulation: Actors, Capacities, Venues and Effects, pp. 76–99, New 
Horizons in Public Policy, Cheltenham, Edward Elgar Publishing, 2015 

Lehtonen, M., Sébastien, L., Bauler, T., The multiple roles of sustainability indicators in informational 
governance:  between  intended  use  and  unanticipated  influence,  Current  Opinion  in  Environmental 
Sustainability Volume 18, pp. 1–140, 2016 

Lomas, J., Culyer, T., McCutcheon, C., McAuley, L., Law, S., Conceptualizing and Combining evidence 
for health system guidance, Ottawa, Canadian Health Services Research Foundation, 2005 

Lupton, D., Risk, 2nd edition (2013), London, Routledge, 2013  

Michaelson,  J.,  Seaford,  C.,  Abdallah,  S.,  Marks,  N.,  ‘Measuring  what  matters’  in  Huppert,  F.  A., 
Cooper, C. L., (eds.), Interventions and Policies to Enhance Well-Being, Volume VI, Wiley-Blackwell, 
Chichester, 2014 

Organisation for Economic Co-operation and Development (OECD), OECD Environmental Indicators 
— Development, measurement and use, 2003 

Organisation  for  Economic  Co-operation  and  Development  (OECD),  Handbook  on  Constructing 
Composite Indicators. Methodology and User Guide, 2008 

Organisation for Economic Co-operation and Development (OECD), Green Growth in Action: Czech 
Republic, 2016  

Oxman, A. D., Lavis, J. N., Lewin, S., Fretheim, A., Support tools for evidence-informed health policy 
making (STP) I: What is evidence-informed policy making? Health Research Policy and Systems 7 
(Suppl I): SI, 2009 

Patton,  Q.  M.,  Utilization  Focused  Evaluation:  The  New  Century  Text  (3rd    Ed.),  London,  Sage 
Publications, 1997 

Perrin, B., The Use and Misuse of Performance Measurement, American Journal of Evaluation 19(3), 
pp. 367–379, 1998 

Pintér, L., Hardi, P., Martinuzzi, A., Hall, J., Bellagio STAMP: Principles for sustainability assessment 
and measurement, Ecological Indicators, Vol.17, pp. 20–28, 2012 

Porter, T. M., Trust in numbers: the pursuit of objectivity in science and public life, Princeton University 
Press, Princeton, 1995 

Towards a harmonised methodology for statistical indicators 

42 

 
 
 
Bibliography 

Radermacher, W., Indicators, green accounting and environment statistics: information requirements 
for sustainable development, International Statistical Review: a journal of the International Statistical 
Institute and its associations 67(3), pp. 339–354, 1999 

Radermacher, W., Les statistiques comptent — éviter le syndrome du lampadaire et avoir un citoyen 
européen éclairé, LA REVUE FRANCE FORUM no. 61, 2016  

Rosenström,  U.,  Sustainable  Development  Indicators,  in  Monographs  of  The  Boreal  Environment 
Research, 2009  

Segone, M., ‘Evidence-based policy making’, in UNICEF, Bridging the gap — The role of monitoring 
and evaluation in evidence-based policy making, 2008 

Segone, M., Pron, N., The role of statistics in evidence-based policy making, UNECE Work Session 
on Statistical Dissemination and Communication, Geneva, 13–15 May 2008 

Supiot, A., La Gouvernance par les nombres, Librairie Arthème Fayard, Nantes, 2015 

TA-SWISS,  All  sized  up:  Counting,  calculating  and  controlling  in  the  knowledge-based  society, 
Abridged version of the TA-SWISS study «Messen, werten, steuern. Indikatoren — Entstehung und 
Nutzung in der Politik», 2010 

The Economist, ‘The post-truth world – Yes, I’d lie to you’, The Economist, September 2016 

The Swiss Federal Statistical Office (FSO), Cercle Indicateurs 

The Swiss Federal Statistical Office (FSO), MONET 

The Swiss Federal Statistical Office (FSO), Creation of clusters to supplement legislature indicators, 
2015 

Turnhout, E., et al., Ecological indicators: Between the two fires of science and policy, in: Ecological 
Indicators 7, pp. 215–228, 2007 

United  Nations  (UN),  Indicators  to  Measure  Violence  against  Women,  Report  of  the  Expert  Group 
Meeting 8 to 10 October 2007, Geneva, Switzerland, 2007    

United Nations (UN), Inter-agency Expert Group on SDG Indicators, 2016 

United Nations (UN), Local and Regional Governments at the Heart of the Global Agenda 2030, 2016b  

United  Nations  Division  of  Sustainable  Development  (UNDSD), 
Indicators  of  Sustainable 
Development: Guidelines and Methodologies, 3rd edition, UN Division for Sustainable Development, 
Department of Economic and Social Affairs, United Nations, New York, 2007 

United Nations Environment Programme (UNEP), Green Economy — A Guidance Manual for Green 
Economy Indicators, 2014   

United  Nations  Economic  Commission  for  Europe  (UNECE),  Measuring  Sustainable  Development, 
United Nations New York and Geneva, 2008 

United States Environmental Protection Agency (EPA), Quality of Life Indicators as cited in Dilks, D., 
Lura Group, 1996 

United  States  Environmental  Protection  Agency  EPA,  A  Framework  for  Sustainability  Indicators  at 
EPA, EPA/600/R/12/687, United States Environmental Protection Agency, Washington, D.C, 2012 

Van den Hove, S., A Rational for Science-policy Interfaces, Futures, 39:7, 2007 

Towards a harmonised methodology for statistical indicators 

43 

 
 
 
Bibliography 

Vestman, O. K., Conner, R. F., The Relationship between Evaluation and Politics, in: United Nations 
Children's Fund (UNICEF), Bridging the Gap — The Role of Monitoring and Evaluation in Evidence-
based Policy Making, 2008 

Weiss,  C.  H.,  The  Interface  between  Evaluation  and  Public  Policy,  Evaluation,  5  (4),  pp.  468–486, 
1999   

White,  S.,  Zwirner,  O.,  The  Use  of  Indicators  in  the  European  Commission,  Directorate-General 
Environment  (DG  ENV),  Unit  Sustainable  Development  and  Economic  Analysis,  European 
Commission, 2007 

Towards a harmonised methodology for statistical indicators 

44 

 
 
 
Getting in touch with the EU 

In person 

All over the European Union there are hundreds of Europe Direct Information Centres. You can find 
the address of the centre nearest you at: http://europa.eu/contact 

On the phone or by e-mail 

Europe Direct is a service that answers your questions about the European Union. You can contact 
this service  

– by freephone: 00 800 6 7 8 9 10 11 (certain operators may charge for these calls),  

– at the following standard number: +32 22999696 or  

– by electronic mail via: http://europa.eu/contact 

Finding information about the EU 

Online 

Information about the European Union in all the official languages of the EU is available on the 
Europa website at: http://europa.eu   

EU Publications 

You can download or order free and priced EU publications from EU Bookshop at: 
http://bookshop.europa.eu. Multiple copies of free publications may be obtained by contacting 
Europe Direct or your local information centre (see http://europa.eu/contact) 

EU law and related documents 

For access to legal information from the EU, including all EU law since 1951 in all the official 
language versions, go to EUR-Lex at: http://eur-lex.europa.eu 

Open data from the EU 

The EU Open Data Portal (http://data.europa.eu/euodp/en/data) provides access to datasets from 
the EU. Data can be downloaded and reused for free, both for commercial and non-commercial 
purposes.