COMMISSION STAFF WORKING DOCUMENT Accompanying the document REPORT FROM THE COMMISSION TO THE EUROPEAN PARLIAMENT, THE COUNCIL, THE EUROPEAN ECONOMIC AND SOCIAL COMMITTEE AND THE COMMITTEE OF THE REGIONS Mid-term evaluation of the third Health Programme Regulation No 282/2014 on the establishment of a third Programme for the Union's action in the field of health (2014-2020)
Date: 2017-10-11
Institutions: Directorate-General for Health and Food Safety, European Commission
Subjects: EU financial instrument, EU programme, cross-border cooperation, disease prevention, health care, health care system, programme budgeting, project evaluation, public awareness campaign, public health

EN 

   EN 

   EUROPEAN COMMISSION  Brussels, 11.10.2017  SWD(2017) 331 final   COMMISSION STAFF WORKING DOCUMENT Accompanying the document REPORT FROM THE COMMISSION TO THE EUROPEAN PARLIAMENT, THE COUNCIL, THE EUROPEAN ECONOMIC AND SOCIAL COMMITTEE AND THE COMMITTEE OF THE REGIONS  Mid-term evaluation of the third Health Programme  Regulation No 282/2014 on the establishment of a third Programme for the Union's action in the field of health (2014-2020)  {COM(2017) 586 final} {SWD(2017) 333 final}  
 
 
Table of Contents 

1. 

INTRODUCTION ....................................................................................................................................... 3 

1.1  Purpose of the evaluation .............................................................................................................................. 3 

1.2  Scope of the evaluation ................................................................................................................................. 3 

2. 

BACKGROUND TO THE HEALTH PROGRAMME 2014-2020 ......................................................... 3 

2.1 Description and objectives ................................................................................................................................ 3 

2.2 Expected outputs, results and impacts  .............................................................................................................. 4 

2.2.1. Under specific objective 1 for promoting health and preventing diseases .................................................... 4 

2.2.2 Under specific objective 2 for protecting citizens from cross-order health threats, the Programme .............. 5 

2.2.3 Under specific objective 3 for contributing to innovative, efficient and sustainable health systems, ............ 5 

2.2.4 Under specific objective 4 for facilitating access to better and safer healthcare, ........................................... 6 

2.2.5 Cross-cutting actions: actions for migrants and refugees: .............................................................................. 6 

3.  METHODOLOGY ...................................................................................................................................... 7 

3.1 Preliminary steps ............................................................................................................................................... 7 

3.2 Process and methods used ................................................................................................................................. 8 

3.3 Limitations – robustness of findings ................................................................................................................. 9 

3.4 Quality assessment of the study ...................................................................................................................... 10 

4. 

THE FIRST THREE YEARS OF IMPLEMENTATION OF THE PROGRAMME ......................... 10 

4.1 Budget distribution per objective and thematic priority .................................................................................. 10 

4.2 Budget distribution per financial mechanism .................................................................................................. 12 

4.3 Budget distribution per country....................................................................................................................... 14 

5.  ANSWERS TO THE EVALUATION QUESTIONS ............................................................................. 16 

5.1 Relevance ........................................................................................................................................................ 16 

5.2 Effectiveness ................................................................................................................................................... 19 

5.3 Efficiency ........................................................................................................................................................ 24 

5.4 EU-added value ............................................................................................................................................... 29 

5.5 Coherence ....................................................................................................................................................... 30 

5.6 Utility .............................................................................................................................................................. 32 

6.  CONCLUSION .......................................................................................................................................... 33 

ANNEX I:  

PROGRAMME DESCRIPTION ........................................................................................ 36 

ANNEX II:  

METHODOLOGICAL ANNEX ......................................................................................... 43 

ANNEX III:   

THE EXCEPTIONAL UTILITY CRITERIA  .................................................................. 51 

ANNEX IV:  

INTERVENTION LOGIC ................................................................................................... 52 

ANNEX V: 

SYNOPSIS REPORT ON STAKEHOLDERS' CONSULTATION 
ACTIVITIES ........................................................................................................................... 55 

2 

 
 
INTRODUCTION 

1.1  Purpose of the evaluation 

This staff working document accompanies the Commission report and external evaluation study1 that 
the Commission is transmitting to the European Parliament, the Council, the European Economic and 
Social  Committee  and  the  Committee  of  the  Regions.  It  presents  the  conclusions  of  the  mid-term 
evaluation of the Health Programme 2014-2020 as required under Article 13(3) of Regulation (EU) No 
282/2014 establishing the third Programme for the Union's action in the field of health. 

The external and independent study supporting the mid-term evaluation of the third Health Programme 
was carried out in 2016-2017. Its purpose was mainly to assess the implementation of the Programme 
at mid-term focusing on the 23 thematic priorities and their relevance in relation to the Programme's 
objectives  and  their  contribution  to  the  Commission's  priorities.  The  evaluation  also  looked  into 
whether the 23 thematic priorities were still valid and needed to be changed. Moreover, the evaluation 
also provides input for drawing up the next programming period. 

1.2  Scope of the evaluation 

The  mid-term  evaluation  of  the  third  Health  Programme  covers  the  first  three  years  of  Health 
Programme  implementation  (2014  -  2016).  It  examines  the  relevance  of  the  choices  made  in  the 
Annual Work Plans 2014, 2015 and 2016 and the decisions on proposals submitted and awarded for 
EU funding under the subsequent calls. Moreover, the evaluation covers the efficiency of the use of 
resources,  the  EU-added  value  of  the  Programme,  the  internal  and  external  coherence,  and  potential 
for  simplification.  The  evaluation  focussed  especially  on  either  new  or  previously  under-examined 
aspects, such as the Annual Work Programmes, the multi-annual planning and the exceptional utility 
criteria. 

1.  BACKGROUND TO THE HEALTH PROGRAMME 2014-2020 

2.1 Description and objectives 

The  Health  Programme  2014-2020  is  the  third  programme  of  EU  action  in  the  field  of  health, 
established by Regulation (EU) 282/2014. With a budget of EUR 449.4 million over seven years, it is 
the Commission's main instrument to underpin and support EU health policy development. Designed 
to help Member States in investing in health, the Programme contributes to the Europe 2020 objective 
of smart, sustainable and inclusive growth.  

The Programme aims to complement, support and add value to the policies of Member States, in 
terms of improving the health of EU citizens and reducing health inequalities. It has four specific 
objectives which are the following:  

(a) promote health, prevent diseases and foster supportive environments for healthy lifestyles,  

(b) protect Union citizens from serious cross-border health threats,  

(c) contribute to innovative, efficient and sustainable health systems, and  

(d) facilitate access to better and safer healthcare for Union citizens.  

These objectives are served by actions on the following 23 thematic priorities, set out in the Annex I to 
Regulation 282/2014. See also in Annex I of this document.  

1  

Mid-term Evaluation of the third Health Programme (2014 – 2020) - Final report, written by Coffey International 
Development, SQW and Economisti Associati; ISBN 978-92-79-68447-0; doi: 10.2875/359384 and its two 
Annexes: Annex A - ISBN 978-92-79-68450-0 ; doi: 10.2875/292289 and Annex B - ISBN 978-92-79-68449-4; 
doi: 10.2875/16031 

3 

 
 
 
                                                           
 
 
 
The  thematic  priorities  aim  to  focus  the  Health  Programme  on  types  of  issues  and  action  where  the 
potential  to  generate  EU-added  value  is  greatest  and  therefore  contribute  to  the  Commission  main 
priorities for: 

 (i)  ‘growth,  jobs  and  a  resilient  society’  (health  of  the  population  and  healthcare  services  as  a 
productive factor for growth and jobs); 

 (ii) ‘digital single market’ (including eHealth); 

 (iii)  ‘internal  market’  (for  pharmaceuticals,  medical  devices,  cross-border  healthcare  directive,  and 
Health Technology Assessment); 

 (iv) ‘justice and fundamental rights’ (fighting against health inequalities); 

 (v) ‘migration’ policy, and  

 (vi) security in Europe (preparedness and management of serious cross-border health threats). 

The Programme is managed by the Commission and implemented through annual work programmes 
that are adopted following a positive opinion of a programme committee composed of Member States’ 
representatives. The work programmes are based on a multiannual perspective in order to ensure that 
all thematic priorities are covered aligned to the available resources, clustering priorities and actions. 
The implementation is entrusted with the Consumers, Health, Agriculture and Food Executive Agency 
(CHAFEA).  

In  the  Programme  participate  in  addition  to  the  28  EU  Member  States,  the  two  EFTA  countries 
Norway and Iceland, Serbia and Moldova since 2016, and recently Bosnia and Herzegovina has joined 
in  April  2017.  The  increasing  interest  of  candidate  countries  and  potential  candidates  and 
neighbouring countries could be seen as a sign of the increasing interest the Programme has for those 
countries that have decided to align their health policies with those of the EU. 

2.2 Expected outputs, results and impacts  

2.2.1. Under specific objective 1 for promoting health and preventing diseases, the Programme works for 
the  identification,  dissemination,  and  promotion  of  an  effective  uptake  of  evidence  based  best 
practices,  including  measures  to  be  taken  for  addressing  the  health  inequalities  (migrants,  ethnic 
minorities, refugees and other vulnerable groups).  

The previous Health Programmes have supported the identification of a considerable number of best 
practices  related  to  a  big  number  of  health  issues  such  as  the  promotion  of  healthy  lifestyles,  the 
prevention and management of non-communicable diseases and chronic conditions, the patient safety, 
etc.  The  third  Health  Programme  does  not  limit  itself  to  identifying  new  best  practices  but  rather 
makes the step further to organise the existing ones, validates and promotes best practices for direct 
implementation by Member States that wish to make use of them in their national policies.  

Article  3  (1)  of  Regulation  282/2014  provides    that  objective  1  must  be  measured,  in  particular, 
through  the  increase  in  the  number  of  Member  States  involved  in  health  promotion  and  disease 
prevention,  using  evidence-based  and  good  practices  through  measures  and  actions  taken  at  the 
appropriate level in Member States.  

Risk factors 

Baseline: 12 Member States have had a national initiative on the reduction of saturated fat in 2013 

Target: By 2020, all EU-28 should have had a national initiative on the reduction of saturated fat. 

4 

 
 
 
Chronic diseases including cancer 

Baseline:  0  Member  States  where  the  European  accreditation  scheme  for  breast  cancer  services  is 
implemented – establishment of the scheme. 

Target:  by  2020  all  EU-28  should  have  implemented  the  accreditation  scheme  for  breast  cancer 
services. 

2.2.2  Under  specific  objective  2  for  protecting  citizens  from  cross-order  health  threats,  the  Programme 
requires  the  identification and  development  of  coherent  approaches  and  supports  and  promotes  their 
implementation for better preparedness and coordination in health emergencies.  

The  Programme  brings  together  and  supports  competent  authorities  in  addressing  a  challenge  that 
affects all Member States which in most of the cases can have consequences to the global health and 
which  is  best  dealt  with  through  cooperation.  Supported  actions  under  this  objective  have  served  to 
identify  gaps  in  Member  States’  capacities,  prioritise  actions  and  implement  capacity  building 
activities  to  fill  in  those  gaps.  For  example  through  toolkits  and  guidelines,  simulation  and  post 
command exercises. 

Part of the role of the Health Programme is also to bring together relevant stakeholders to give them an 
opportunity to revise or agree on emergency procedures, as well as enhancing the evidence base for 
decision-making which in turn builds capacities. Through the Programme, technical expertise to tackle 
a  specific  set  of  high  risk  groups  (risk  group  3  bacteria  and  risk  groups  3  and  4  viruses)  has  been 
developed meaning that citizens would be better protected.  

Article 3(2) of Regulation 282/2014 provides that objective 2 on crisis preparedness and management 
must  be  measured,  in  particular,  through  the  increase  in  the  number  of  Member  States  integrating 
coherent approaches in the design of their preparedness plans.  

Baseline:  in  2014,  there  was  no  Member  State  with  integrated  coherent  approaches  in  the  design  of 
their preparedness plans. 

Target:  by  2020,  all  EU-28  should  have  integrated  coherent  approaches  in  the  design  of  their 
preparedness plans.  

At the time of drafting this document, 16 Member States have reached this goal. Updated indicators 
are expected to be available after the next reporting exercise by Member States on preparedness and 
response planning under Article 4 of Decision 1082/2013/EU (to be completed by November 2017).   

2.2.3 Under specific objective 3 for contributing to innovative, efficient and sustainable health systems, the 
Programme identifies and develops tools and mechanisms at EU level to address shortages of human 
and financial resources and facilitate the voluntary uptake of innovation in public health intervention 
and  prevention  strategies.  The  main  ones  are  the  Health  Technology  Assessment,  eHealth,  the 
European  Innovation  Partnership  in  Active  and  Healthy  Ageing,  the  Expert  Panel  on  health  and  the 
Commission Scientific Committees.  

Article  3  (3)  of  Regulation  282/2014  provides  that  this  objective  must  be  measured,  in  particular, 
through  the  increase  in  the  advice  produced  and  the  number  of  Member  States  using  the  tools  and 
mechanisms identified in order to contribute to effective results in their health systems. 

Health Technology Assessment (HTA) 

Baseline: 2 HTA per year 

Target in 2020: 50 HTA annually involving all EU-28 

5 

 
 
 
In 2014 and 2015, respectively 6  and 9  HTA have been developed. For further  information on 
HTA please refer to Annex I part B. 

specific  objective  4 

2.2.4  Under 
safer  healthcare,  
the expectations are for increased access to medical expertise and information for specific conditions 
also  beyond  national  borders  and  the  facilitation  of  the  application  of  the  research  results  and  the 
development of tools for improved healthcare quality and patient safety through actions contributing 
to improved health literacy. 

facilitating  access 

to  better  and 

for 

Article  3  (3)  of  Regulation  282/2014  provides  that  this  objective  must  be  measured,  in  particular, 
through  the  increase  in  the  number  of  European  reference  networks  established  under  Directive 
2011/24/EU of the European Parliament and of the Council2, the increase in the number of healthcare 
providers  and  centres  of  expertise  joining  European  reference  networks,  and  the  increase  in  the 
number of Member States using the tools developed.  

Baseline:  0 ERN 

Target in 2020 :  20 ERNs . This target has been achieved already in 2017 with the establishment of 24 
European Reference Networks (ERNs). 

Reviewed new target in 2020: 33 ERNs 

ERNs are expected to deliver a number of benefits over the medium and long-term: provide the 27-36 
million people affected by rare diseases, representing 6-8% of the overall EU population, with greater 
access  to  high  quality  health  care  and  information,  accurate  diagnosis  and  appropriate  treatment.  In 
this first phase (2016-2018), ERNs include 24 networks and almost 1000 clinics across the EU, set up 
for  complex  and  rare  diseases.  In  the  coming  years  the  concept  of  ERNs  could  also  cover  other 
complex diseases such as more common cancers or specialised paediatric interventions if resources are 
made  available.  This  will  result  in  better  chances  for  patients  to  receive  an  accurate  diagnosis  and 
advice on the best treatment for their specific condition.  

ERNs  are  not  directly  accessible  to  individual  patients.  However,  with  the  patients’  consent  and  in 
accordance with the rules of their national health system, patient case can be referred to the relevant 
ERN  member  in  their  country  by  their  healthcare  provider.  By  pooling  expertise  and  knowledge, 
ERNs are able to maximise the speed and scale with which innovations in medical science and health 
technologies  are  developed  and  put  into  use.  Economies  of  scale  can  be  achieved  by  connecting 
existing  reference  networks  found  in  individual  Member  States,  and  greater  efficiency  and 
coordination can be achieved by sharing resources and expertise across the EU. Reference networks 
are  also  ideal  environments  for  research  and  innovation  (clinical  trials,  patient  registries,  training  of 
professionals).   

2.2.5 Cross-cutting actions: actions for migrants and refugees: 

The extraordinary influx of migrants and refugees in 2015 and 2016 was an unpreceded situation and 
led to unexpected and considerable challenges for Member States health systems providing services to 
an increased population, sometimes with special health needs.  

The Health Programme was able to make a meaningful -while proportionate to the Health Programme 
budget- contribution to the EU policy on migration, particularly on integration, and to the immediate 
EU response, assisting Member States in their response. 

2   Directive 2011/24/EU of the European Parliament and of the Council of 9 March 2011 on the application of patients' 

rights in cross-border healthcare (OJ L 88, 4.4.2011, p. 45). 

6 

 
 
 
 
                                                           
Therefore the Health Programme responded to this crisis, through 11 actions financed with over EUR 
14 million. The aim was to support countries in their immediate response to health needs, develop and 
implement roadmaps and models for improving healthcare access of vulnerable migrants and refugees 
in Europe, increase awareness and commitment towards improving health and healthcare for refugees 
and  migrant,  with  focus  on  pregnant  women,  deliver  recommendations,  technical  guidance,  and 
training to health professionals, law enforcement officers and health mediators working at  local level 
with migrants/refugees. 

Training at EU level is in preparation, including health professionals, law enforcement officers, social 
workers, and health mediators and will be provided in 2017-2018. It includes knowledge, attitudes and 
awareness on migrants needs. A first piloting will take place in 10 EU countries and a second phase 
will address professionals in the 32 countries of the Health Programme. 

More specific trainings took place at local level and for the professionals locally involved in some of 
the funded projects. For example, 494 health professionals and 443 sessions for migrant groups, in one 
of the projects, 5 trainings on health mediations, 8 training of the health assessment platform and 12 
demonstrations,  with  71  health  mediators  and  2  815  migrants  reached  with  awareness  raising 
messages. Online training course packages were also produced, especially for health managers. 

Guides  for  the  assessment  of  health  needs  of  migrants/refugees  were  produced  as  well  as  country 
profiles and guidelines for the development of action plans. Country missions to Member States were 
developed. 

Health assessments have been carried out: In 2015, 4 275 health assessments have been carried out by 
the  International  Office  for  Migration ̶  which  benefited  from  a  Health  Programme  direct  grant ̶  and 
8 938  assessments  were  carried  out  in  2016.  After  positive  assessment,  migrants  considered  able  to 
travel  departed  to  21  EU  countries  and  3  EFTA  countries.  1586  personal  health  records,  aiming  to 
reconstruct  the  medical  history  of  the  migrants  and  refugees,  were  completed,  with  685  persons 
referred for additional medical diagnostics. 

In another project, 2 459 urgent individual health assessments were done with newly arrived migrants 
and 49 707 physical health assessments, 1 775 mental health ones and 4 811 social consultations, 12 
mobile teams were deployed in 9 countries.  

In a third project, 11 057 medical examinations were performed in Italy. Protocols for age assessment 
into hotspots were developed, as well as a syndromic surveillance system. 

2.  METHODOLOGY 

3.1 Preliminary steps  

An  Interservice  Steering  Group  for  the  mid-term  evaluation  established  in  October  20153,  discussed 
and  validated  the  Roadmap  for  the  mid-term  evaluation  which  was  open  for  public  consultation  for 
four weeks (14/12/2015-14/01/2016) 4. 

On  the  basis  of  this  Roadmap  and  feedback  from  Member  States  and  non-governmental 
organisations5,  the  Steering  Group  drafted  the  evaluation  questions  and  agreed  on  the  terms  of 

3   The  Commission’s  Interservice  Steering  Group  was  composed  by  the  SEC  GEN,  DG  AGRI,  DG 

CONNECT, DG DEVCO, DG EMPL, DG RTD, DG SANTE and CHAFEA. 

4   http://ec.europa.eu/smart-regulation/roadmaps/docs/2015_sante_680_evaluation_mid-

term_health_programme_en.pdf 

5   Four  contributions  were  received  (two  from  one  Member  State)  and  two  from  two  different  non-

governmental organisations.  

7 

 
 
 
                                                           
reference  for  the  specific  contract.  Feedback  was  taken  into  account,  for  example,  by  including  one 
case study on the thematic priority on alcohol policy in the EU or by examining the exceptional utility 
criterion to see if it really works. This took the form of a specific request for services with a reopening 
of competition  within the Directorate-General for Health  and Food safety (DG SANTE)  Framework 
Contract  for  Evaluations,  Impact  Assessment  and  other  studies,  Lot  1  "  Public  Health".  The 
independent study started in May 2016 and the final report was delivered a year later. The group met 
six times to also discuss the inception, interim and draft final reports and provided comments on the 
methods  and  organisation  of  the  evaluation.  It  provided  feedback  on  the  quality  assessment  of  the 
work done and has also been involved in the drafting of the present Staff Working Document which 
follows the Commission’s Better Regulation rules.  

3.2 Process and methods used 

The evaluation involved a variety of quantitative and qualitative data collection and review methods 
and analytical tools to respond to specific information needs and requirements respecting the principle 
of triangulation and the Commission standards for evaluation.  

Figure 1:  Evaluation approach 

Annex I contains a matrix showing the various tools used to make assessments on the basis of agreed 
judgment criteria and answer each of the evaluation questions. The evaluation questions are grouped 
under six categories: 1) relevance, 2) effectiveness, 3) efficiency, 4) EU-added value, 5) coherence and 
6)  utility.  Every  category  was  extensively  assessed  by  the  mid-term  evaluation  through  an  entire 
programme assessment, the case studies and the open public consultation.  

The  contractors  used  desk  research  to  analyse  existing  sources  (e.g.  Annual  Work  Programmes, 
legislative  texts,  internal  notes,  etc.),  reviewed  conclusions  and  recommendations  from  past 
Programme evaluations and made direct observations. 

They interviewed key Programme stakeholders including Commission staff and the WHO, CHAFEA 
project  officers  and  project  leaders  of  funded  actions.  They  also  discussed  with  independent6  public 

6  

In the sense that these experts were not involved in actions funded by the Health Programme.  

8 

 3. Public consultation 2. Case studies on eight thematic priorities (two from each specific objective) 1. Programme assessment covering entire HP Analysis of existing sources Consultation with key HP stakeholders Online focus groups with public health experts Overall evaluation results 1. Health promotion 2. Cross-border health threats 3. Health systems Questionnaire design Stakeholder analysis Test conclusions / recommendations 4. Access to healthcare   
 
 
                                                           
health experts in the four main relevant areas7 of the Programme and have conducted e-surveys with 
the Health Programme National Focal Points and the Members in the Programme Committee.  

To  explore  and  showcase  how  actions  in  the  thematic  areas  are  implemented  and  delivered  their 
specific results, and the main factors and processes that enabled or hindered their success, particularly 
for aspects such as EU-added value, the contractors have conducted eight case studies on 8 out of the 
23 thematic priorities. .  

The eight thematic priorities for the case studies were selected on the basis of the following criteria:  

-  actions  that  already  started  in  2016  and  were  at  the  stage  of  delivering  at  least  an  interim 
report; and 

- a sample of actions representing the spectrum of all the different financial mechanisms (e.g. 
operating grants have not been examined in previous evaluations). 

The detailed list with the selected priorities and actions can be found in Annex II, table 2.  

Then  an  open  public  consultation  carried  out  by  the  Commission  has  served  to  test  preliminary 
conclusions  and  recommendations.  The  on  line  public  consultation  which  took  the  form  of  a 
questionnaire  was  available  from  23  November  2016  to  23  February  2017.  The  public  consultation 
was available in English, however replies could be sent in any of the official languages of the EU. A 
total  of  133  contributions  were  received  mainly  from  individuals/private  persons  (32%),  non-
governmental organisations (29%), relevant to health public authorities (19%), academic organisations 
(9%), international organisations (9%) and professional associations (8%).  

The  results  of 
http://ec.europa.eu/health/programme/consultations/midterm_evaluation_en 

the  open  public  consultation  are  published  with  a  synthesis  report  at 

A  synopsis  report  on  stakeholders’  consultation  activities  can  be  found  in  Annex  V  of  the  present 
document. 

3.3 Limitations – robustness of findings 

The mid-term external evaluation study was launched in late April 2016 in order to deliver by 30 June 
2017.  Therefore  the  evaluation  came  at  a  quite  early  stage  in  the  implementation  phase  of  the 
Programme:  external  contractors  have  had  access  to  the  2014  Programme  data;  however  the  second 
year  of  Programme  implementation  (2015)  was  still  in  progress  while  for  the  third  year  (2016)  the 
calls were published but the awarding decision was not yet taken. Therefore only partial financial data 
were  available  for  2016.  This  limits  the  focus  of  the  evaluation  to  actions  from  the  Annual  Work 
Programme 2014 and in part for 2015. The actions assessed are mainly those having a short duration 
of only one year (meaning small actions, such as grants to NGOs for support to migrants and refugees 
or operating grants to NGOs and service contracts for studies). Actions with longer time schedule and 
bigger potential to contribute to the Programme objectives such as the Health Technology Assessment 
Joint  Action  (EUR  12  million)  in  2016-2020  had  only  just  started  and  therefore  it  was  too  early  to 
assess  them.  For  this  reason,  the  independent  study  was  from  the  outset  called  to  focus  mainly  on 
effectiveness of the Programme management rather than the effectiveness of the Programme actions.  

Furthermore,  improvements  addressing  problems  of  monitoring  and  dissemination  raised  in  the  ex-
post  evaluation  of  the  second  Health  Programme  that  was  published  in  May  2015  and  agreed  in  an 
Action  Plan  between  DG  SANTE  and  CHAFEA  in  December  2015,  had  not  yet  been  fully 
implemented.  Therefore  some  recommendations  in  the  mid-term  evaluation  address  the  same 
challenges that were already highlighted in the ex-post evaluation of the previous Programme.  

7   Health promotion and diseases prevention, Cross-border Health Threats, Innovative and sustainable health systems and 

Access to better and safer health. 

9 

 
 
                                                           
Other  limitations  are  related  to  the  sheer  number  and  heterogeneity  of  thematic  priorities  and 
individual  actions.  Progress  indicators  exist  but  these  exist  at  the  level  of  actions  not  at  the  level  of 
priorities  making  it  difficult  to  summarise  in  a  quantifiable  way  how  the  whole  set  of  actions  fits 
together and contribute to the Programme objectives.  

3.4 Quality assessment of the study 

The Interservice Steering Group agreed with the contractors’ findings, the answers to the evaluation 
questions  and  its  conclusions  within  the  limitations  described  above.  They  approved  the  final 
evaluation report and made the following observations:  

  The  contractors  respected  the  terms  of  reference  and  delivered  a  report  on  time  that  was  of 
satisfactory quality. The data they provided are accurate. After several in-depth reviews where 
the  contractor  addressed  all  the  issues  and  further  information  was  exchanged,  the  report 
became  very  clear,  legible  and  precise.  On  the  financial  situation  of  the  Programme 
implementation in 2016, the contractors based  themselves on what was actually available as 
the situation was evolving and made appropriate reference to this kind of limitations.  

  The  analysis  and  judgement  are  based  on  a  clearly  explained  methodology,  the  criteria  are 
transparent and the evidence used comes from different sources involving in a balanced way 
all Programme stakeholders.  

  The  conclusions  are  sound  and  limit  themselves  to  what  is  feasible  at  this  stage  of  the 
Programme implementation. The recommendations flow logically from the conclusions. They 
are practical, realistic, and addressed to the relevant Programme actors.  

3.  THE FIRST THREE YEARS OF IMPLEMENTATION OF THE PROGRAMME 

4.1 Budget distribution per objective and thematic priority 

So far, objectives 1 (on health promotion) and 3 (on health systems) have received the highest amount 
of  funding,  33  %  and  31%  respectively.  The  rest  of  the  funding  was  allocated  19  %  to  objective  4 
(access  to  healthcare),  10  %  on  horizontal  or  cross-cutting  activities  and  7%  to  objective  2  (cross 
border health threats). 

10 

 
 
 
 
 
 
 
Table 1: Allocation of budget by objective, 2014 - 2016  

Objective 

2014 

2015 

2016 

2014 – 2016  

1  Promote health, prevent disease and foster 

31% 

23% 

43% 

33% 

supportive environments for healthy lifestyles 

2  Protect citizens from serious cross-border health 

10% 

3% 

8% 

7% 

threats 

3  Contribute to innovative, efficient and sustainable 

33% 

45% 

16% 

31% 

health systems  

4 

Facilitate access to better and safer healthcare for 
Union citizens  

20% 

11% 

26% 

19% 

Horizontal 

7% 

18% 

6% 

10% 

Across the 23 thematic priorities, the funding ranges significantly.  

The work on migrants and refugees (a major part is under horizontal actions) cuts across several action 
areas  like  health  inequalities,  chronic  diseases,  mental  health,  communicable  diseases  and  health 
information. If looked at as a single entity, action on migrants and refugees represents a public health 
investment  of  EUR  14.4  million  (  EUR  9.5  million  funded  under  horizontal  activities,  another 
EUR 3.5  million  under  capacity  building  (priority  2.2)  and  EUR  1.4  under  priority  of  health 
information (priority 3.7).  

With  the  remaining  EUR  7.8  million  under  horizontal  activities,  the  Programme  supports  also 
activities for maintenance and licence of information technologies (e.g. the DG SANTE Web page, the 
Health  Policy  Platform),  communication  on  EU  health  policies  and  dissemination  of  Programme 
results, etc.  

The priorities of health legislation and health information, are attributed systematically under each of 
the four objectives covering each time relevant topics such as tobacco legislation under objective 1 for 
health promotion and prevention of diseases, legislation on health threats under objective 2, legislation 
on  pharmaceuticals  and  medical  devices  under  objective  3  for  health  systems,  legislation  on  blood, 
tissues and cells under objective 4.  

The  same  applies  also  for  the  cross-cutting  priority  of  health  information,  which  is  split  among  the 
four Programme objectives as priority 1.6, 2.4, 3.7 and 4.6; In total support to EU legislation amounts 
to EUR 27.5 million and support to health information amounts EUR 13.2 million. 

11 

 
 
 
 
 
 
 
 
 
 
 
Table 2: Allocation of budget by thematic priority, 2014 - 2016 

Source: DG SANTE and CHAFEA 

Note: figures are rounded; 2016 allocated budget is based on available data.  

*While  the  funding  to  thematic  priority  2.1  drugs-related  health  damage  appears  low, five actions  which  address    among  other secondary 
prevention  for  drugs  users    have  been  funded  under  the  3HP  and  are  accounted  under  thematic  priority  1.3  HIV/AIDS,  Tuberculosis  and 
hepatitis as the main subject was on these communicable diseases.  

**While very little funding appears to be allocated to thematic priority 4.4 on antimicrobial resistance, this topic is in fact addressed through 
some of the action funded through thematic priority 4.3 on patient safety and healthcare quality.  

***The scope of health information under objective 2 for health threats relies mainly with the European Centre for Diseases Control (ECDC) 
mission and this is the reason that no budget has been spent on this priority in the period 2014-2016.  

Every  year,  the  Commission  adopts  Annual  Work  Programme  in  close  cooperation  with  Member 
States.. In every Annual Work Programme, priorities for actions with high public health relevance and 
EU added value are set out. The Annual Work Programmes are supported by a multiannual planning. 
This mechanism was introduced for the first time in this third Health Programme and contains a more 
comprehensive, long-term approach into the programming process. 

The  following  subsections  provide  a  general  overview  of  the  funding  mechanisms  in  the  Health 
Programme and the beneficiaries that can be involved.  

4.2 Budget distribution per financial mechanism 

The  Annual  Work  Programmes  are  implemented  through  various  financial  mechanisms,  which  all 
assist  different  kinds  of  actions  and  objectives.  For  this  implementation,  DG  SANTE  of  the 
Commission is supported by CHAFEA, which manages calls for grants and tenders. The Commission 

12 

201420152016TotalTotalHorizontal€3.7 M€10.0 M€3.6 M€17.3 M1.4Chronic diseases€6.6 M€0.8 M€9.9 M€17.2 M1.1Risk factors€5.2 M€4.8 M€4.6 M€14.6 M1.3HIV / AIDS, TB & hepatitis€3.3 M€5.3 M€4.6 M€13.2 M3.1Health Technology Assessment€0.3 M€12.0 M€0.4 M€12.7 M4.1European Reference Networks€5.5 M€0.4 M€6.7 M€12.6 M3.5EIP on Active & Healthy Ageing€5.4 M€6.8 M€0.0 M€12.2 M3.6Union legislation on medicinal products and medical devices€4.0 M€3.8 M€4.2 M€12.0 M3.7Health information €5.0 M€1.3 M€2.9 M€9.2 M2.2Capacity building€1.8 M€1.4 M€4.3 M€7.5 M4.5Union legislation on blood, tisues and cells€3.3 M€1.9 M€2.1 M€7.3 M4.3Patient safety & healthcare quality€0.9 M€1.0 M€4.2 M€6.0 M4.2Rare Diseases€0.8 M€2.3 M€1.6 M€4.7 M1.5Tobacco legislation€0.2 M€1.4 M€3.1 M€4.7 M1.6Health information€1.5 M€0.4 M€1.8 M€3.8 M2.3Union legislation on cross-border health threats€3.5 M€0.0 M€0.0 M€3.5 M3.2Innovation and e-health€2.4 M€0.1 M€0.3 M€2.8 M3.4Mechanism to pool expertise€0.3 M€1.0 M€0.5 M€1.8 M3.3Health workforce€0.2 M€0.2 M€1.0 M€1.3 M1.2Drugs-related health damage*€0.0 M€0.0 M€0.6 M€0.6 M4.4Antimicrobial resistance**€0.0 M€0.4 M€0.0 M€0.5 M4.6Health information€0.0 M€0.2 M€0.0 M€0.2 M2.1Risk assessment€0.0 M€0.0 M€0.1 M€0.1 M2.4Health information***€0.0 M€0.0 M€0.0 M€0.0 M 
 
 
                                                                                      
 
 
 
 
 
 
 
is  responsible  for  the  implementation  of  highly  policy-relevant  service  contracts  and  cross-cutting 
actions.  In  the  years  2014  to  2016,  EUR  165.6  million  euros  (36.8%  of  the  total  funding)  has  been 
allocated  in  total,  of  which    EUR  53.8  million  for  2014,  EUR  55.4  million  for  2015  and  EUR  56.4 
million for 2016. 

The available mechanisms of the third Health Programme are listed below. 

Table 3: Financial mechanisms of the Health Programme8 

Financial 
mechanism 

Description 

Project grants 

They are used to fund a collaborative effort between different organisations in various EU 
MS,  which  join  forces  to  perform  various  tasks  on  a  common  set  of  objectives  for  a 
defined period of time9 

Operating 
grants 

They  provide  financial  support  towards  the  functioning  of  a  non-governmental  body  or 
network, over a period that is equivalent to its accounting year, in order to carry out a set 
of core activities10 

Direct  grants 
to 
international 
organisations 

Joints actions  

Procurement  
contracts 

Presidency 
Conferences 

They  are  awarded  to  international  organisations,  such  as  the  WHO,  with  the  capacities 
needed to tackle relevant health priorities.  

They  have  a  clear  EU  added  value  and  are  co-financed  either  by  competent  authorities 
that  are  responsible  for  health  in  the  Member  States  or  in  the  participating  third 
countries,  or  by  public  sector  bodies  and  non-governmental  bodies  mandated  by  those 
competent authorities11 

These  contracts  cover  specific  needs  related  to  the  support  of  EU  health  policies  (e.g. 
studies, development of IT tools, etc.)12 (Also called service contracts or tenders) 

Thematic conferences on health topics such as personalised medicine to mark Presidency 
of the EU. 

Others 

For example: “Payment of membership fee and reimbursement of expert mission costs", 
"Reimbursement of auditor mission costs", "Cross sub-delegation to EUROSTAT". 

The  total  budget  for  2014-2016  is  split  between  the  seven  funding  mechanisms.  Figure  2  below 
illustrates  how  and  compares  this  to  the  second  Health  Programme.  The  largest  part  of  funding  is 
allocated  to  joint  actions  (30%),  procurement  contracts  (27%)  and  projects  (24%).  Already  in  the 
second Health Programme, a progressive shift in funding from projects to joint action was provided, 
which also applies to this Programme. Smaller proportions of the budget were reserved for operating 
grants  (9%)  and  direct  grant  agreements  (7%).  "Other"  mechanisms  and  conferences  took  only  2  % 
and <1 %, respectively, of the total funding in the first three years.  

8  

9  

http://ec.europa.eu/chafea/health/index.html 

http://ec.europa.eu/chafea/documents/health/hp-factsheets/project-grants/factsheets-hp-pg_en.pdf  

10   http://ec.europa.eu/chafea/documents/health/hp-factsheets/operating-grants/factsheets-hp-og_en.pdf  

11   http://ec.europa.eu/chafea/documents/health/hp-factsheets/joint-actions/factsheets-hp-ja_en.pdf  

12   http://ec.europa.eu/chafea/health/tenders.html  

13 

 
 
 
 
 
 
 
                                                           
Figure 2: Proportion of total funding13 by mechanism for 2HP and 3HP (2014 – 2016)  

                                                                                                                    Sources: DG SANTE and CHAFEA 

The  procurement  contracts  are  mainly  managed  by  CHAFEA  while  a  smaller  number  are  sub-
delegations,  administrative  agreements  and  reimbursement  of  experts  being  implemented  by 
DG SANTE.  

Table 4: Budget allocation between DG SANTE and CHAFEA, 2014 – 2016 

2014 

2015 

201614 

TOTAL 

CHAFEA  

€44.4 M 

€48.6 M 

€47.2 M 

€140.2 M 

DG SANTE  

€9.4 M 

€6.8 M 

€9.2 M 

€25.4 M 

TOTAL  

€53.8 M 

€55.4 M 

€56.4 M  €165.6 M 

                                                                                  Source: CHAFEA and DG SANTE 

4.3 Budget distribution per country 

In  the  third  Health  Programme  participate  the  28  EU  Member  States,  two  EFTA  countries  (Norway 
and Iceland), and since 2016 Serbia and Moldova; in 2017 Bosnia and Herzegovina has also signed a 
bilateral agreement for participation in the Programme.  

65% of  the total budget for Joint Actions is distributed  between seven  of the EU 15 Member States 
(see graph below) and 52 % of the total budget for projects between four Member States (ES, IT, UK 
and NL). All organisations receiving below of the average funding are from low GNI countries.  

13   Percentages are rounded to the nearest whole percentage point 
14   Figures are rounded 

14 

 
 
 
 
 
 
 
 
   
 
 
 
                                                           
Figure 3: Allocation of funding for joint actions across participating countries 

Figure  4:  Average  funding  received  for  organisations  participating  in  joint  actions  by  participating 
country 

Figure 5: Allocation of budget to organisations across participating countries, 2014 and 2015 

15 

 
 
 
 
 
 
4.  ANSWERS TO THE EVALUATION QUESTIONS 

The  main  conclusions  to  each  of  the  16  evaluation  questions  (EQ)  presented  in  the  mid-term 
evaluation are summarised below.  

5.1 Relevance 

EQ1: To what extent are the 3rd Health Programme objectives still valid and in accordance with 
health needs in Europe? 

The Programme objectives are valid and respond to emerging EU needs and evolving challenges. The 
replies  of  National  Focal  Points  and  Programme  Committee  members  on  what  is  the  single  most 
important  reason  for  EU  action  (as  opposed  to  action  at  the  national,  regional  or  local  level)  can  be 
summarised as follows:  

Table 5: Why do Member States need an EU action following the Programme objectives? 

Objective 

Most important reason(s) given for EU action by mini-survey respondents 

1.  Promote health, 

prevent diseases & 
foster supportive 
environments  

Exchange of best practice and expertise, potential for mutual learning between 
Member States. At the EU level it is possible to promote activities and 
engagements that would not otherwise be a priority for national 
government. Also pooling resources was mentioned. 

2.  Protect citizens from 
serious cross-border 
health threats 

The fact that communicable diseases are trans-boundary in nature and 
require a trans-boundary response (the EU can support development of early 
warning systems which prevent the rapid spread of communicable diseases). 

3. 

Innovative, efficient & 
sustainable health 
systems 

4.  Facilitate access to 
better and safer 
healthcare  

Exchange of best practices and innovation. EU action supports MS especially 
vis-à-vis eHealth and the EIP on active and healthy ageing. 

Responses here were wide ranging, somewhat reflecting the objective itself. 
Reponses ranged from citing specific areas like antimicrobial resistance 
(AMR) and European Reference Networks where economies of scale can be 
achieved through EU level action to the establishment of common standards to 
prevent disease and ensure equal treatment of all EU citizens thus creating greater 
cohesion among Member States. 

The Programme is addressing to a large extent the relevant health needs. In comparison to the second 
Health  programme  it  has  increased  the  focus  on  issues  generating  the  most  EU  added  value,  while 
remaining flexible to respond quickly to emerging health needs. In some cases and despite the fact that 
the  Programme  covers  mental  health  under  the  priority  of  chronic  diseases,  public  health  experts 
pointed out the need to have this as a distinguished thematic priority for attracting more attention on 
the importance of mental health issues.  

Moreover,  the  evaluation  found  that  the  division  in  the  rationale  for  action  under  the  objectives  is 
visible.  The  potential  EU-added  value  is  clearest  for  objective  2  (cross-border  health  threats)  and  4 
(access to healthcare), which seems to attract less attention in the Member States but the EU role is 
clearer  in  the  sense  that  Member  States  would  struggle  to  address  them  on  their  own.  Objective  1 
(health promotion) and 3 (health systems) draw larger resources and there the Health Programme can 
sharing. 
enable 

best-practices 

knowledge 

exchange 

and 

the 

of 

16 

 
 
 
 
 
 
 
 
 
EQ2: To what extent are the 3rd Health Programme thematic priorities sufficient and 
sufficiently covered to achieve the 3HP objectives and Commission wider priorities? 

The thematic priorities are successful in  defining the purpose of the Programme more precisely than 
its predecessor programmes thus increasing its coherence and focus. They are relevant and reflect well 
health needs of Member States. They also serve in the need for actions to be more aligned with  high 
priorities  of  Member  States  and  with  wider  policy  objectives  of  the  European  Commission.  Not 
enough  Member  States  health  policy  documents  were  available  for  the  evaluators  to  rank  their 
priorities, but replies from the National Focal Points and the Programme Committee members on the 
three thematic priorities that are most important for public health in their country in relation to each 
objective of the Programme15 together with replies to the Open Public Consultation provided further 
insights  and  showed  consistency  in  priorities  among  stakeholders  and  also  with  the  real  Programme 
spending.  

Figure  6:  Importance  of  thematic  priorities  for  public  health  in  Member  States  according  to  the 
Survey with Programme Committee members and National Focal Points 

15   Graphic  1 presents  an overview  of  the responses  received  (with  the  percentages of respondents who have 

chosen each option). The sum up of %ages can be over 100% 

17 

 
 
 
 
 
 
 
 
 
                                                           
Figure  7:  Respective  importance  of  the  23  thematic  priorities  according  to  the  open  public 
consultation, (number of contributions = 133)  

Objective 1 (health promotion): Risk factors (1.1) and chronic diseases (1.4) are overwhelmingly cited 
as most important in both surveys.  

Objective 2 (cross-border health threats) : the most frequently selected as important thematic priorities 
are  capacity  building  against  health  threats  (2.2),  Health  information  and  knowledge  systems  to 
evidence-based  decision  making  (2.4)  and  the  Implementation  of  EU  legislation  for  cross-border 
health threats(2.3) 

Objective 3 (health systems): respondents seem to agree that innovation and eHealth (3.2) is the most 
important in this context.  

Objective 4 (access to health care): measures to prevent antimicrobial resistance and control healthcare 
associated  infections  (4.4)  together  with  patient  safety  and  quality  of  healthcare  (4.3)  attract  the 
interest of Member States’ representatives and Programme stakeholders.  

Stakeholders  did  not  miss  any  priorities  in  the  list  though  some  wanted  more  visibility  for  mental 
health (currently this is an item under chronic diseases). In the open public consultation other specific 
areas  such  as  endocrine  disruptors  in  air,  soil  and  water  were  mentioned  that  are  covered  under  the 
environmental programme16. On the issue of endocrine disruptors the Scientific Committees financed 
under the Programme have also issued two opinions17. 

16   http://ec.europa.eu/environment/chemicals/endocrine/index_en.htm 
17   http://ec.europa.eu/health/sites/health/files/scientific_committees/emerging/docs/scenihr_o_047.pdf 
http://ec.europa.eu/health/sites/health/files/scientific_committees/emerging/docs/scenihr_o_040.pdf 

18 

 
 
 
 
                                                           
EQ3: To what extent are the actions prioritised in the Annual Work Plans (AWPs) relevant vis-
à-vis the 3rd Health Programme's thematic priorities? 

The  Health  Programme  focused  and  prioritised  actions  mainly  on  chronic  diseases  in  Annual  Work 
Programme  2014,  Health  Technology  Assessment  in  2015,  and  the  establishment  of  European 
Reference Networks in 2016, while actions for increasing the capacity of Member States to respond to 
cross-border health threats were continuously supported by the Programme throughout the first three 
years  with  a  particular  focus  on  the  refugees'  crisis  in  years  2015  and  2016.  All  these  actions  were 
supported  by  substantial  budgets  and  correspond  to  the  four  Programme  objectives  (1)  for  health 
promotion and disease prevention, (2) for preparedness and crisis management, (3) for innovative and 
efficient health systems and (4) for better and safer health. 
The  evaluation  examined  in  the  case  studies  and  assessed  positively  the  relevance  of  these  actions18 
and reconstructed the intervention logic of the thematic priorities showing how the actions contribute 
in the Programme objectives (see in Annex IV).  

The current structure of the Programme with explicit objectives and relevant thematic priorities serves 
to ensure the appropriateness of individual actions. The funded actions have shown to correspond to 
health needs and are built on clear and relevant objectives, especially when a legal basis exists in the 
fields of their predilection. In this case the action focuses immediately on achieving concrete outputs 
(e.g.  the  establishment  of  European  Reference  Networks).  Otherwise,  for  broader  priorities  in  fields 
without legal commitment (e.g. chronic diseases, expert panel on Health) it is necessary to anticipate 
collective thinking which is conducive to better planning and agreed objectives already in the design 
phase of the actions.  

The choice of the financial mechanism is also an important aspect and in the case of operating grants it 
became obvious that this financial mechanism enshrined in the Programme Regulation, supports more 
a  mission  rather  than  specific  objectives.  This  means  that  the  relevance  of  the  mission  should  be 
assessed  and  monitoring  and  evaluation  processes  should  be  available  for  keeping  track  of  how  the 
organisations benefiting from operating grants perform in relation to the health policy priorities.  

EQ4: To what extent are the actions co-funded through the AWPs relevant to achieving the 

objectives set out in Article 168 TFEU? 

This evaluation question was addressed together with evaluation question 15 on the coherence and 
consistency of the Health Programme.  

5.2 Effectiveness 

EQ5: To what extent is the process for defining and prioritising actions through Annual Work 

Programmes (AWPs) transparent, equitable and impartial? 

Each  year  an  Annual  Work  Programme  is  prepared  by  DG  SANTE  in  close  coordination  with  the 
Member  States  and  adopted  through  Comitology  (examination  procedure).  The  Annual  Work 
Programme  identifies  the  actions,  describes  the  desirable  outcomes  and  results,  decides  on  the 
financial mechanisms to be used for each of the actions and the criteria for the selection and award of 
funding.  This  is  the  master  piece  in  the  successful  Programme  implementation  and  the  contractors 
approached the question by examining to what extent the Annual Work Programme is open to scrutiny 
by  the  relevant  stakeholders  (transparency),  whether its  preparation  includes  the  fair  participation  of 
and consideration of actors concerned (equity) and is free from political bias (impartiality).  

18   With the exception of Health Technology Assessment. 

19 

 
 
 
 
 
 
                                                           
The  evaluation  found  that  the  formal  process  for  setting  Annual  Work  Programmes  is  largely 
transparent, equitable and impartial.  

The  formal  consultation  process  includes  two  rounds  of  discussions  and  a  final  agreement  with  the 
Programme  Committee  which  is  comprised  of  Member  States  representatives.  Draft  versions  of  the 
Annual  Work  Programme  are  presented  in  time,  discussed  and  voted  on,  while  a  summary  of  the 
procedure,  views  expressed  and  decisions  taken  is  recorded  with  minutes  that  are  also  sent  to  the 
European Parliament. Given this is made publicly available, it can therefore be considered to be both 
well-documented and open to scrutiny.  

The  formal  consultation  of  Member  States,  CHAFEA  and  other  European  Commission  departments 
works well and annually defined priorities synthesize actions in line with current public health needs.  
The Programme legal basis provides for an informal consultation19 with the relevant experts, including 
National  Focal  Points.  This  is  done  from  2016  onwards  only  with  the  Programme  Committee 
Members as an informal consultation before the formal Comitology opinion.  

On  this  informal  procedure  there  is  minor  criticism  on  its  transparency  and  fairness  implying  that 
some Member States are more influential than others.  

The aspects of fairness and transparency in the consultation process are important for the involvement 
of all countries in the planning process. Some countries feel also that a growing proportion of funding 
is  dedicated  to  top-down  financial  mechanisms,  such  as  joint  actions  and  procurement  contracts 
leaving fewer possibilities for Member States to promote their ideas for other actions.  

Respondents to the open public consultation asked also for greater transparency implying that there is 
limited awareness of the consultation processes outside the Programme's core stakeholder groups. 

EQ6: How effective was the Multi-Annual Planning (MAP) for the preparation of the Annual 

Work Programmes?  

The evaluation shows that the Multi-Annual Planning has improved the preparation of Annual Work 
Programmes. 

Following  recommendations  from  evaluations  of  previous  Programmes,  for  the  third  Health 
Programme the Commission introduced the Multi-Annual Planning as an internal planning exercise in 
order to incorporate a more holistic, longer-term mind-set into the programming process.  

The first Multi-Annual Planning covered the years 2014-2016 (which is the chronological scope of the 
mid-term evaluation) and explained in an internal note its caveat in planning ahead in the more distant 
future. A second Multi-Annual Planning exercise was organised in the second half of 2016, covering 
the remaining years 2017-2020.  

The Multi-Annual Planning has contributed to a smoother and faster process for agreeing the Annual 
Work Programmes. Moreover, it proved to be quite accurate. Taking the number of amendments to the 
work programme as an indicator we can see a clear reduction of amendments from the second to the 
third Health Programme; in the second Health Programme, three out of six annual work programmes 
needed amendment  indicating that the planning  was not so effective or lacked  flexibility. Until now 
the third Health Programme needed to be amended in 2015 under specific circumstances for helping 
Member States to cope with the refugees’ crisis.  

The coherence between the Multi-Annual Planning and the Programme implementation is a measure 
of its effectiveness as it illustrates that the Multi-Annual Planning not only facilitated the drafting of 
the Annual Work Programmes but that it was realistic and accurate; its objective was not to set plans 
in stone but rather a plan which will be flexible to new/unexpected developments (refugees crisis) or 
to changing timetables (delay in the adoption of legislation for medical devices).  

19   See Regulation (EU) N° 282/2014 whereas n° 33): "In the application of the Regulation, the Commission should consult 

the relevant experts, including National Focal Points".  

20 

 
 
 
 
                                                           
Moreover,  the  main  focus  for  years  2014,  2015  and  2016,  respectively  on  prevention  of  chronic 
diseases,  health  technology  and  innovation  and  establishing  European  reference  Networks,  has  been 
confirmed and acknowledged by the mid-term evaluation.  

The  independent  study  found  that  the  introduction  of  the  Multi-annual  Planning  has  improved  the 
ability  to  purposely  focus  spending  on  key  areas  as  reflected  in  the  data  provided  in  detail  under 
efficiency (EQ9). 

EQ7: How effective is the introduction of ‘exceptional utility’ criteria in the Regulation 

establishing the 3HP (paragraph 19) in order to incentivise the participation of low GNI 
Member States? 

The exceptional utility criterion has not been so far effective in incentivising the involvement of low 
GNI  Member  States  and  helping  them  to  be  more  actively  involved  and  absorb  bigger  parts  of  the 
Programme budget.  

The exceptional utility criterion addresses a significant problem for low GNI countries to participate in 
the  Programme.  While  exceptional  utility  was  also  used  in  the  previous  Health  Programme,  the 
‘exceptionally  utility’  criterion  was  enshrined  for  first  time  in  the  Programme  Regulation  (EU) 
N° 282/2014 covering only Joint Actions. The rationale behind was to incentivise low GNI countries 
and  keep  them  participating  in  the  Health  Programme  especially  under  the  difficult  economic 
circumstances  of  the  financial  crisis  started  after  2009-2010.  From  the  beginning  of  the  Programme 
implementation in 2014, the criterion has been slightly adapted and has been extended to cover also 
grants for projects and operating grants.  

The new criteria (for their definition see Annex III) were providing for a greater number of countries 
to be considered for applying higher co-funding rate (up to 80 % for EC contribution).  

Table 6: MS triggering higher co-funding in 2HP and 3HP 

BG   CY  CZ  EE 

ES 

EL  HR  HU 

LT 

LV  MT  

PL 

PT  RO  SI  SK  TOTAL 

2HP 

3HP 

 

 

 

 

  20   

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

7 

16 

However,  there  is  no  evidence  that  participation  of  low  GNI  countries  is  incentivised  by  the 
simplified,  extended  and  expanded  exceptional  utility  criteria.  On  the  contrary  contractors  found 
examples where both joint actions and projects were fulfilling the criteria but they have not applied for 
‘exceptional utility’ funding of up to 80 %.  

In the first three years of Programme implementation the part of funds awarded to high and low GNI 
countries seem to show a rather retro-gradation (see the below table) implying that either the criteria 
are not clearly understood by the applicants, or that other unknown factors may intervene.  

20   Taken as low GNI country for 2015 only 

21 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
                                                           
Table 7: Balance of funds to joint action beneficiaries from low and high GNI countries 

Low GNI  

High GNI  

2HP  

3HP21  

29% 

25% 

71% 

75% 

                                                Source: CHAFEA 

For projects the situation is also quite similar with no difference between the second and third Heath 
Programmes.  

Table 8: Balance of funds to projects beneficiaries in low GNI and high GNI countries 

Low GNI  

High GNI  

2HP  

3HP22 

28% 

26% 

72% 

74% 

                                              Source: CHAFEA 

The instrument has a low up-take so far and functions less for projects and joint actions in comparison 
to the operating grants (14 out of 41 of them received exceptional utility). Possible reasons for this low 
take-up  are  limited  awareness  or  understanding  of  the  criteria  or  that  the  criteria  are  insufficiently 
addressing the needs of the GNI countries and so less attractive to overcome other challenges. Indeed, 
securing  co-funding  is  very  important  for  low  GNI  countries  (11  respondent  to  the  e-survey  with 
National Focal Point and Programme Committee members corresponding to 44% of the total selected 
and from those eleven, seven are from low GNI countries) but is not the only barrier of participation.  

Administrative burden is also mentioned as quite prohibitive by respondents from high GNI countries 
while  irrespectively  of  the  relative  income  of  a  given  Member  State,  lacking  skills  and  institutional 
resourcing challenges were also cited.  

EQ8: In practice, to what extent are the actions in the AWP contributing to the 3HP objectives 

and thematic priorities and to the Commission policy priorities? 

The  Health  Programme  is  being  implemented  focusing  on  major  Commission  priorities,  such  as: 
(a) jobs, growth and investment (health of population and health care services as a productive factor 
for growth and jobs), (b) internal market (for tobacco, pharmaceuticals, medical devices, cross-border 
health  care  directive,  and  Health  Technology  Assessment),  (c)  digital  single  market  (including 
eHealth), (d) justice and fundamental rights (fighting against health inequalities), (e) migration policy, 
and (f) security (preparedness and management of serious cross-border health threats). 

From  the  evidence  provided  by  the  case  studies  (29  actions  corresponding  to  8  thematic  priorities), 
contractors showcase key examples of meaningful achievement:  

Diagnostic  microbiology  provides  crucial  results  for  the  clinical  management  of  infectious  disease 
patients,  but  also  for  the  identification  of  a  given  pathogen  for  public  health  surveillance,  outbreak 
alert  and  response.  The  Commission  supports  Member  States'  cooperation  on  laboratories  under  the 

21   The figure corresponds to 2014 and 2015 committed spending 

22   Years 2014 and 2015 committed amounts only. 

22 

 
 
 
 
 
 
 
                                                           
 
 
EU  Health  programme  to  strengthen  preparedness  and  response  to  serious  cross-border  threats  to 
health. The funded laboratory networks addressed preparedness activities aiming at continuous quality 
improvement  in  the  rapid  diagnosis,  surveillance,  epidemic  intelligence  and  public  health  risk 
assessment  for  infectious  diseases  as  well  as  risk  management  activities  through  rapid  identification 
and characterization of infectious agents which may threaten public health. 

The establishment of 24 European Reference Networks is also a key achievement for the development 
of  better  and  safer  healthcare,  after  years  of  previous  work  on  rare  diseases,  and  facilitating  the 
implementation of Directive 2011/24/EU on patients’ rights in cross-border healthcare (see point 2.2.4 
above).  

Moreover, the Programme's major contribution to innovative, efficient and sustainable health systems 
are the approaches to the implementation of Health Technology Assessment (a subject on which the 
Programme works since 2002-2007). Now under the third Health Programme, the biggest Joint Action 
to date with EUR 12 million aims to anchor these in Member States practices. As already described 
under point 2.2.3, the cooperation is currently based on a voluntary mechanism supported by Article 
15  of  Directive  2011/24/EU  which  provides  that  the  EU  cooperation  on  Health  Technology 
Assessment  can  receive  EU  aid.  Although  successful,  there  are  also  identified  shortcomings  in  the 
current  cooperation  arrangement.  The  Commission  is  currently  finalising  an  Impact  Assessment 
assessing  different  options  how  to  strengthen  cooperation  thus  building  on  the  achievements  of  the 
Joint  Actions,  address  their  shortcomings,  and  ensure  sustainability  of  the  cooperation  beyond  2020 
when the current Joint Action ends. 

In  parallel,  the  Health  Programme  supports  wider  Commission  priorities  such  as  the  Digital  Single 
Market  by  assisting  Member  states  in  including  eHealth  into  health  policies  while  better  aligning 
eHealth investments to health needs23. Mechanisms such as the Expert Panel on Health and the Expert 
Group  on  Health  Systems  Performance  Assessment  produced  opinions  and  advice  for  the  benefit  of 
Member States in the context of the health systems’ reforms.  

The support to legislation on medicinal products and medical devices guarantees the free movement of 
these  products  in  the  internal  market,  the  safety  of  Europeans  and  is  also  related  to  the  good 
performance  of  health  systems.  The  actions  funded  concern  the  development  and  maintenance  of 
databases  e.g.  the  European  Database  for  Medical  devices  (EUDAMED),  which  gives  relevant 
agencies in Member States access to information on medical devices available on the market24 and the 
list  of  economic  operators;  management  of  marketing  authorisations  for  medicinal  products  and  of 
maximum residue limits of veterinary medicinal products practices (e.g. European Medicinal Products 
database);,  the  issuing  of  guidelines  for  harmonised  practices,  inspector  training  programmes 
implemented  for  medical  devices,  studies  for  informing  the  relevant  regulatory  frameworks  for 
medicinal products and medical devices. 

The Health Programme has supported the identification, dissemination and takes up of best practices 
in risk factors and chronic diseases (for more information please refer to above point 2.2.1). 

The Programme supports also the collection of indicators and data for the State of Health in Europe 
and the European Core Health Indicators.  

The nature of contribution is different from one action to another and among the different objectives 
they  serve.  For  instance  while  objectives  2  (health  threats)  and  4(better  and  safer  health)  have 
appeared  likelier  to  generate  concrete  benefits  in  the  near  future,  related  to  the  existence  of  EU 
legislation  and  consequently  much  better  focused  actions  with  undisputable  EU-added  value, 
objectives 1(promotion of health) and 3(innovation in health systems) address bigger health needs in 

23   http://jasehn.eu/ 
24   Instructions  for  use,  clinical  data,  post-market  surveillance  data  including  vigilance  information  and 
information about the economic operators (manufacturers, importers, …) and the notified bodies certifying 
the medical devices.  

23 

 
 
                                                           
Member  States  and  their  EU-added  value  depends  on  identifying  gaps  in  the  sharing/uptake  of  best 
practices.  

Actions, especially under objectives 1 and 3, are in need of improved intervention logic and credible 
plans for follow-up. There are also some common challenges for  generating sustainable impacts, for 
instance by ensuring that activities are anchored in the Member States and local contexts, that there is 
ownership of and input in the results. Also accessibility of the programme to all Member States and 
other participating countries (see question on exceptional utility) and new players is another important 
aspect for the success of the Programme objectives.  

5.3 Efficiency 

EQ9: To what extent does the design of the 3HP lead to an efficient allocation of resources 

among objectives / thematic priorities? 

The  Programme  Regulation  (EU)  282/2014  does  not  include  an  obligation  for  spending  precise 
amounts  of  the  EU  budget  on  the  four  specific  objectives  and  the  23  thematic  priorities.  On  the 
contrary the legislation provides for enough flexibility and paragraph (5) states that:  

‘The  emphasis  should  be  placed,  in  accordance  with  the  principle  of  subsidiarity,  on  areas 
where  there  are  clear  cross-border  or  internal  market  issues  at  stake,  or  where  there  are 
significant advantages and efficiency gains from collaboration at Union level’.  

This  said,  the  Programme  has  to  cover  the  maximum  of  the  23  thematic  priorities  still  valid  (as 
concluded by the independent evaluation study) while prioritising and focusing more on some of them.  

The table below shows that more than 73% of the overall Programme budget in 2014-2016 was used 
to  focus  on  a  series  of  major  priorities  throughout  the  four  specific  Programme  objectives.  Another 
23 % has been attributed to actions that are of high EU-added value but on which important synergies 
exist:  it  needs  to  be  noted  that  other  funding  programmes/actions  cover  similar  objectives.  For 
example health threats are addressed by the European Centre for Disease Control, which has an annual 
budget of nearly EUR 60 million dedicated to fighting communicable diseases, and by DG RTD with 
research funding on rare diseases and antimicrobial resistance. 

A very small part of the Programme (approximately 4 %) has been used for supporting action in areas 
such as the health workforce, the prevention and information on drugs related health damage and the 
expert panel on health. eHealth is an exception under this category and funds for this priority will be 
increased in the second half of the Health Programme (see below under EQ 16) as the interest in this 
area  increases  with  the  Digital  Single  Market  Strategy.  The  Health  Programme  will  work  more 
intensively in the coming years to help policy-makers to better understand the digital revolution that is 
taking place across societies and bring this broader perspective to the specific work on digitalisation in 
health  making  sure  that  Europeans  can  benefit  from  the  new  era  of  digitalisation  in  managing  their 
own health. 

24 

 
 
 
 
 
 
 
 
 
Table 9: Budget allocation throughout the thematic priorities 

As a consequence, the budget allocation in the first three years of the Programme implementation is in 
comparison  to  the  past  Programme,  more  strategic  and  balanced  among  the  different  specific 
objectives as the table below shows:  

Figure 8: Budget allocation by objective 2HP and 3HP (2014 – 2016) 

Although the design of the third Health Programme is different from the second, the above observation 
is quite clear as depicted in the following figures. Objective 1(health promotion) has seen its budget 
decreased, the new objective 3 (innovation in health systems) receives a similar proportion of budget 
as objective 1, and objectives 2(health threats) and 4(better and safer health) which with the exception 
of  ERNs  and  rare  diseases  cover  what  in  the  previous  Programme  was  called  ‘health  security’  have 
received together with the priority  ‘HIV/AIDS, tuberculosis and hepatitis’ (currently under objective 
1)25 slightly more credits26 than before.  

25   This priority under the previous second Health Programme was under objective ‘Health Security’.  

25 

Priorities receiving more than 12 million Chronicdiseases(incl.cancer,mentalhealth and dementia)17.2Horizontalactivities(incl.communicationand dissemination activities)7.8Innovation in eHealth2.8Risk factors14.6Capacitybuidingandlegislationonhealth secutity 7.5Mechanismstopoolexpertise 1.8Actions on Migrants and Refugees14.4UnionlegislationonBlood,tissuesandcells7.3Health Workforce1.3HYIV/AIDS- Tuberculosis & hepatitis13.2PatientSafetyandAntimicrobialResistance6Drugsrelatedhealthdamage0.6Health Technology Assessement 12.7Rare diseases4.7European Reference Networks12.6Tobacco legislation4.7Health information11.8EuropeanInnovationonActiveandHealthy Ageing12.2EUlegislationonmedicinalproductsandmedical devices12Total amount120.7Total amount38.0Total amount6.5% of budget 2014-201672.89%% of budget 2014-201622.95%% of budget 2014-20163.93% receiving between approximately 5 to 8 millionPriorites receiving less than 3 million 
 
 
 
 
 
 
                                                           
EQ10: To what extent does the allocation of resources allow for an efficient implementation of 
the 3HP in terms of: funding mechanisms, simplification measures and operational costs? 

The implementation of the first three years of the Programme confirms trends appeared already at the 
end  of  the  second  Health  Programme.  The  part  of  the  budget  dedicated  to  Joint  Actions  and 
procurement  contracts  continue  to  be  increased  in  comparison  to  grants  for  projects.  Moreover,  the 
average  budget  per  action  increases  also  and  this  is  a  good  indication  that  the  Programme  becomes 
more  focused  as  the  consequence  is  to  limit  the  number  of  actions.  Ultimately  the  aim  is  to  gain  in 
efficiency.  

Table 10: Average budget of financial mechanisms, 2HP and 3HP 

for  the  submission  and  evaluation  of 
proposals  the  management  of  grants 
and  monitoring 
e-reporting 
and 
Financial mechanism 

Joint action 

Project 

Operating grant 

Direct grant agreement 

Service contracts – CHAFEA 

Service contracts – DG SANTE  

Conferences 

Average budget per action 

2HP 

3HP 

€2.1 m 

€0.7 m 

€0.2 m 

€0.4 m 

€0.2 m 

€0.2 m 

€0.07 m 

€3 m 

€1.3 m 

€0.3 m 

€0.3 m 

€0.4 m 

€0.07 m 

€0.07 m 

Also the costs needed to administer the third Health Programme, provided for an overall of EUR 10.5 
million  as  administrative  support  credits  and  another  EUR  29.2  million  for  the  functioning  of  the 
Executive  Agency,  for  2014-2020,  suggests  that  the  percentage  of  the  total  budget  proportion 
dedicated to operational costs is 9 %. Benchmarking with other programmes of similar size (consumer 
programme) and with much higher budget (Horizon 2020) demonstrated its efficient execution.  

Table 11: Budgetary data for 3HP, Consumer Programme and Horizon 2020 (2014 – 2020)   

Programme 

Administrative 

support budget 

Total budget 

Executive 
Agency  

budget 

Health Programme 

 €10.5 m 

€29.2m 

€449.4 m 

Consumer Programme 

€7.5 m 

€10.9 m 

€188.8 m 

Horizon 2020 

€3 594.6 m 

€946.2 m 

€74 320.4 m 

Proportion of 
total budget to 
administrative 
support and 
executive 
agency 

9 % 

10 % 

6 % 

Moreover, some simplifications already in the design of the Programme such as harmonisation of co-
funding rates to 60% (or in case of exceptional utility to 80%), the use of framework contracts on a 
three years period for the operating grants, simplifications to requirements for amendment procedures, 
most  importantly  the  ability  for  beneficiaries  to  transfer  resources  between  different  cost  categories 
without the need for an amendment, the introduction of the Multi-annually planning all these converge 
into a smoother and less complex Programme implementation.  

26   All together represent EUR 37.5 million which corresponds to 22.6 %; see detail in the list with budget per 

thematic priority. 

26 

 
 
 
 
                                                                                                                                                                                     
Despite the fact that electronic tools have been introduced with the third Health Programme in 2014 
for  the  submission  and  evaluation  of  proposals,  the  management  of  grants  and  e-reporting  and 
monitoring,  and  Programme  beneficiaries  are  satisfied  with  the  online  migration  of  action-level 
monitoring  (it  has  reduced  the  administrative  burden  for  both  CHAFEA  and  beneficiaries  while 
making  it  easier  to  monitor  progress  in  real  time),  the  administrative  burden  mentioned  by 
stakeholders  as  one  of  the  key  barriers  for  participation  in  the  Health  Programme  (especially  from 
Member States representatives from high GNI countries) and confirmed again through the open public 
consultation, implies that there is still room for improvement in the efficiency.  

However,  the  contractors  recognize  that  actions  require  significant  investment  from  all  stakeholders 
for  being  well  prepared  and  that  more  importantly  and  irrespectively  of  the  size  of  an  action,  the 
biggest driver for efficiency is how effective the action is in achieving its goals and therefore the value 
added.  

EQ11: How may the efficiency of the 3HP be improved regarding number of priorities; funding 

mechanisms; application and implementation procedures and available resources? 

Increasing the effectiveness and impact of a Programme also increases its efficiency. That is why it is 
important to keep the Programme focused on a limited number of outcome-oriented actions relevant to 
Member States needs and aligned to the major Commission priorities.  

Although  no  specific  recommendation  was  made,  the  independent  evaluation  study  did  suggest 
revising the mission of operating grants to non-governmental organisations and making the link with 
the specific Programme objectives more explicit. 

Additionally, choosing the most appropriate financial mechanisms (see table 7 above) and publishing 
clear descriptions of the expected outputs in the Annual Work Programme, after having made a good 
estimation  of  the  necessary  budget  for  the  actions,  gives  stakeholders  and  other  economic  actors 
everything  they  need  to  prepare  good  proposals  and/or  offers  and  facilitates  effective  administrative 
management for the selection and award of actions to be funded (e.g. avoiding the re-launch of calls 
for proposals and calls for tenders and making the best use of CHAFEA resources).  

Moreover,  simplification  measures  that  have  been  already  taken  by  CHAFEA  (e.g.  introduction  of 
electronic tools for the e-submission of proposals, e-managing grants and tenders and e-reporting) are 
expected  to  yield  positive  results  and  in  the  next  years  make  for  more  efficient  use  of  CHAFEA 
resources  to  improve  the  monitoring,  reporting  and  dissemination  of  Programme  results,  which  will 
also have a positive impact on the Programme’s efficiency.  

EQ12: To what extent are the monitoring processes and resources (at the Commission and MS 

level) sufficient and adequate to plan and promote the results of the Health Programme? 

Following up from the findings on the second Health Programme, substantial efforts have been made 
to improve the  monitoring  of the third Health  Programme. Indicators at the level of objectives have 
been introduced27.  

For Objective 1 (promotion of health and diseases prevention) : Number of Member States involved 
in  health  promotion  and  diseases  prevention,  using  evidence-based  and  good  practices  through 
measures and actions taken at appropriate level in the Member Sates 

o  Number of Member States with national initiative for the reduction of saturated fat; 
o  Number  of  Member  States  in  which  the  European  accreditation  scheme  for  breast  cancer 

services is implemented –establishment of the scheme. 

27   Please refer to the point 2.2. ‘Expected outputs, results and impacts’ 

27 

 
 
 
                                                           
For  Objective  2  (health  threats):  Number  of  Member  States  integrating  coherent  approaches  in  the 
design of their preparedness plans;  

For  Objective  3  (health  systems):  Advice  produced  (in  particular  the  number  of  HTA  produced  per 
year)  and  the  number  of  Member  States  using  the  tools  and  mechanisms  identified  in  order  to 
contribute  to  effective  results  in  their  health  systems  (patient  summaries  data/ePrescriptions  in  line 
with EU guidelines) 

For  Objective  4  (better  and  safer  health):  Number  of  ERNs  established  in  line  with  Directive 
2011/24/EU;  number  of  health  care  providers  and  centres  of  expertise  joining  ERN;  number  of 
Member States using the tools developed.  

These  indicators  are  still  far  from  being  comprehensive.  In  some  cases  they  are  much  narrower  and 
more focused than the priorities outlined in the Annex I of the Programme Regulation. For objective 4, 
the indicator is related only to ERNs which are certainly a big part (EUR 12.6 million) of the overall 
budget devoted to this objective (EUR 30.6 million).  

The diversity of actions funded makes it difficult to design and implement standardised indicators (at 
objective  and/or  at  priority  level)  beyond  the  delivery  of  concrete  actions.  For  this  reason  and  in 
addition  to  the  above  indicators  at  Programme  level,  a  set  of  more  specific  indicators  have  been 
defined and, through the use of e-management tools allowing for electronic monitoring at action level, 
have  been  introduced  as  e-questionnaires  for  all  grants  (operating  grants,  grants  to  projects,  joint 
actions and direct grants). The idea is to gather information on type of beneficiary, deliverables and 
the concerned  and targeted population as well as dissemination strategy deployed and  dissemination 
activities  implemented  as  part  of  the  action.  These  indicators  common  to  all  actions  (with  the 
exception of operating grants) could facilitate aggregation and later assessment of EU-added value.  

For operating grants the indicators also include a particular set of indicators on support given on the 
development of health policies to the Commission, support to patients’ empowerment and increase of 
health literacy.  

Unfortunately, due to time constraints the study found little evidence of how the specific indicators for 
projects, joint actions and direct grants have been used, and therefore was not possible to analyse the 
information  and  assess  its  accuracy  for  reporting  purposes.  The  e-questionnaires  were  sent  to 
beneficiaries  in  late  2016,  while  the  ones  for  operating  grants  should  have  been  available  for  the 
evaluation since it has been in place for a while.  

The problems with the monitoring of the Health Programme are well known also from evaluations of 
previous  Programmes.  In  those  cases,  the  main  reason  was  the  complete  absence  of  monitoring 
indicators.  The  mid-term  evaluation  revealed  that  despite  the  substantial  efforts  for  the  definition  of 
indicators, significant shortcomings continue to be unaddressed. Information is not stored or organised 
in an efficient manner so that data are ready to be analysed. The fact that a part of the Programme is 
managed by CHAFEA and another smaller part by DG SANTE makes it difficult, in the absence of a 
well-design  monitoring  tool,  to  have  rapidly  the  complete  and  correct  picture  of  how  the  budget  is 
spent and for which kind of outputs. Linking up data manually with thematic priorities and collate and 
cross-check  the  information,  as  we  have  done  for  the  outsourced  evaluation,  is  a  very  resource 
intensive exercise and prone to error. 

It seems that the monitoring is used only for financial accountability purposes, but this is only a part of 
what has to be the monitoring of a Programme. With a good monitoring tool, the information gathered 
on  actions  and  their  state  of  implementation  including  information  on  progress  indicators  could  be 
used to better publicise the activity of the Health Programme. By making results of actions and other 
information  publicly  available  and  trying  to  ensure  that  the  results  reach  the  right  kinds  of 
stakeholders, this can facilitate their take-up beyond those who are directly involved in the actions.  

The creation of a new project database has  been repeatedly postponed by CHAFEA and the existing 
one  has  not  been  updated  with  the  new  Programme  actions  neither  improved  at  the  moment  this 

28 

 
 
document was being drafted. The project database28 currently only contains all actions for year 2014, 
but  no  information  for  Joint  Actions  2015,  and  no  information  about  awarded  actions  in  2016. 
Moreover,  none  of  the  procurement  contracts  and  direct  grants  are  covered  by  this  database,  for  all 
previous Programmes and for the current one.  

Positive developments are noted on publication and dissemination actions. Info sheets29 on 12 health 
topics and in all 23 EU languages have been published on the CHAFEA Website. CHAFEA together 
with  the  National  Focal  Points  have  run  a  variety  of  well-attended  events:  16  events30  have  been 
organised in 2015-2016, and another 11 that are planned or already implemented in 2017. The report 
CHAFEA made on these dissemination activities has noted the high-level of satisfaction and success 
of these events organised mainly in Member States and financed under the third Health Programme.  

CHAFEA  has  developed  in  cooperation  with  DG  SANTE  the  dissemination  strategy,  this  also 
recommended  in  the  ex-post  evaluation  of  the  second  Health  Programme.  The  study  notes ̶the 
increasing  prioritisation  and  professionalism  awarded  to  dissemination’  and  mention  also  the  Health 
Policy Platform31 launched in 2016 by DG SANTE and the use made of it to disseminate information 
among relevant stakeholders for instance training packages for health professionals and frontline staff 
working with refugees and other migrants.  

5.4 EU-added value 

EQ13: To what extent are the seven EU added value criteria addressed in proposals?    

The  EU-added  value  is  defined  in  the  third  Health  Programme  and  enshrined  in  the  Programme 
Regulation (EU) 282/2014 in paragraph (6):  

‘The  Programme  should  be  a  means  of  promoting  actions  in  areas  where  there  is  a  Union 
added  value  that  can  be  demonstrated  on  the  basis  of  the  following:  exchanging  good 
practices  between  Member  States;  supporting  networks  for  knowledge  sharing  or  mutual 
learning;  addressing  cross-border  threats  to  reduce  their  risks  and  mitigate  their 
consequences; addressing certain issues relating to the internal market where the Union has 
substantial  legitimacy  to  ensure  high-quality  solutions  across  Member  States;  unlocking  the 
potential  of  innovation  in  health;  actions  that  could  lead  to  a  system  for  benchmarking  to 
allow informed decision-making at Union level; improving efficiency by avoiding a waste of 
resources due to duplication and optimising the use of financial resources.’ 

The above with the exception of issues related to the internal market and the potential of innovation in 
health, come from the criteria the Executive Agency has defined and which were used in the second 
Health Programme.  

With  the  third  Health  Programme  these  criteria  became  a  clear  while  not  distinctive  part  of  the 
evaluation  criterion  ‘contribution  to  public  health  in  Europe’  for  Joint  Actions  and  ‘policy  and 
contextual relevance’ for projects and operating grants. The procurement contracts are not assessed for 
their EU-added value as this is part of the decision process for launching the procurement procedure 
and defined in the terms of reference.  

The case studies investigating the application of the EU-added value criterion showed that the process 
for  assessing  EU-added  value  is  improving  and  becoming  more  systematic.  Building  the  EU-added 
value  criteria  into  the  application  process  for  funding  has  been  a  major  improvement:  most  of  the 

28   http://ec.europa.eu/chafea/projects/database.html 
29   http://ec.europa.eu/chafea/health/hp-infosheets_en.html 
30   http://ec.europa.eu/chafea/news/news.html 
31   https://webgate.ec.europa.eu/hpf/ 

29 

 
 
                                                           
 
potential  beneficiaries  consider  EU-added  value  when  preparing  their  proposals,  and  the  assessment 
panels in turn take it into account as part of the decision to award funding.  

The short time elapsed since the action plan agreed on recommendations of the ex-post evaluation of 
the second Health Programme in March 2016, was the main reason for specific recommendations on 
EU-added  value  to  not  being  fully  implemented  by  CHAFEA  and  DG  SANTE.  This  is  about  the 
redefinition of three EU-added value criteria related to the exchange of best practice, networking and 
benchmarking  for  better  decision-making.  The  recommendation  was  to  make  them  more  detailed, 
concrete  and  outcome  focused  (e.g.  plausible  plans  to  see  best  practices  actually  implemented  than 
only  shared).  For  best  practices  the  discussion  has  already  started  in  the  Steering  Group  on  Health 
Promotion,  Disease  Prevention  and  Management  of  Non-Communicable  Diseases,  established  in 
November 201632.(see above point 2.2.1). 

The study showed that there is still scope for being more systematic in the assessment of the EU-added 
value  criteria  and  to  make  it  clearer  for  applicants  through  concrete  examples  what  the  EU-added 
value is and how it feeds into the award process.  

5.5 Coherence 

EQ14: To what extent have the thematic priorities of the 3HP led to more synergy, focus and 

coherence between the funded actions in delivering on the objectives? 

Compared to the previous Health Programme, the structure of the third Health Programme has been 
revised with improved specific objectives providing more clarity and focus, and further elements for 
operationalisation have been introduced which was not the case in the previous Programmes. Instead 
of longer  priorities and sub-priorities often repeated under different objectives, now the third Health 
Programme  has  consolidated,  concrete  thematic  priorities  unique  under  each  of  the  four  specific 
objectives.  In  almost  all  cases  actions  now  fall  under  one  thematic  priority,  even  if  synergies  with 
other  priorities  are  possible,  the  increased  purposiveness  and  topical  focus  of  the  thematic  priorities 
enables greater clarity and coherence in the scope of actions. In some cases, it was possible to organise 
a  sizeable  action  (e.g.  ‘Best  practices  in  care  provision  for  vulnerable  migrants  and  refugees’  in 
Annual Work Programme 2016) targeting on purpose two thematic priorities simultaneously (priority 
1.3  on  HIV/AIDS,  tuberculosis  and  hepatitis,  and  priority  1.4  relating  to  chronic  diseases.  There  is 
consensus from all interviewees, public health experts in focus groups, case studies and respondents to 
the  open  public  consultation  that  the  new  structure  of  the  Programme  is  a  success  enabling  more 
concretely defined areas of intervention.  

Moreover,  synergies  exist  between  the  actions  under  the  same  or  different  thematic  priorities.  The 
structure of the Programme certainly enables such cross-cutting work, but most importantly this is the 
result of successful exploitation of relations between the different actors/beneficiaries as well as good 
programme  management  and  communication  with  stakeholders.  For  instance  the  European  Cancer 
League  (ECL)  which  is  funded  through  an  operating  grant  under  thematic  priority  1.4  for  chronic 
diseases has a memorandum of understanding with the Smoke Free Partnership which receives also an 
operating  grant  from  the  third  Health  Programme  under  priority  1.4  on  risk  factors,  creating  a 
framework for joint work and the sharing of perspectives. The ECL is also an associated partner in the 
CANCON  joint  Action  (funded  under  the  second  Health  Programme)  and  provided  inputs  on 
survivorship  and  rehabilitation  throughout  2016.  ECL  uses  the  CANCON  newsletter  to  promote  the 
European Code Against Cancer one of its key activities funder by the operating grant.  

Other  examples  illustrate  how  the  third  Health  Programme  builds  on  the  success  of  previous  work 
undertaken in the previous Programme. For instance the Joint Action EMERGE launched in 2014 is 

32  http://ec.europa.eu/newsroom/sante/newsletter-specific-archive-

issue.cfm?newsletter_service_id=327&newsletter_issue_id=2820&page=1&fullDate=Fri%2017%20Mar%2
02017&lang=default 

30 

 
 
                                                           
 
the result of merging of two networks of highly pathogenic agents, funded in 2010 as the QUANDHIP 
Joint Action.  

In some other cases, the sequencing of different actions (e.g. the Joint Action on rare diseases (RD-
Action)  which  built  on  the  work  of  the  previous  joint  action  in  the  second  Health  Programme  and 
supports  the  adoption  of  a  codification  and  knowledge  management  system  for  rare  diseases,  which 
will be necessary for ERNs to diagnose patients and share knowledge and expertise across the EU , 
together with the Study on the Manual and Toolbox for Assessing ERNs whose objective is to develop 
an assessment manual that will serve as an evaluation framework for ERNs, all that together prepares  
and contribute to the creation , implementation and operation of ERNs (see point 2.2.4). 

However,  there  is  still  risk  in  areas  where  the  third  Health  Programme  has  ambitious  goals  and  the 
scope  is  broad  (e.g.  chronic  diseases)  to  see  actions  fail  either  because  they  do  not  relate  closely  to 
other  ones  (different  actions  in  different  areas  such  as  mental  health,  dementia,  cancer)  or  simply 
because they cannot make the most of potential links.   

EQ15: To what extent are the objectives and thematic priorities of the 3HP externally 

consistent/coherent i.e. is there correspondence between: the 3HP and health objectives of 
Article 168 TFEU, and those of other public interventions (e.g. national health policies, EU 
policies and Programmes, other international actions)? 

The Health Programme is linked to the high-level priorities of Europe 2020 Strategy33 and since 2015 
is  oriented  towards  the  Commission  10  priorities34  (known  as  "Juncker  priorities")  and  DG  SANTE 
strategic objectives.  

It  emphasises  the  links  between  economic  growth  and  a  healthy population,  to  a  greater  extent  than 
previous  programmes,  concentrates  on  health  systems  as  a  productive  factor  for  growth  and  jobs, 
contributes  to  smooth  functioning  of  the  internal  market  by  supporting  EU  legislation  on  tobacco, 
medicinal  products  and  medical  devices,  cross-border  health  care  directive,  and  Health  Technology 
Assessment, as well as to the digital single market with actions related to eHealth ; the Programme has 
demonstrated  its  ability  to  undertake  rapidly  appropriate  actions  for  migrants  and  refugees35  as 
migration is also an important issue, and fights against health inequalities for the respect of justice and 
fundamental rights. Last but not least, its actions on preparedness and management of serious cross-
border health threats for a coordinated response to highly dangerous outbreaks (e.g. Ebola and Zika) 
ensure the protection of EU population's health and security.  

In almost all the above mentioned areas the Work Programme works closely with other EU Policies 
and  Programmes  such  as  Horizon  2020  (e.g.  on  Health  Technology  Assessment,  anti-microbial 
resistance etc.), migration policy, chronic diseases and reduced ability to work, taking into account the 
principle of ‘health in all policies’.  

Other  financial  mechanisms  such  as  the  European  Social  Fund  (ESF)  and  the  European  Regional 
Development Fund (ERDF) from the EU Structural and Investment Funds (ESIF) could also be better 
used to scale-up actions and results of the Health Programme as objectives and thematic priorities are 
coherent with these instruments. While the Programme informs about these possibilities and promotes 
ESIF, evaluators have not found evidence that this happens practically probably due to the early stage 
of the third Health Programme’s implementation.  

33   Regulation (EU) 282/2014 paragraph (2). 
34   https://ec.europa.eu/commission/priorities_en 
35   € 14.4 million was dedicated for actions on migrants and refugees, making this the second most important 

budget envelop behind chronic diseases. 

31 

 
 
 
                                                           
At international level, through EuropeAid the EU supports global health initiatives such as the Global 
Fund  to  fight  HIV/AIDS,  Tuberculosis  and  Malaria,  and  GAVI,  the  Global  Alliance  for 
Immunizations. The Commission has been a founding member of the Global Fund and has contributed 
more than EUR 1.6 billion, which corresponds to about 6 % of the USD 30 billion that the Fund has 
committed  to  140  countries  so  far.  As  a  result,  10  million  people  are  on  anti-retroviral  therapy  for 
AIDS, 16.6 million people have been tested and treated for tuberculosis, and 713 million insecticide-
treated nets have been distributed to protect families against malaria.  

Commissioner Mimica pledged in September 2016 a further EUR 475 million to the Global Fund for 
the period 2017-2019, which is an increase of EUR 105 million (or 28 %) to the current contribution. 
GAVI  has  supported  the  full  vaccination  of  277  million  children  in  2010-2015,  with  the  goal  of  a 
further  300  million  fully  vaccinated  children  in  the  period  2016-2020.  These  global  initiatives 
complement thematic priority 1.3 on HIV/AIDS. 

Tuberculosis and hepatitis and objective 2 on health threats of the Programme and under this objective 
close cooperation is developed with WHO and G7 within the Global Health Security Initiative for the 
creation of an effective and well-organised global strategy for preparedness and responses to potential 
health threats.  

The EU through its development instruments also supports initiatives to provide technical assistance to 
regional  capacities  in  order  to  build  preparedness  and  response,  such  as  the  West  African  Regional 
Centre  for  Disease  Surveillance  and  Control,  and  working  with  the  WHO  to  build  robust  national 
animal  health  and  veterinary  public  health  surveillance  systems.  EU  support  to  the  WHO  has  also 
made  it  possible  to  tackle  the  growing  threat  of  antimicrobial  resistance  with  a  first  survey  on 
availability and prices of antibiotics being carried out in the sub-Saharan region. Also through the third 
Health  Programme  the  EU  makes  an  annual  contribution  to  the  WHO’s  European  Observatory  on 
Health Systems and Policies, and the  Framework Convention  on Tobacco Control (FCTC)  in which 
the EU is part, complements the Programme’s priorities 1.1 on risk factors (tobacco use) and 1.5 on 
tobacco legislation. Furthermore EuropeAid provides support to strengthen the capacity of developing 
countries  that  are  parties  to  the  WHO  FCTC  so  that  they  can  meet  their  obligations  under  the 
Convention. 

However,  the  Programme  remains  quite  invisible  as  an  international  policy  instrument  and  this  is 
expected as it concentrates by definition on capacity building within the EU and the other participating 
countries.  From  2016  onwards,  Serbia  and  Moldova  participate  in  the  third  Health  Programme  and 
Bosnia  and  Herzegovina  from  2017  onwards,  while  other  Western  Balkan  countries  have  expressed 
also interest. This may be interpreted as the increasing importance of the health Programme objectives 
and priorities for these countries. Moreover, the Programme has a great potential in dealing with issues 
that come up on the international agenda such anti-microbial resistance and the impacts of the climate 
change on health and environment.  

5.6 Utility  

EQ16: To which extent is the 3HP overall useful and, if necessary, how could its overall utility be 
increased?  What  are  the  specific  needs  of  Member  States  to  which  the  Programme  could 
provide a concrete solution but has not done so yet? 

The Health Programme is our unique financial instrument to support policy coordination with Member 
States  and  key  health  stakeholders  (including  health  professionals),  and  to  make  progress  on  the 
agreed Programme objectives and strategic targets linked to the Commission priorities.  

The  Programme  is  well  positioned  to  serve  Member  States  in  areas  where  EU-added  value  can  be 
generated:  

- 

under the Commission priority on ‘growth, jobs and a resilient society’, the Programme is 
promoting the health of the population and health care services as a productive factor for 
growth and jobs,  

32 

 
 
- 

- 

on the ‘digital single market’, the Programme promotes eHealth and helps Member States to 
invest in new technologies,  
on the ‘internal market’,  the Programme finances actions supporting  EU legislation on 
tobacco, pharmaceuticals, medical devices, the cross-border health care directive, and Health 
Technology Assessment,  
on‘justice and fundamental rights’, the Programme helps to eradicate health inequalities, and  

- 
-  makes a significant contribution to the ‘migration’ policy through actions on migrants and 

- 

refugees; 
on ‘security in Europe’ the Programmes increases the capacity of Member States to prepare 
for and manage serious cross-border health threats. 

Moreover,  the  Programme  covers  the  development  and  implementation  of  EU  health  legislation  (on 
tobacco, cross-border health threats, medicinal products, medical devices, cross-border healthcare and 
substances  of  human  origin)  and  provides  support  for  policy  coordination  and  the  development  of 
health information infrastructure.  

The  programme  facilitates  networking  among  the  public  health  community  in  Europe  and  with 
international  initiatives,  promotes  best  practices  and  helps  Member  States  to  reach  their  sustainable 
development  goals,  in  particular  SDG  3  ‘Ensure  healthy  lives  and  promote  wellbeing  for  all  at  all 
ages’ . 

In the coming years the Programme will prioritise the following areas while addressing health 
inequalities in a cross-cutting manner. : 

- 
- 
- 

- 

country specific and cross country knowledge (addressing all objectives of the programme) 
structural support to health systems and link to digital single market 
cross-border health threats, preparedness and response, including  antimicrobial resistance; 
and 
promotion of health and prevention of non-communicable diseases.  

The utility of the Programme increases with the use of the Programme outcomes and the integration of 
best practices into the national health policies. The Programme works as a tool for policy coordination, 
for the collection and analysis of harmonised and comparable health data and as an incubator of good 
practices and pilot projects. The Programme outputs are mainly addressed to policy makers and health 
professionals who are expected to make use of them on a large scale (e.g. colorectal cancer screening). 
This will be a real test for the Programme's usefulness.  

By undertaking the challenge to create synergies with other bigger EU financial instruments such as 
the European Structural Funds and the Structural Reform Support Service for the up-scale of its 
actions, the Programme can take a decisive step that maximises its impact on the health of Europeans 
and boosts the sustainability of the health systems in the EU.  

5.  CONCLUSION 

The third Health Programme is  highly relevant to Member States health needs and the objectives 
set are clear, explicit and specific. The 23 thematic priorities are still valid and help the Programme to 
better  focus  on  outcome  oriented  actions.  In  the  longer  term,  contractors  recommended  to  consider 
further  streamlining  some  thematic  priorities  to  avoid  any  potential  overlap  or  ambiguities.  All  of 
these are indeed relevant to real needs in Member States and are in line with high-level Commission 
priorities and existing challenges.  

33 

 
 
 
 
 
The Programme demonstrates significant progress in a number of areas building on previous work 
and the first signals of its effectiveness are the establishment of 24 European Reference Networks 
for  rare  diseases,  the  adoption  of  common  methodologies  for  Health  Technology  Assessment  ,  the 
delivery of tools and piloting of joint assessments, the support to Member States for increasing their 
capacity  building  and  respond  to  outbreaks,  the  continuing  updating  of  skills  for  health 
professionals and other front line staff working on emergent issues such as the refugees crisis.  

Moreover, the Health Programme supports implementation of EU health legislation in a series of areas 
such as tobacco, medicinal products, medical devices, cross-border healthcare, etc., closely related to 
the priority of safe and well-functioning internal market.  

The Programme demonstrated its responsiveness and flexibility to take into consideration emerging 
needs such as the influx of refugees in 2015-2016. From 2017 onwards, The Programme supports also 
the Solidarity Corps.  

The  Programme  management  has  become  increasingly  effective.  The  Multi-annual  planning  has 
enabled a more strategic approach to medium term planning. The process for the adoption of Annual 
Work  Programmes  is  clear,  well-defined  and  impartial  while  it  seems  to  be  not  so  well  known  by 
stakeholders and has to be better explained in order to ensure its buy-in and avoid misunderstandings. 

The  Programme  management  has  been  improved  in  comparison  to  the  past  with  introduction  of 
indicators, and simplified with the use of electronic tools for submission of proposals and management 
of  grants,  but  also  simplified  rules  and  procedures.  There  is  still  need  for  improvement  in  better 
monitoring  and  reporting  on  actions,  while  the  dissemination,  although  improved,  needs  to  be 
prioritised  (new  Project  database  and  operationalization  of  the  newly  established  Dissemination 
strategy).  Otherwise,  the  Programme  suffers  from  low  visibility  and  risks  to  lack  credible  elements 
demonstrating its progress and utility.  

By and large the allocation of resources is efficient and the operational costs for its execution remain 
at acceptable level. By increasing its effectiveness, the Programme can also increase its efficiency.  

The  Programme  attracts  participation  of  all  EU  Member  States  and  EFTA  countries  (Norway  and 
Iceland)  and  three  new  countries  from  the  Balkan  Peninsula  (Serbia,  Moldova,  and  Bosnia  and 
Herzegovina) joined it recently. Financial incentives are proposed through exceptional utility criteria 
for boosting participation from low GNI countries and the increased co-funding rate of 80% applies to 
all  the  participants  in  an  action.  Evidence  from  the  first  three  years  of  Programme  implementation 
shows that these opportunities have not been sufficiently used and low GNI Member States mentioned 
in  the  e-survey  and  in  the  open  public  consultation  that  assuring  co-funding  is  one  of  the  most 
important barriers for participation in the Health Programme.  

The  Programme  has  increased  its  ability  to  target  important  health  needs  where  it  can  add 
value.  By  anchoring  the  seven  EU-added  value  criteria  into  the  Programme  Regulation  and  making 
them  part  of  the  decision-making  process  for  awarding  funding,  the  applicants  are  informed  well  in 
advance and consider the EU added value when they are preparing their proposals. This is conducive 
to better designed and explained proposals for the consideration of the assessment panels and so, the 
actions awarded for funding can operate in a way that adds value at EU level. The study showed that 
there is still scope for being more systematic in the assessment of the EU added value criteria and to 
make it clearer for applicants through concrete examples what the EU added value is and how it feeds 
into the award process. Objectives for responding to cross-border health threats and giving access to 
better and safer health seem to produce outcomes with undisputable high EU-added value.  

The  Health  Programme  is  coherent  with  major  Commission  priorities  and  policies,  and  with 
Sustainable  Development  Goals.  To  achieve  outcomes  and  results  that  addresses  the  existing 
challenges in a comprehensive manner, the Health Programme operates in synergies, either among its 
own priorities and their corresponding actions (internal coherence) but also with other DGs (external 
coherence) on the basis of the principle of ‘health in all policies’ and with specific EU Programmes 
such as the Horizon 2020 and/or with its predecessor the 7th Framework Programme. Possibilities to 

34 

 
 
increase synergies and ensure the up-scale of successful Programme actions and results exist through 
other financial instruments, such as the EU Structural and Investment Funds and should be used.  

35 

 
 
 
 
ANNEX I: PROGRAMME DESCRIPTION 

A. Thematic priorities 

1. Promote health, prevent diseases and foster supportive environments for healthy lifestyles 
1.1. Risk factors such as use of tobacco and passive smoking, harmful use of alcohol, unhealthy 
dietary habits and physical inactivity 
1.2. Drugs-related health damage, including information and prevention 
1.3. HIV/AIDS, tuberculosis and hepatitis  
1.4. Chronic diseases including cancer, age-related diseases and neurodegenerative diseases  
1.5. Tobacco legislation  
1.6. Health information and knowledge system to contribute to evidence-based decision-making  

2. Protect Union citizens from serious cross-border health threats 
2.1; Additional capacities of scientific expertise for risk assessment  
2.2. Capacity-building against health threats in Member States, including, where appropriate, 
cooperation with neighbouring countries  
2. 3. Implementation of Union legislation on communicable diseases and other health threats, 
including those caused by biological and chemical incidents, environment and climate change  
2. 4. Health information and knowledge system to contribute to evidence-based decision-making 

3. Contribute to innovative, efficient and sustainable health systems 
3.1 Health Technology Assessment  
3.2. Innovation and e-health  
3.3. Health workforce forecasting and planning  
3.4. Setting up a mechanism for pooling expertise at Union level  
3.5. European Innovation Partnership on Active and Healthy Ageing  
3.6. Implementation of Union legislation in the field of medical devices, medicinal products and cross-
border healthcare  
3.7. Health information and knowledge system including support to the Scientific Committees set up 
in accordance with Commission Decision 2008/721/EC  

4. Facilitate access to better and safer healthcare for Union citizens 
4.1. European Reference Networks  
4.2. Rare diseases  
4.3. Patient safety and quality of healthcare  
4.4. Measures to prevent antimicrobial resistance and control healthcare-associated infections  
4.5. Implementation of Union legislation in the fields of tissues and cells, blood, organs  
4.6. Health information and knowledge system to contribute to evidence-based decision-making 

36 

 
 
 
 
 
 
 
B. Further explanations on what results are being achieved and which are the impacts expected 
from the Programme? 

Objective  1:  Promote  health,  prevent  diseases  and  foster  supportive  environments  for  healthy 
lifestyles 

The  third  Health  Programme  action  is  helping  to  collect  evidence  and  data  on  e.g.  alcohol 
consumption and alcohol policies in all Member States, as well as national policies related to nutrition, 
physical inactivity, and overweight and obesity related diseases. To help Member States develop better 
and more coordinated policies, an online reference guide as one-stop clearing house for independent, 
reliable  and  up-to  date  information  on  a  wealth  of  topics  related  to  nutrition,  physical  activity  and 
prevention  of  chronic  conditions  is  being  developed  by  the  Commission’s  Joint  Research  Centre 
financed by the third Health Programme. The guide is written for policy-makers in EU Member States 
and provides concise information that can be quickly used to answer their questions and translate to 
their policy work, while it is also open to the other professionals in the relevant fields as well as to the 
general public. 

Tobacco use, unhealthy diet, physical inactivity and alcohol consumption are the main risk factors 
responsible for the biggest part of chronic diseases that affect the capability to work, are costly to treat 
and  ultimately  impact  economic  performance.  Chronic  diseases  account  for  more  than  80  %  of  the 
disease burden of the EU and are thus a major cause for high health expenditures. Through the third 
Health Programme the European Union joins its forces with the 28 Member States to combat chronic 
diseases  and  promote  active  &  healthy  ageing.  To  do  so,  based  on  previous  work  undertaken 
identifying  good  practices,  e.g.  in  the  area  of  health  promotion  or  defining  better  care  models  for 
multi-morbid  patients,  the  third  Health  Programme  focuses  on  supporting  Member  States  to  make 
decisive step towards implementing good practices: what has functioned well in other Member States 
should  be  piloted  elsewhere  or  up-scaled  internally.  To  strategically  enhance  the  transfer  of  best 
practices  DG  SANTE  established  a  Steering  Group  on  Health  Promotion,  Disease  Prevention  and 
Management of Non-Communicable Diseases in November 201636. 

The Health Programme has developed the EU-Compass for Action on  mental health and wellbeing 
which is a web-based mechanism used to collect exchange of best practices and analyse information 
on policy and stakeholder activities in mental health. The Compass is piloting the common criteria on 
best practices selected at EU level and approved by the Steering group on Promotion and Prevention, 
and the results will be presented in October 2017.  

On  dementia,  and  in  particular  post-diagnostic  support,  crisis  and  care  coordination,  quality  of 
residential care and dementia friendly communities, best practices already selected are being piloting 
under second Joint Action with Member States competent authorities.  

On  cancer,  the  European  Quality  Assurance  scheme  is  being  developed  in  harmonised,  evidence-
based  and  flexible  way  to  grant  equal  and  quality-benchmarked  treatment  to  patients.  The  mid-term 
evaluation  of  the  breast  cancer  initiative37  show  that  all  the  activities  planned  toward  the  official 
launch of the European Quality Assurance scheme planned for its final release in 2019 is on-track with 
the  publication  of  European  Breast  Guidelines  currently  ongoing  and  with  the  timeline  for  the 
launching of the Guideline platform, the digital breast screening training template and the piloting of 
the European Quality Assurance scheme in 2018. So far 10 recommendations have been published and 
are currently available for consultation by patients, health care workers and policy makers through the 

36  The  Steering  Group  on  Health  Promotion,  Disease  Prevention  and  Management  of  Non-Communicable 
Diseases  is  currently  an  informal  group.  For  more  information:  http://ec.europa.eu/dgs/health_food-
safety/dyna/enews/enews.cfm?al_id=1736 

37   Interim reports have been regularly provided and a number of annexes and deliverables have been published 

on the JRC web site. 

37 

 
 
                                                           
 
Joint  Research  Centre  (JRC)  interactive  IT  tool38.  A  further  publication  of  20  recommendations  has 
been forecasted before the end of 2017. On the implementation of breast centres in the Member States 
the Commission receives the official information that although breast units are required by law in few 
countries. They are recommended and implemented in 17 Member States.  

The  activities  of  the  European  Network  of  Cancer  Registries  coordinated  by  the  JRC  delivers 
important results providing a ‘data-brokering’ service to ensure integrity of a single European dataset 
for different purposes. 128 Cancer Registries from 29 European countries are regularly providing data 
to JRC with more than 25 900 000 records so far in the database.  

The  CANCON  joint  action  from  the  previous  second  Health  Programme  reached  the  end  in  March 
2017 delivering the European Guide on Quality Improvement in Comprehensive Cancer Control and 
five policy papers. The Guide has been adopted by the Expert Group on Cancer Control and it will be 
the  base  for  the  further  upgrade  of  the  National  Cancer  Plans  and  their  implementation  during  the 
oncoming years39.   

Finally the Health Programme supports the involvement of civil society in a continuous dialogue with 
the EU on health issues by granting almost EUR 5 million annually to non-governmental bodies. It is 
expected that these non-governmental bodies assist the Commission with the information and advice 
necessary for the development of health policies and the implementation of the Programme objectives 
and priorities. It is also expected that non-governmental bodies will work on increased health literacy 
and promotion of healthy life styles, the organisation of science policy conferences and contribute to 
the  optimisation  of  healthcare  activities  and  practices  by  providing  feedback  from  and  facilitating 
communication with patients thus empowering them.  

A  prize  for  non-governmental  organisations  is  organised  annually  by  the  Health  Programme  for 
encouraging  those  organisation  that  made  a  significant  contribution  fighting  against  the  Ebola 
outbreak in 2015, reducing antimicrobial resistance in 2016, and for the promotion of vaccination in 
2017. 

Objective 2: Protect Union citizens from serious cross-border health threats 

In brief, actions funded under this objective are geared towards ensuring on the ground safety, i.e. it 
included  the  possibility  of  switching  ‘mode’  from  preparation  to  response  mode  in  the  event  of  an 
outbreak. 
For instance during  the Ebola and Zika outbreaks, part  of the funds of the programme  were used to 
support interventions to limit the spread of these threats by strengthening Member State preparedness 
and  response  in  particular  through  the  actions  of  the  Health  Security  Committee  (entry  screening, 
medical  evacuations,  prevention  of  transmission  in  transport  and  hospital  settings).  The  budget  in 
2014-2016 for strengthening EU response to health threats amounted to EUR 11 million.  

38   See at: http://ecibc.jrc.ec.europa.eu/ 
39   The implementation of measures and the update of the National cancer Plans will be developed as part of the 
new  Joint  Action  on  Cancer  Prevention  and  Control  (Annual  Work  Programme  2017).  Specific  best 
practices will be integrated in the National Cancer Plans with the support of the Steering Group on Health 
Promotion, Disease Prevention and management of Non-Communicable Diseases. 

38 

 
 
 
 
 
 
                                                           
Objective  3:  Contribute 

to 

innovative,  efficient  and 

sustainable  health 

systems 

Health Technology Assessment (HTA) 

Based on the lessons, successes and products of the earlier EUnetHTA Joint Actions , the third Health 
Programme in 2015 provided EUR 12 million for a third Joint Action on HTA in 2016-2020.  

The cooperation has grown to include 81 organisations from 29 countries which constitute a network 
of  strong  partners  across  Europe  working  together  for  better  access  to  health  technologies  for 
Europeans. 

This  joint  action  contributes  to  the  use,  quality  and efficiency  of joint  HTA  work  at  European  level 
and supports evidence-based, sustainable and equitable choices in healthcare and health technologies 
and support re-use in regional and national HTA reports and activities.  

The  previous  joint  actions  produced  in  total  20  Joint  Assessment,  15  on  pharmaceuticals  and  5  on 
medical  devices.  The  target  of  the  third  joint  action  is  to  have  by  2020,  a  total  of  80  Joint  Health 
Technology Assessments done including on pharmaceuticals and medical devices.   

Although the joint actions have been successful in building the necessary trust between HTA bodies to 
establish  methodologies,  tools  and  piloting  of  joint  assessments,  they  have  also  showed  some 
shortcomings. Notably, 1) the non-sustainable financing (i.e. Joint Actions are only supposed to "kick-
start"  cooperation  and  are  not  a  tool  to  sustain  scientific  and  technical  cooperation  indefinitely)  ,  2) 
limited uptake of joint work (i.e. joint assessments or jointly agreed tools and methodologies, are not 
always  used  in  national/regional  HTA  activities)  and  3)  a  project  based  model  not  providing  the 
necessary  continuity  of  established  work  (i.e.  the  time  need  to  set  up  and  close  a  Joint  Action  and 
negotiate a new one, lead to important production gaps, during which no joint assessments or scientific 
work was performed) .  

Innovation and ehealth  

The overall ambition from EU Member States (MS) is to better include eHealth into health policy and 
better align eHealth investments to health needs. A central aspect is the transferability of health data 
across  borders  of  Member  States  and  therefore  the  organisational,  technical,  semantic  and  legal 
interoperability. In order to ensure progress and to bridge the gaps between the governance, strategy 
and operational levels, a dedicated mechanism for eHealth at EU level has been established in 2011   
and represents the highest decision-making body at EU political level.  

The Health Programme has provided regular support to the eHealth Network and in the period 2014-
2017  a  Joint  Action  has  brought  together  Member  States'  competent  authorities  to  ensure  further 
common political leadership and ongoing integration of eHealth into health policy in order to continue 
developing eHealth services responding to health systems needs and health objectives.  

For this political recommendations and instruments for cooperation have been developed in the below 
four specific priority areas that are specified in the eHealth Network's multi-annual work plan 2015-
2018 and were adopted by the eHealth Network in May 2014: (i) Interoperability and standardisation, 
(ii) monitoring and assessment of implementation, (iii) exchanging knowledge and (iv) cooperating on 
a  global  level.  These  results  can  feed  into  the  reflection  on  the  contribution  of  health  in  the  Digital 
Single Market. 

39 

 
 
 
 
 
European Innovation Partnership on Active and Healthy Ageing  

Based  on  several  projects  and  one  joint  action  which  focus  on  the  topics  of  frailty,  one  overall 
European Framework for Frailty Prevention is being developed as well as practical tools e.g. to screen 
older adults for (pre)frailty and innovative care paths.  

Expert Panel on Health  

Sound  and  timely  scientific  advice  is  an  essential  requirement  for  providing  information  and 
knowledge  so  as  to  pursue  modern,  responsive  and  sustainable  health  systems.  To  this  end,  the 
Commission  has  set  up  a  multi-sectorial  and  independent  Expert  Panel40  which  provides  advice  on 
effective ways of investing in health. The concept of this expert panel became particularly compelling 
during  the  times  of  financial  crisis  and  budgetary  constraints  in  part  due  to  the  ongoing  reflection 
processon  health  systems  within  the  European  Commission.  In  the  period  2014-2016,  the  panel 
delivered 12 Opinions41 in line with the Mandates raised (i.e. in relation to primary care and access to 
healthcare),  and  has  provided  expert  advice  and  input  to  Member  States  and  the  European 
Commission. By calling on additional expertise from a database of experts and from the European and 
international  health  care  expert  community,  the  expert  panel  has  the  potential  for  promoting 
innovation in health, and generating economies of scale by connecting expertise in areas of identified 
importance.  

Scientific Committees 

The Health Programme supported the functioning of the Commission Scientific Committee on Health 
and Environmental Risks (SCHER), the Committee on Health, Emerging and Newly Identified Risks 
(SCENIHR)42  and  the  Scientific  Committee  on  Consumer  Safety  (SCCS)  which  provided  scientific 
advice  and  risk  assessment  to  the  Commission  in  the  areas  of  health,  environment,  consumer  safety 
and  emerging  risks.  In  the  period  2014-2016,  the  SCHER  produced  5  Opinions,  the  SCENIHR  18 
Opinions  and  the  SCCS  64  opinions,  on  topics  such  as  the  safety  of  cosmetic  ingredients  including 
nanoform, the safety of medical devices such as surgical meshes or PIP breast implants, the health and 
environmental  effects  of  mercury  from  dental  amalgam,  the  potential  health  effects  of  exposure  to 
electromagnetic fields, health effect of UV from sunbeds used for cosmetic purposes, the definition of 
synthetic  biology  and  risk  assessment  methodology,  additives  in  tobacco  products.  A  report  on  the 
activities of the Scientific Committees is publicly available43 as well as the opinions issued44.   

Objective 4: Facilitate access to better and safer healthcare for Union citizens 

European Reference Networks on Rare Diseases 

The  programme  finances  the  coordination  activities  of  23  of  the  networks  applying  for  multiannual 
grants.  These  are  virtual  networks  involving healthcare  providers across  Europe.  They  aim  to  tackle 
complex  or  rare  medical  diseases  or  conditions  that  require  highly  specialised  treatment  and  a 
concentration  of  knowledge  and  resources.  To  review  a  patient’s  diagnosis  and  treatment,  ERN 
coordinators will convene a “virtual” advisory board of medical specialists across different disciplines, 
using  a  dedicated  IT  platform  and  telemedicine  tools.  This  way,  it  is  the  medical  knowledge  and 

40   Commission Decision 2012/C 198/06. 
41   http://ec.europa.eu/health/expert_panel/opinions_en?page=1 
42   End  of  April  2016,  SCHER  and  SCENIHR  merged  to  become  the  Scientific  Committee  on  Health, 

Environmental and Emerging Risks (SCHEER) 

43  See at http://ec.europa.eu/health/sites/health/files/scientific_committees/docs/activity_report_sc_20132016_en.pdf 
44   http://ec.europa.eu/health/scientific_committees/policy_en 

40 

 
 
 
                                                           
expertise that travel rather than the patients, who have the comfort of staying in their supportive home 
environments.  

The first ERNs are up and running in 2017. Invitation for new healthcare providers to join the existing 
Networks, as well as calls for new ERNs will follow. The new target is to establish and finance the 
coordination  of  up  to  33  European  Reference  Networks  till  2020.    Over  the  next  five  years,  as  the 
ERNs  reach  full  capacity,  thousands  of  EU  patients  suffering  from  a  rare  or  complex  condition  can 
expect  to  benefit.  This  investment  will  need  to  be  sustained  over  time.  For  more  information  please 
refer to https://ec.europa.eu/health/sites/health/files/ern/docs/2017_brochure_en.pdf 

Rare diseases 

The  Joint  Research  Centre  is  developing  and  maintaining  the  European  Platform  on  Rare  Diseases 
Registration  receiving  specific  financial  support  from  the  Health  Programme.  The  Platform  is 
promoting  the  interoperability  of  existing  registries  and  will  help  in  the  creation  of  new  ones, 
including those developed by the European Reference Networks. The migration of the two databases - 
the  European  Surveillance  of  Congenital  Anomalies  (EUROCAT)  and  the  Surveillance  of  Cerebral 
Palsy in Europe (SCPE) – has been successfully implemented. 

Joint  Action  on  rare  diseases  (RD-ACTION)  is  supporting  Member  States  in  the  development  and 
implementation of actions in the area of rare diseases. Three main goals of the RD-Action are to:   

1) support the further development and sustainability of the Orphanet database; 

2)  contribute  to  solutions  to  ensure  an  appropriate  codification  of  RD  in  health  information 
systems; and  

3) continue implementation of the priorities identified in Council Recommendation on patient 
safety including the prevention and control of healthcare associated infections (2009/C151/02) 
and the Commission Communication on rare diseases (COM(2008)679).  

In the area of rare diseases codification general rules for routine coding with Orphacodes have been 
established and guidelines are provided to achieve internationally standardised data collection. 

For the current and previous work in the area of rare diseases (EU collaboration to help patients with 
rare disease) DG SANTE won the first EU Ombudsman Award for Good Administration45.  

EU health legislation and health information  

The  programme  also  provides  the  resources  for  implementing  the  EU’s  political  commitments  and 
legal  obligations  in  health  (e.g.  implementation  of  the  tobacco  or  health  threats  legislation,  the  EU 
regulatory framework for medicinal products and medical devices, for substance of human origin, and 
cross-border  health  care).  It  also  supports  the  Member  States  implementation  of  this  legislation 
through the development of common tools, such as networks, IT platforms, guidance and sharing of 
best  practices.  The  development  and  maintenance  of  these  tools  (e.g.  EUDAMED  database,  Euripid 
database) is essential in order to ensure the smooth operation of the Internal Market in these sectors. It 
is  estimated  that  in  the  years  2014-2016,  30%  of  the  Programme’s  overall  annual  budget  was 
dedicated for the above mentioned purposes. 

The  programme  supports  work  on  health  information:  EUR  13.2  million  spent  in 2014-2016  for  the 
collection and analysis of health information, data and indicators in cooperation with the Organisation 
for Economic Cooperation and Development (OECD), the World Health Organisation (WHO) and the 

45   https://www.ombudsman.europa.eu/en/activities/eventdocument.faces/en/77458/html.bookmark 

41 

 
 
                                                           
Observatory  on  Health  Systems  and  Policies,  as  well  as  ESTAT  and  Joint  Research  Centre  (JRC), 
contributing to the development of country specific and cross-country knowledge to inform policies at 
national and EU level, through such actions as the State of Health in the EU cycle46. 

The  BRIDGE  health  project  has  been  coordinating,  improving,  and  advancing  some  of  the  most 
influential  EU  health  indicator  development  networks  since  its  2015  inception,  creating  synergies 
between the efforts of several earlier projects on health information. These expert networks have been 
developing and maintaining indicators in the domains of population health and health systems, health 
examination surveys, and population injury, and developed methods to produce health indicators more 
cost-effectively  in  various  areas  using  disease  registries  and  administrative  health  data  collection 
systems.  Various  options  to  improve  the  sustainability  of  producing  indicators  to  underpin  health 
policy  and  research  in  the  EU  have  been  developed,  and  a  blueprint  for  a  more  sustainable 
organisation of these activities is being finalised.  

46  

Health at a Glance: Europe 2016 report, and country health profiles 

42 

 
 
                                                           
 
ANNEX II: METHODOLOGICAL ANNEX 

Table 1: Evaluation questions matrix 

Evaluation 
criteria 
Relevance  

Evaluation questions 

Judgement Criteria 

Indicators 

Sources of evidence  

1.  To  what  extent  are 
objectives 
valid 
still 
accordance  with  health  needs 
Europe? 

the  3HP 
in 
and 
in 

2.  To  what  extent  are 
the  3HP 
thematic  priorities  sufficient  and 
sufficiently  covered  to  achieve  the 
3HP  objectives  and  Commission 
wider priorities? 

- 

- 

- 

- 

- 

- 

The extent to which the 3HP 
objectives address the current 
needs of health stakeholders 
across the EU 
The extent to which new and 
emerging health needs at the EU 
and MS level can be 
accommodated through the 
existing HP objectives 

The extent to which the thematic 
spread is narrow / wide enough 
to cover the 3HP objectives 
The extent to which the number 
of thematic priorities is small / 
large enough to allow focused 
actions 
The extent to which individual 
priorities are balanced across, 
and in line with, the different 
needs of  stakeholder groups 
Taken together, the extent to 
which the thematic priorities 
support Commission wider 
priorities  

3.  To  what  extent  are  the  actions 
the  Annual  Work 
prioritised 

in 

- 

To what extent do the individual 
actions prioritised in the AWP 

The 
level  of  agreement  among 
stakeholder groups that the design of 
the HP and its objectives are: 
- 

aligned with identified health 
needs  
aligned with the health needs of 
policy-makers, stakeholders and 
/ or citizens in their MS 
flexible enough to address 
emerging / new health needs  

- 

- 

Instances  where  new  and 
/  or 
emerging health needs are identified, 
these  were 
in  which 
and  ways 
incorporated within the programme.  
The 
stakeholders 
priorities: 
- 

level  of  agreement  among 
thematic 

that 

the 

are sufficient in number to cover 
the objectives in a focused and 
practical way  
are clearly defined and narrow / 
broad enough in scope to allow 
focused action  
are aligned with objectives 
are important and individually 
relevant to stakeholders’ health 
needs  
fit with the Commission’s wider 
priorities 

- 

- 
- 

- 

The  extent  to  which  the  priorities  are 
based  on  a  clear  and  coherent 
intervention logic 
The 
stakeholders 

level  of  agreement  among 
actions 

that 

the 

Programme assessment  
Documentation  on  public  health 
problems and needs (Annex B – table 
1 & table 4) 
Survey with NFP and PC (Annex C) 
Interviews  with  external  experts  (see 
Table 8 main report) 
Focus Group (Annex E) 
Case studies  
Especially “Policy context” 
See  findings  from  question  13  (on 
EU added value) 

Programme assessment  
Documentation  on  public  health 
problems and needs (Annex B – table 
1) 
Interviews  with  DG  SANTE  (see 
Table 8 main report) 
Focus Group (Annex E) 
Case studies 
Especially 
(“Strategy and design”)  

“Theory  and  Practice” 

Programme assessment  
on 
Documentation 

HP 

43 

 
 
 
 
 
 
Evaluation 
criteria 

Evaluation questions 

Judgement Criteria 

Indicators 

Sources of evidence  

Programmes  (AWPs)  relevant  vis-à-
vis the 3HP thematic priorities? 

support progress under each 
thematic priority?  

supported: 
- 

are sufficient in number to cover 
the thematic priorities in a 
focused and practical way  
are clearly defined and relevant / 
targeted to each thematic priority 

- 

4. To what extent are the actions co-
funded through the AWPs relevant to 
achieving  the  objectives  set  out  in 
Article 168 TFEU? 

Effectiveness  5.  To  what  extent  is  the  process  for 
defining  and  prioritising  actions 
through Annual Work Programmes 
(AWP) 
impartial  and 
equitable? 

transparent, 

Considered to cover the coherence of the HP (treated later under question 15) 

The extent to which the actions in the 
priorities’ 
support 
AWP 
intervention logic 

the 

- 

To  what  extent  can  the  process  for 
defining  and  prioritising  actions 
through the AWP be considered: 
Transparent (i.e. open to 
- 
scrutiny)? 
Impartial (i.e. without political 
bias towards any one Member 
State)? 
Equitable (i.e. fair participation of 
and consideration for needs of all 
concerned actors)? 

- 

The 
level  of  agreement  amongst 
stakeholders  on  whether  the  process 
for  defining  and  prioritising  actions 
through the AWP is: 
Transparent:  
- 

Existence of a full documentation 
trail for this decision-making 
process (i.e. meeting minutes 
available to EP) 
There is public access to the 
AWP and its supporting 
documentation 

- 

Impartial:  
- 

The process for defining AWP 
allows for and demonstrates the 
incorporation of proven EU 
health needs (e.g. through the 
consultation of experts / use of 
existing evidence of health 
needs) 

Equitable: 
- 

(in)existence of participation 
mechanisms for all concerned 
actors (MS, CHAFEA and other 
DGs)  
Stakeholder awareness of these 
participation mechanisms  

- 

implementation (Annex B – table 2) 
Interviews with CHAFEA (see Table 8 
main report) 
Case studies 
Especially 
(“Delivery”) 

“Theory  and  Practice” 

Programme assessment  
implementation 
Documentation  HP 
documentation  and  past 
reports, 
evaluations  and  audits  (Annex  B  – 
table 2 and 3) 
Survey questions for PC (Annex C) 
Interviews with SANTE and CHAFEA 
(see Table 8 main report) 

44 

 
 
 
 
 
Evaluation 
criteria 

Evaluation questions 

Judgement Criteria 

Indicators 

Sources of evidence  

- 

- 

Evidence that these participation 
mechanisms are perceived by 
stakeholders to be fair and 
inclusive. 
Instances where stakeholders 
have participated in the AWP 
process and ways in which their 
suggestions and needs were 
incorporated into the 
programme.  

6.  How  effective  was 
Annual  Planning  (MAP) 
preparation of the AWP? 

the  Multi-
the 
for 

- 

To what extent has the MAP 
process led to an enhanced 
strategic approach across years, 
resulting in more clarity in 
resources available over time 
and budget allocation (by 
objective and thematic priority)? 

7. How effective is the introduction of 
the  "exceptional  utility"  criteria 
(para 19, Regulation of 3HP) in order 
to  incentivise  the  participation  of  low 
GNI countries? 

To what extent has the introduction of 
"exceptional 
criteria 
incentivised  the  participation  of  low 
GNI countries (i.e. lower than 90% of 
EU average)? 

utility" 

8.  In  practice,  to  what  extent  are  the 
actions  in  the  AWP  contributing  to 
the:  
- 

3HP objectives and thematic 
priorities 

-  Commission policy 

priorities? 

The  extent 
the  actions 
to  which 
implemented  to  date  have  (or  are 
expected) to contribute to:  
3HP objectives and 
- 
corresponding thematic priorities 

-  Commission policy priorities 

level  of  agreement  amongst 

The 
stakeholders that: 
- 

the setting of the priorities on a 
multi-annual basis is more 
strategic than it previously was 
(there is more long-term thinking 
evidenced in the MAP) 
there is more clarity regarding 
the availability of resources and 
budget allocation over time 
the adopted AWP provide 
evidence that the priorities set in 
the MAP have been followed (or 
it can be explained why this is 
not the case) 

level  of  agreement  amongst 

The 
stakeholders that: 
- 

The "exceptional utility" criteria 
address the needs (i.e. the 
barriers to participation) of the 
low GNI countries  
There is (increased) participation 
of low GNI MS in the 3HP 

- 

- 

- 

level  of  agreement  amongst 

The 
stakeholders confirming that: 
- 

The actions operationalised to 
date are clearly defined, aligned 
with and directly address the 
individual 3HP objectives and 
thematic priorities  
The actions implemented to date 

- 

Programme assessment  
implementation 
Documentation  HP 
documentation  and  past 
reports, 
evaluations  and  audits  (Annex  B  – 
table 2 and 3) 
Interviews  with  SANTE  (see  Table  8 
main report) 

Programme assessment  
implementation 
Documentation  HP 
documentation  and  past 
reports, 
evaluations  and  audits  (Annex  B  – 
table 2 and 3) 
Survey (Annex C) 
Interviews with SANTE and CHAFEA 
(see Table 8 main report) 
See also question 2 and 3 (evidence 
of relevance of thematic priorities and 
actions) 
Case studies  
Especially 
Practice) 

(Theory  and 

“Benefits” 

45 

 
 
 
 
 
 
Evaluation 
criteria 

Evaluation questions 

Judgement Criteria 

Indicators 

Sources of evidence  

have concrete links with the 
Commission policy priorities 
which can be evidenced  

Evidence  and  examples  confirming 
that  progress  has  been  made  and 
that  the  actions  constitute  action  that 
would  not  have  been  achieved 
without the HP (EU added value) 

Evidence and examples showing that 
steps 
to 
disseminate results. 

taken 

been 

have 

Efficiency 

9. To what extent does the design of 
to  an  efficient 
the  3HP 
allocation  of 
resources  among 
objectives / thematic priorities? 

lead 

10.  To  what  extent  does 
the 
allocation  of  resources  allow  for  an 
efficient implementation of the 3HP 
in terms of: 
- 
- 
- 

funding mechanisms  
simplification measures 
operational costs? 

- 

- 

- 

- 

- 

To what extent is the distribution 
of funding across the thematic 
priorities aligned with the EU 
health needs which the HP aims 
to address? 
To what extent is funding at the 
level deemed critical to have an 
impact? 

To what extent are the 
implementation costs 
proportional to the expected 
results? 
The extent to which the 
simplification measures allow for 
the most cost-effective 
implementation of the 3HP 
To what extent do the current 
funding mechanisms lead to an 
allocation resources that allows 
for the most cost-effective way of 
implementing the 3HP? 

level  of  agreement  amongst 
a 

there 

is 

The 
stakeholders 
that 
correlation  between: 
- 

The identified relevance of 
thematic priorities and allocation 
of resources; and  
The allocation of resources are 
proportional to the expected 
results 

- 

level  of  agreement  amongst 

The 
stakeholders that: 
- 

(management costs of) funding 
mechanisms are reasonable 
given the objective and expected 
result 
the funding mechanisms allow 
for an allocation of resources 
that achieves the most cost-
effective implementation of the 
3HP  
simplification measures have 
reduced the administrative costs 
for applicants and for CHAFEA  
the operational costs  associated 
with the 3HP (design and 
implementation) are at the 
minimum level deemed 
necessary to achieve expected 

- 

- 

- 

See  findings  from  assessment  of 
Relevance (questions 1-4) 
Programme assessment  
Documentation  HP 
implementation 
documentation  and  data  analysis 
(Annex B – table 2) 
Interviews with SANTE and CHAFEA 
(see Table 8 main report) 

Programme assessment  
implementation 
Documentation  HP 
documentation  and  data  analysis 
(Annex B – table 2) 
Interviews with CHAFEA (see Table 8 
main report) 
Case studies  
Cross-cutting  analysis  of 
learned from Delivery  

lessons 

46 

 
 
 
 
Evaluation 
criteria 

Evaluation questions 

Judgement Criteria 

Indicators 

Sources of evidence  

results   

11.  How  may  the  efficiency  of  the 
3HP be improved regarding: 
number of priorities  
- 
funding mechanisms 
- 
application / implementation 
- 
procedures 
available resources? 

- 

To  what  extent  is  there  scope  to 
improve  the  efficiency  of  the  3HP  in 
terms of:  
- 

increasing / decreasing the 
number of priorities   

-  making the funding mechanisms 

- 
- 

more cost-effective  
streamlining procedures 
achieving the highest impact 
possible with the available 
resources (financial and human) 

12.  To  what  extent  are 
the 
monitoring processes and resources 
(at  the  Commission  and  MS  level) 
sufficient  to  plan  and  promote  the 
results of the 3HP? 

At  both  the  Commission  and  MS 
level: 
- 

To what extent do the current 
monitoring processes allow for 
the efficient management of 
actions supported?  
To what extent does the 
monitoring framework in place 
consist of: indicators at the 
action level, targets at the level 
of thematic priority and 
objectives and measurement of 
results against targets? 
To what extent are these 
indicators and targets relevant, 
clearly defined and aligned with 

- 

- 

- 

- 

- 

- 

- 
- 

- 

- 

The 
level  of  agreement  amongst 
stakeholders that efficiency gains can 
be realised vis-à-vis:   
- 

The number of priorities: 
evidence that there should be 
more / fewer priorities so that 
resources are not too 
concentrated / stretched  
Funding mechanisms: 
evidence that the current 
balance of mechanisms can be 
adjusted to allow for a more 
optimal and cost-effective 
allocation of resources 
Established procedures, 
including additional simplification 
measures – evidence that the 
current procedures need to be 
further fine-tuned / simplified.  
The availability of resources 
(financial and human) – evidence 
showing a need for more / fewer 
resources to be able to deliver 
the expected results efficiently 
Evidence that the resources, 
processes and systems allocated 
to the monitoring of results are 
considered to be reasonable and 
sufficient (at both the EU and MS 
level). 
Indicators have been set 
Indicators are SMART and there 
are processes in place to 
measure them 
Evidence of there being a 
dissemination strategy in place 
and examples to show that this is 
being implemented.  
The level of agreement amongst 
stakeholders that  the 

See question 9 and 10 (findings and 
sources). 

Programme assessment  
Documentation  on  HP high level and 
implementation  (Annex  B  –  table  1 
and 2) 
Survey questions for NFP (Annex C) 
Interviews with SANTE and CHAFEA 
(see Table 8 main report) 
Case studies  
Especially 
Practice) 

(Theory  and 

“Benefits” 

47 

 
 
Evaluation 
criteria 

Evaluation questions 

Judgement Criteria 

Indicators 

Sources of evidence  

individual thematic priorities and 
3HP objectives?  
To what extent is there an 
effective dissemination strategy 
in place for the promotion of 3HP 
results? 
To what extent are there 
adequate resources allocated to 
support the monitoring and 
promotion of 3HP results? 
To what extent are these 
monitoring processes equally 
effective across each MS?  
In principle, to what extent are 
the 8 EU added value criteria 
well defined and evidenced in 
the proposals? 
To what extent is each criterion 
relevant? 

- 

- 

- 

- 

- 

added 

EU 
value 

13.  To  what  extent  are  the  eight  EU 
added  value  criteria  addressed  in 
proposals?   

Coherence  / 
consistency 

14. To what extent have the thematic 
priorities  of  the  3HP  led  to  more 
coherence  and  synergies  between 
the  funded  actions  in  delivering  on 
the objectives? 

Internal coherence 
To  what  extent  are  the  thematic 
priorities: 
- 
- 
- 

not contradictory  
complementary 
not unnecessarily overlapping? 

15. To what extent are the objectives 
and  thematic  priorities  of  the  3HP 
externally  consistent/coherent,  i.e. 
correspondence between: 
- 

the 3HP and health objectives of 

External coherence  
To what extent are the objectives and 
thematic priorities: 
- 
- 

not contradictory  
complementary 

dissemination strategy is working  
- or is expected to work - in 
practice (i.e. internal and 
external stakeholders make - or 
are expected to make -  use of 
results) 

level  of  agreement  amongst 
The 
stakeholders confirming that the 8 EU 
added value criteria are: 
- 

Sufficient in number and 
necessary  
SMART 
- 
-  Relevant 
- 

Equally addressed in proposals  

level  of  agreement  amongst 

The 
stakeholders confirming: 
- 

The extent to which there are 
synergies between the funded 
actions  
the types of synergies at the 
level of the thematic priority 

- 

level  of  agreement  amongst 

The 
stakeholders confirming the: 
extent to which there are 
- 
synergies 
the types of synergies  

- 

HP 

“Strategy  and  Design” 

Programme assessment  
Documentation 
on 
implementation (Annex B – table 2) 
Interviews  with  external  experts 
(Table 8) 
Focus  Groups  discussion  analysis 
(Annex E) 
Case studies  
Especially 
(Theory and Practice) 
Programme assessment  
Documentation  on  public  health 
problems and needs (Annex B – table 
1 & table 4) 
Interviews  with  SANTE  (see  Table  8 
main report) 
Analysis of Focus Group  discussions 
(Annex E) 
Case studies  
Especially 
Practice) 
And, 
objective 
Programme assessment  
Documentation  on  high-level  HP 
documentation  and  public  health 
problems and needs (Annex B – table 
1 & table 4) 

(Theory  and 

cross-cutting 

“Benefits” 

analysis 

by 

48 

 
 
Evaluation 
criteria 

Utility 

Evaluation questions 

Judgement Criteria 

Indicators 

Sources of evidence  

- 

Article 168 TFEU  
the 3HP objectives and those of 
other public interventions (e.g. 
national health policies, EU 
policies and Programmes, other 
international actions)? 

16. To what extent is the 3HP overall 
useful  and,  if  necessary,  how  could 
its overall utility be increased?  
What  are  the  specific  needs  of 
Member  States  to  which  the  3HP 
could provide a concrete solution but 
has not done so yet? 

…  with  health  objectives  in  Article 
168  TFEU 
public 
and 
interventions? 

other 

- 

- 

- 

To what extent does the 3HP 
create added value in meeting 
EU public health needs?  
To what extent does the 3HP 
meet stakeholders’ health needs 
in a way that other public health 
interventions do not? 
To what extent is there a need to 
expand the 3HP in meeting more 
/ other health needs in MS? 

between  different 
interventions (EU and non-EU) 

types  of  public 

The  extent  to  which  each  type  of 
intervention  contributes  a 
public 
unique  added  value 
that  other 
interventions do not offer 
- 

Instances where the 3HP could 
meet certain health needs that it 
has not already met 

Interviews  with  SANTE  and  external 
experts  and  other  EU  funds  (see 
Table 8 main report) 
Analysis of Focus Group  discussions 
(Annex E) 

Summary of questions 1 - 15 

level  of  agreement  amongst 

The 
stakeholders that: 
- 

The 3HP has met health needs 
which other public interventions 
have not been able to do  
The 3HP has justified costs and 
proven its value for money  
The 3HP can be made more 
efficient whilst still meeting 
intended results  
There are no specific needs of 
stakeholders which the 3HP 
should in principle address, but 
does not 

- 

- 

- 

This is considered to be covered under effectiveness (see question 8) 

17.  Regarding  the  objective  n°  3  for 
increasing  the  public  health  capacity 
building, how has this been achieved 
a)  in  terms  of  generic  public  health 
capacity  building  and  b)  in  terms  of 
capacity 
specific 
that 
supports 
implementation  of  policy 
priorities  such  as  HTA,  e-health  and 
the Innovation Platform? 

building 

49 

 
 
 
 
 
Table 2: Case studies 

The methodology for the case studies consisted of eight mini theory-based evaluations (including the reconstruction of their 
own intervention logic), with a review of relevant documentation for a maximum of five individual funded actions per case 
study under the selected thematic priorities, followed by interviews and consultation with the public health experts. 

The contractors described  the policy context  for each  of  the  eight  selected  thematic  priorities explaining  how the thematic 
priority relates to EU health needs and then makes the case for EU action. Then, they have presented the intervention logic 
for  the  thematic  priority,  and  discussed  in  depth  its  main  parts  in  terms  of  both  theory  and  practice.  The  aim  was  to  get 
insights  that  are  useful  for  answering  higher-level  questions  relating  in  particular  to  relevance,  effectiveness  and  lessons 
learned, all these feeding the overall evaluation findings and conclusions. 

50 

Thematic priority ActionsYear of callFunding mechanismBudget (EUR)[1]1.    Monitoringofthenationalpoliciesrelatedtoalcohol consumption and harm reduction Direct grant agreement 500,0002.     Joint Action on Nutrition and Physical ActivityJoint action1,200,0003.    ObesityTrainingAndInformationServicesforEuropeOperating grant 162,6194.    Smokingpreventioninaction:theSmokeFreePartnership CoalitionOperating grant 352,0545.    EPHA2015:Protectingandimprovingpublichealth and well-being in all policiesOperating grant 487,4411.     Joint Action on Dementia 2015-2018Joint action1,498,7102.     Mental Health - Trimbos InstituutService contract 799,7773.    ParticipationtoHealthyWorkplacesAndInclusiveStrategiesintheWorkSector(PATHWAYS)Project 969,3794.     Alzheimer Europe (2015-2017)Operating grant422,8805.    CancerLeaguesCollaboratinginCancerPreventionandControlattheEUandNationalLevelOperating grant314,9721.    Preparednessactivitiesrelevanttothemonitoring,theassessmentandthecoordinationof the response 643,5592.    Postexerciseonseriouscrossborderthreatsto health 458,9893.    Preparednessandresponseactivitiesinthecontext of the Ebola epidemic in West Africa 499,7194.    Studyoncost-benefitanalysisofreferencelaboratories for human pathogens 199,9425.StudyonthePublicHealthlawnetworksupportingtheimplementationofDecision1082/2013/EU2015303,4902.3.ImplementationofUnionlegislationoncommunicablediseasesandotherhealththreats,includingthosecausedbybiologicalandchemicalincidents,environmentandclimate change 1.    Efficientresponsetohighlydangerousandemerging pathogens at EU level2014Joint action3,499,873242,55016,1701.     European PharmacopoeiaDirect grant agreement 1,100,0002.    StatisticaldataandGuidanceDocumentformedicinal product pricing and for the use of ERPProject 300,0003.     Study on the regulation of advanced therapies161,5004.    Studyonofflabeluseofmedicinalproductsinthe European Union226,5005.     Market surveillance of medical devices2015Joint Action849,4881.    PromotingImplementationofRecommendations on Policy, Information and Data for Rare DiseasesJoint action4,379,9792.Developmentofamanualandtoolboxfortheassessment of European reference networks499,2543.StudyonservicestobeprovidedbyEuropeanreference networks172,6601.    VigilanceandInspectionfortheSafetyofTransfusion,AssistedReproductionandTransplantation2014Joint action 2,328,6642.GoodPracticesfordemonstratingsafetyandquality through recipient follow-up1,032,0303.EuropeanCorneaandCellTransplantationRegistry424,5674.   Organ donationService contract199,030[1] Rounded to the nearest EURService contracts20141.1.Riskfactorssuchasuseoftobaccoandpassivesmoking,harmfuluseofalcohol,unhealthydietaryhabitsand physical inactivity1.4.Chronicdiseasesincludingcancer,age-relateddiseases and neurodegenerative diseases 20142.2.Capacity-buildingagainsthealththreatsinMemberStates,including,whereappropriate,cooperationwithneighbouring countries 20143.4.SettingupamechanismforpoolingexpertiseatUnionlevel 1.    Expertpaneloneffectivewaysofinvestinginhealth[2]2014 2 Service contracts3.6.ImplementationofUnionlegislationinthefieldofmedicaldevices,medicinalproductsandcross-borderhealthcare 2014Service contract4.1 European Reference Networks 2014Service contracts4.5.ImplementationofUnionlegislationinthefieldofmedicaldevices,medicinalproductsandcross-borderhealthcare 2015Project 
 
 
ANNEX III:    THE EXCEPTIONAL UTILITY CRITERIA FOR BOOSTING LOW GNI 

COUNTRIES' PARTICIPATION 

Through the use of exceptional utility criteria the Health Programme provides financial incentives to involve as 
much  as  possible  all  Member  States  and  participating  countries,  in  particular  for  participation  in  Joint  Action 
where it is question for more close cooperation with health competent authorities on a series of health issues for 
political endorsement and direct implementation of results. The criteria for exceptional utility in Joint actions are 
enshrined in the Programme Regulation (EU) N° 282/2014 Article 7 point 3. Criteria for exceptional utility in 
Projects  and  operating  grants  have  been  defined  on  a  similar  basis  and  been  introduced  in  the  Annual  Work 
Programmes.  

Table 1: Key features of the “exceptional utility” criterion 

Mechanism  
Joint Actions  

Criteria for exceptional utility under 3HP (2014 – 2016)  
1. At least 30% of the budget of the proposed action is allocated to MS whose gross national 
income (GNI) per inhabitant is less than 90% of the Union average. This criterion intends to 
promote the participation from Member States with low GNI.  

2. Bodies from at least 14 participating countries participate in the action, out of which at least 
four are countries whose gross national income (GNI) is less than 90 % of the Union average. 
The  criterion  promotes  wide  geographical  coverage  and  the  participation  of  Member  States 
authorities from countries with a low GNI.  

Projects  

1. At least 60 % of total budget must be used to fund staff. This criterion intends to promote 
capacity building for development and implementation of effective health policies . 

2.  At  least  30  %  of  the  budget  of  the  proposed  action  is  allocated  to  Member  States  whose 
gross  national  income  (GNI)  per  inhabitant  is  less  than  90  %  of  the  Union  average.  This 
criterion  intends  to  promote  the  participation  of  health  actors  from  Member  States  with  low 
GNI.  

3. The proposal must demonstrate excellence in furthering public health in Europe and a very 
high EU-added value.  

Operating grants  

1. At least 25 % of the members or candidate members of the non-governmental bodies come 
from Member  States whose gross national income (GNI) per inhabitant is less than 90 % of 
the Union average … to promote the participation of non-governmental bodies from Member 
States with a low GNI.  

2.  The  reduction  of  health  inequalities  at  EU,  national  or  regional  level  is  manifested  in  the 
mission  as  well  as  the  AWP  of  the  applicant…  to  ensure  that  co-funded  non-governmental 
bodies directly contribute to one of the main objectives of the third Health Programme, i.e. to 
reduce health inequalities  

51 

 
 
 
 
 
 
 
ANNEX IV: INTERVENTION LOGIC 

Figure 1: Intervention logic for thematic priority 1.4 (Chronic diseases) 

52 

 
 
 
 
 
 
Figure 2: Intervention logic for thematic priority 2.2 (Capacity building) 

53 

 
 
 
 
 
 
Figure 3: Intervention logic for thematic priority 4.1 (European Reference Networks) 

54 

 
 
 
ANNEX V 

SYNOPSIS REPORT ON STAKEHOLDERS' CONSULTATION ACTIVITIES 

INTRODUCTION 

The present report presents the outcome of the consultation activities with stakeholders conducted for 
the mid-term evaluation of Regulation (EU) No 852/2014 on the establishment of the third Programme 
for  the  Union's  action  in  the  field  of  health  (hereinafter:  the  Regulation).  The  consultation  covered 
aspects  relating  to  the  evaluation  criteria  used  in  this  context,  namely  relevance,  added  value, 
efficiency, effectiveness, and coherence. It addressed institutional stakeholders notably the Programme 
Committee  members  and  National  Focal  Points  as  well  as  the  direct  beneficiaries  of  the  grants 
awarded under the Regulation, and stakeholders which are indirectly involved in the funded activities, 
especially  non-governmental  organisations,  public  health  authorities,  academic  and  research 
organisations,  international  organisations,  professional  association,  private  companies  and  individual 
persons. 
Stakeholders  had  the  opportunity  to  provide  their  feedback  on  a  Commission  evaluation  roadmap47 
before the elaboration of the Terms of Reference for the mid-term evaluation on Regulation (EU) No 
852/2014, during a 4-week period starting on 15 December 2015.  

During  the  evaluation  and  as  part  of  that  exercise,  targeted  on-line  consultations  with  Public  Health 
Experts  and  e-surveys  with  National  Focal  Points  and  Programme  Committee  members  were 
conducted,  and  complemented  by  targeted  interviews  of  Commission  and  International  Health 
Organisation  officers,  and  direct  beneficiaries  mainly  project  leaders/coordinators  of  actions  funded 
under the Programme. 

In addition, an open public consultation (OPC)48 of all interested parties has been conducted using the 
European Commission ‘Public consultations’ website and the DG SANTE ‘Health Programme’ web 
page.  The  link  to  the  OPC  was  disseminated  via  the  CHAFEA  and  SANTE  websites,  shared  on  the 
Health  Policy  Platform  and  distributed  via  stakeholder  mailing  lists  and  National  Focal  Points.  The 
participants were invited to complete the online questionnaire available only in English. However, the 
participants were able to submit contributions in any official EU language. No use of this possibility 
has been made by them. 

The open public consultation was carried out between 23 November 2016 and 23 February 2017. 

1. FEEDBACK ON THE EVALUATION ROADMAP 

Two  NGOs  provided  feedback  to  the  Evaluation  Roadmap:  the  first  through  the  Secretariat  General 
platform on 14  January 2016   and the second one 21 October 2016 (with 10 months delay) when the 
evaluation was entering in its final stage.  

Hungary  provided  also  feedback  through  the  National  Focal  Point  and  the  Programme  Committee 
Member with direct messages to DG SANTE.  

All three relevant feedbacks have been taken into consideration in the evaluation :  

  One  of  the  case  studies  includes  the  thematic  priority  covering  the  alcohol  policy  in  the  EU 
and how the Health Programme contributes to reduce the risk factors such as harmful use of 
alcohol 

47  http://ec.europa.eu/smart-regulation/roadmaps/docs/2015_sante_680_evaluation_mid-

term_health_programme_en.pdf 

48  https://ec.europa.eu/health/programme/consultations/midterm_evaluation_en 

55 

 
 
 
 
                                                           
 
  The  analysis  of  data  provides  clear  information  on  how  the  budget  is  spend:  per  thematic 

priority, per country, per financial mechanism 

  The exceptional utility criteria have been examined and conclusions have been drawn on their 
effectivenes  and  the  need  of  incentives  for  low  GNI  MS  to  participate  in  the  Health 
Programme. 

2. TARGETED CONSULTATIONS 

Targeted  consultation  have  been  used  to  acquire  specific  information  from  particular  stakeholders' 
groups.  The  stakeholders  were  identified  by  the  stakeholder  mapping  exercise  during  the  ex-post 
evaluation of the second Health Programme49 and the strategy of consultation in has been based on the 
folowing stakeholders’map.  

Figure 1 : Stakeholders’ map 

2.1 ON-LINE FOCUS GROUPS 

Three on-line focus groups with relevant public health experts were organsined on the Health Policy 
Platform  using,  one  each  for  specific  objectives  1  (Health  promotion),  3  (Innovative,  efficient  and 
sustainable health systems) and 4 (Access to better and safer healthcare) of the Programme50. 

Overall 15 experts have participated representing mainly academics, politicians and representatives of 
international  organizations  with  comprehensive  knowledge  on  Health  Promotion,  the  Health 
Programme and the current developments. The experts were from Austria, Belgium, Estonia, Finland, 
Germany,  Hungary,  Italy,  Malta,  Norway,  Poland  Portugal,  Slovenia,  Spain,  Sweden,  United 
Kingdom, and as well as experts from EU Office of WHO. 

49   https://ec.europa.eu/health/sites/health/files/programme/docs/ex-post_ev-hp-2008-13_final-report.pdf, p. 45 

50   Given  the  political  sensitivity  of  specific objective  2  (Cross-border  health  threats),  the  originally  foreseen 
focus group was replaced with interviews with members of the Health Security Committee, which are still 
being carried out.  

56 

 
 
 
 
 
                                                           
The purpose was to gain insight into such issues as the relevance of the Programme and its specific 
objectives and thematic priorities in relation to health needs, EU added value and synergies with other 
actions at EU and other levels.  

Each group was moderated by one of the evaluation team’s sector experts, based on a discussion paper 
prepared  and  circulated  in  advance.  The  groups  were  carried  out  on  17-18  August  2016  and  each 
lasted from 1.5 to 2 hours.  

CONCLUSIONS  

FROM THE ON-LINE GROUP ON HEALTH PROMOTION OBJECTIVE 

The  experts  rated  the  objective  as  “very  relevant”,  but  pointed  out  that  the  “specification  via  the 
thematic  priorities  could  be  improved,  made  more  logical,  and  complemented  by  a  more  social 
determinant-based approach”. It was suggested to focus on major determinants and large and growing 
burdens of diseases. The Health Programme should encourage more integrated approaches and put the 
emphasis on scaling up and dissemination across MS. 

Capacity building efforts should focus on mapping existing capacities and addressing the weaknesses. 
With  regard  to  policy-making,  the  experts  considered  the  EU  was  adding  value  in  the  definition  of 
economic,  social  and  health  policies  and  the  formulation  of  country–specific  recommendations  (e.g. 
through the European Semester, the EU's annual cycle of economic policy guidance and surveillance). 
The experts concluded to put more emphasis on evidence of economic benefits of any action as they 
felt that the lack of resources is the biggest threat to health prevention and care.  

A stronger leadership of the Health Programme and more sustainability were identified as necessities 
for an enhanced EU added value. 

FROM  THE  ON-LINE  GROUP  ON  INNOVATIVE,  EFFICIENT  AND  SUSTAINABLE  HEALTH 
SYSTEMS’ OBJECTIVE 

The  experts  unanimously  agreed  that  “Contributing  to  innovative,  efficient  and  sustainable  health 
systems” is a laudable goal which relies upon a strong infrastructure through a capable, well trained 
and  flexible  workforce  and  should  be  absolutely  included  as  the  objective  in  the  Programme  of  the 
EU.  However,  they  propose  that  the  objective  should  include,  amongst  others,  healthcare  staff 
governance and dissemination of innovation, health self-management and sharing of best practices.  

The  EU  can  contribute  by  informing  policy-makers  on  good  practices  that  improve  efficiency  and 
sustainability and by helping countries to adapt and implement such practices. Money remains a good 
incentive to convince decision-makers to make the right choices and the EU should make the best of 
its financial capacity to influence policy-making in the right direction.  

The  EU  has  the  responsibility  of  supporting  actions  that  benefit  all  its  Member  States  in  delivering 
highly complex services or highly specialized training and research on the treatment of autoimmune, 
infectious  and  rare  diseases,  health  literacy  and  mental  health  including  mental  health  of  youth  and 
health workforce.  

Cost reduction strategy in health services cannot be implemented without considerations for its health 
outcomes as it could have a significant negative impact on the care provided. Since effective, satisfied 
and  well  trained  health  workforce  is  the  backbone  of  the  sustainable  health  system,    the  action, 
attention  and  political  will  to  support  its  governance,  development  and  planning  should  be  clearly 
reflected and addressed by the EU Programme. 

FROM THE ON-LINE GROUP ON ACCESS TO BETTER AND SAFER HEALTH OBJECTIVE 

“Facilitate  access  to  better  and  safer  healthcare  for  Union  citizens”  would  be  achieved  through 
increasing  access  to  medical  expertise  and  information  for  specific  conditions,  also  beyond  national 

57 

 
 
 
 
 
borders.  It  would  also  entail  helping  to  apply  research  results  and  developing  tools  for  the 
improvement  of  healthcare  quality  and  patient  safety  through,  inter  alia,  actions  contributing  to 
improve health literacy.  

Even if goals (partly political) are clear, they are not credible if sufficient resources are not allocated 
for implementation. In addition, goals must be more specific: experts suggested to put the emphasis on 
monitoring efficiency and effectiveness to ensure the sustainability of healthcare systems.  

Greater emphasis should be put on lesson learning, the definition of guidelines and constitution of a 
strong  evidence  base,  and  the  dissemination  of  these  best  practices  in  all  MS  –  rather  than  leaving 
national stakeholders responsible for the implementation without much guidance. To some extent this 
is  already  happening,  but  we  need  robust  descriptions  of  best  practices  also  on  broader  healthcare 
areas.  

Too many priorities imply a fragmented budget, which cannot translate into any meaningful impact. In 
this  regard,  instead  of  very  specific  targets  which  affect  (the  health  of)  a  small  proportion  of  EU 
citizens,  those  issues  need  to  be  addressed  which  are  more  prevalent  and  in  the  interest  of  a  wider 
population.  

Needs  like  smoking,  sexually-transmissible  diseases,  infant  mortality  rates,  rare  diseases,  migrants’ 
health or medical tourism could be domains where funds can be spent more effectively. An enhanced 
focus of health action should be one of the best practices in health policy (including prioritising and 
ethics), supported by robust implementation mechanisms.  

2.2 E-SURVEY WITH PROGRAMME COMMITTEE MEMBERS AND NATIONAL FOCAL POINTS 

The e-survey was distributed by email to all National Focal Points (NFPs) and Programme Committee 
members  (PCs)  of  the  Third  Health  Programme  on  20  July,  2016.  The  survey  remained  open  for 
almost a month and sought to collect information on perceptions of these key stakeholders in terms of 
the following issues: 

  Suitability  of  the  four  objectives  of  the  Programme  and  ability  to  address  (emerging)  health 

needs 

  Existence of health needs which have not been addressed by the Programme and whether they 

should be covered by action at the EU level 

 

Implementation  of  the  Programme:  barriers  to  the  participation  of  interested  organisations; 
sufficiency  of  support  and  guidance  provided  by  DG  SANTE  /  CHAFEA  (NFPs  only);  and 
actions to be supported in the Annual Working Programme (AWP) (PCs only). 

Responses were provided by 45 National Focal Points and Programme Committee members, covering 
all EU-28 Member states with the exception of Estonia, plus Norway and Republic of Serbia.  

The results of the eSurvey have been shared with the national Focal Points and Programme Committee 
Members in the respective regular meetings on 7 and 8 December 2016.  

In  these  meetings  it  was  agreed  with  them  to  complement  the  eSurvey  with  an  additional  three 
questions as to be possible to obtain additional feedback on specific issues:  

 - such as whether or not action is needed at EU level rather than only national / regional / local levels 
(the most important reason according to them justifying EU action for each specific objective of the 
Programme) 

-  outline  the  three  thematic  priorities  that  are  most  important  for  public  health  in  their  country  in 
relation to each objective of the Programme. 

-the initial survey mentioned several potential barriers that could prevent interested organisations from 
participating in the Programme in their country. Respondents to the mini survey were invited to select 
the potential barrier that they consider most important in their country. 

58 

 
 
 
The  mini  survey  was  completed  by  eight  Programme  Committee  Members  (or  alternates),  nine 
National  Focal  Points  (or  alternates)  and  nine  respondents  who  stated  that  they  fulfil  both  a 
Programme Committee Member and a National Focal Point roles (or their alternates).   

Representatives from 11 low-GNI Member States participate, whereas the National Focal Points and 
Programme Committee Members from high-GNI Member States to respond were 14.  

The esurvey was informative for the understanding of EU added value the Programme's institutional 
stakeholders make, confirmed the suitability of Programme objectives, the sufficiency of guidance and 
support  for  the  implementation  of  the  Programme  by  CHAFEA  and  DG  SANTE,  and  provided 
information about the prioritisation of the thematic priorities and the barriers the Member States and 
other third countries are encountering in participating to the Programme. 

Figure 2: Importance of thematic priorities for public health in Member States 

59 

 
 
 
 
 
 
 
 
 
 
 
 
 
Figure 1: Relative importance of potential barriers to participation  

(low and high-GNI Member States) 

Administrative burden
(once project is up and running)

Securing co-financing for actions

44% 

44% 

Complexity of application process

16% 

Challenges in coordination between MS
(e.g. identifying partners, agreeing on roles,
language barriers)

12% 

Availability of information about HP support

0% 

Low-GNI  MS

High-GNI MS

2.3 STAKEHOLDERS INTERVIEWS  

2.3.1 INTERVIEWS WITH INTERNAL AND EXTERNAL STAKEHOLDERS 

On the basis of the literature review and evaluation criteria, a questionnaire was prepared as 
guidance for semi-structured interviews with representatives of the European Commission (12 
officials  from  DG  SANTE,  4  officials  from  DG  HOME,  DG  RTD,  DG  DEVCO,  DG 
CONNECT,  DG  GROW  and  6  staff  members  of  CHAFEA  including  for  the  most  project 
officers but also communication officer, coordinator of the Programme, etc.). The purpose of 
these  interviews  was  to  shed  light  on  all  aspects  of  the  design  and  implementation  of  the 
Programme,  particularly  regarding  new  features  such  as  the  thematic  priorities  and  multi-
annual planning. 

Interviews  have  also  been  organised  with  representatives  from  ECDC,  WHO/Europe  and 
independent  experts  involved  as  external  evaluators  in  the  selection  procedure  for  the 
awarding of funding. 

2.3.2 INTERVIEWS WITH HEALTH SECURITY COMMITTEE MEMBERS 

Finally,  given  the  political  sensitivity  of  specific  objective  2  (Protect  citizens  against  cross-
border  health  threats),  the  originally  foreseen  focus  group  was  replaced  with  a  series  of  in-
depth  interviews  with  members  of  the  Health  Security  Committee  (HSC).  The  interviews 
were  conducted  during  November  and  December  2016  and  included  members  of  France, 
Malta  and  Spain.  An  additional  interview  with  a  representative  from  Croatia  took  place  in 
early January. 

Stemming from what was discussed with the HSC experts during the interviews, it is possible 
to identify some strengths and areas for improvement of how the Programme addresses health 
security: 

Strengths:  Objective  2  is  clear  and  covers  the  most  important  aspects  of  the  protection  of 
citizens  from  serious  cross-border  health  threats.  Thematic  priorities  related  to  risk 
assessment,  capacity  building,  and  coordination/cooperation  between  relevant  stakeholders 
were  considered  the  most  relevant  for  EU  action.  They  are  also  the  areas  were  the  most 
remarkable  achievements  have  been  realised.  In  this  respect,  the  EWRS  was  mentioned  by 

60 

 
 
 
 
 
one expert as an example of a tool developed by the EU that works very well and that is being 
used extensively by Member States. The creation of an EU network of reference laboratories 
was  also  cited  as  a  very  important  initiative,  as  well  as  the  joint  procurement  of  medical 
counter-measures. 

Areas for improvement: The formulation of objective 2 could be improved by emphasising the 
ways in which the protection of citizens will be accomplished, and there was agreement that 
this related to enhancing coordination and networking between Member States, but also with 
the EC, international organisations and third countries. Moreover, developing Member States 
capacities  to  define  and  disseminate  messages  to  the  public  during  an  outbreak  was 
considered  to  be  not  sufficiently  addressed  by  the  current  thematic  priorities  and  a  relevant 
area where the EU could play a more important role. 

2.3.3 INTERVIEWS WITH BENEFICIARIES (CASE STUDY INTERVIEWS) 

Based  on  the  assessment  of  the  results  from  the  desk  study  and  questionnaires,  several 
stakeholders were  identified to  be interviewed for the eight case studies. Mainly  these  were 
project  coordinators  from  the  around  30  actions  examined  for  the  case  studies.  Names  and 
details for interviews under each case study are provided in Annex B (case studies report) of 
the Evaluation Report. 

The  case  study  interviews  overall  confirmed  the  findings  from  the  targeted  on-line 
questionnaire, but allowed a more in-depth insight through the specific actions.  

3. OPEN PUBLIC CONSULTATION 

The  purpose  of  the  consultation  was  to  allow  stakeholder  to  provide  views  on  different 
aspects of the evaluation questions mainly  

  The  objectives  and  priorities  of  the  Programme,  and  the  extent  to  which  these  are 

appropriate and in line with health needs in the EU; 

  The way the Programme is implemented, and the extent to which this is effective and 

efficient; and 

  The overall added value and usefulness of the Programme 

(1) 

In total, 133 responses to the OPC were received, covering the countries listed in below. The 
vast majority of respondents listed a country in the EU-15 MS (which corresponds to the older 
MS of the EU and also MS which are typically more involved in the Programme.  

61 

 
 
 
 
 
 
 
 
 
 
 
 
 
Table 1: Main country of residence / establishment of OPC respondents (as stated) 

EU-15 Member 
States 

EU-13 Member 
States 

Associated countries 

Other 

Respondents  Austria (5) 

Bulgaria (3) 

Norway 

Global (offices in US, South 
Africa and representation in 
Brussels) 

Belgium (29) 

Croatia 

Switzerland 

Denmark (2) 

Cyprus (3) 

 Ukraine 

Finland (5) 
France (5) 
Germany (4) 
Greece  
Ireland (4) 
Italy (12) 
Luxembourg  
Netherlands (10) 
Portugal (3) 
Spain (13) 
Sweden (3) 
United Kingdom 
(7) 
104 

TOTAL  

 

Czech Republic  
Hungary 
Latvia 
Lithuania (4) 
Poland (2) 
Romania 
Slovakia (6) 
Slovenia (1) 

24 

2 

3 

Half  of  the  participants  reported  having  some  knowledge  on  EU  health  policy  and  the 
Programme,  the  number  of  individuals  aware  of  (general)  EU  health  policy  being  slightly 
more important. 

Over 90% of all respondents reported working on health issues that are closely related to the 
ones  supported  by  the  third  Health  Programme  and  three  in  four  were  aware  of  activities 
funded by the Programme relevant to their work.  

A  majority  of  respondents  were  individuals,  followed  closely  by  representatives  of  non-
governmental organisations. Almost one fifth of respondents were representatives of national, 
international 
regional  or 
organisations and professional associations were less numerous to participate. 

local  public  authorities.  Representatives 

from  academia, 

62 

 
 
 
 
 
 
 
 
 
 
Figure 4: Profile of respondents to OPC

51

However, almost half of the respondents noted they have never applied for funding from the 
Programme and one in 10 was unsure if they had applied. The reasons for not have done so 
are depicted in the below graphic. 

Figure 5: Reasons for not applying for funding through Programme, n= 51 

Lack of information on opportunities

29 

Excessive administrative burden

21 

Other

13 

Lack of information on how to apply

9 

Lack of partners in other European countries

The co-funding rates are not attractive enough

The opportunities and activities are not relevant…

7 

6 

6 

Lack of language skills

1 

3.1 Respondents’ views on EU cooperation 

The respondents appeared largely supportive of the activities funded under the Programme in 
the  context  of  EU  level  cooperation  between  actors  of  the  health  sector,  as  over  two  thirds 
stated that the cooperation is essential and should be maintained. Respondents also appeared 
to share the view that the Programme should be maintained as less than one in 10 agreed that 
it should be abandoned. The responses of survey participants were more nuanced on whether 
the  scope  of  the  Programme  should  be  expanded  to  include  other  health  areas.  Here,  the 
number of neutral respondents and those who disagreed with this statement were higher.  

51   As  respondents  were  able  to  select  more  than  one  answer  to  describe  the  two  profile  questions,  the  total 

number of responses adds up to more than 133.  

63 

 
 
 
 
 
 
                                                           
Figure 6: Opinions on the support of cooperation at EU level, n=133 

In practice, the 3HP’s 
results (at least at this 
mid-term stage) are 
not visible and the 
cooperation should be 
abandoned 

The 3HP should be
expanded to include
other health areas

The cooperation is
essential and should
be maintained

4% 

3% 

16% 

38% 

31% 

9% 

32% 

35% 

24% 

8% 

2% 

77% 

20% 

2% 

2% 

Strongly agree

Agree

Neither agree nor disagree

Disagree

Strongly disagree

Don’t know 

3.2 Respondents’ views on the Programme’s objectives and priorities  

The respondents were overwhelmingly of the view that the EU should continue supporting the 
important  health-related  challenges  facing  EU  citizens,  governments  and  health  systems 
reflected  in  the  formulation  of  the  Programme’s  objectives.  The  areas  considered  as  most 
important for EU action were to: 

  promote  health,  prevent  diseases,  and  foster  supportive  environments  for  healthy 

lifestyles (Specific Objective 1)  

  contribute  to innovative, efficient  and sustainable health systems (Specific  Objective 

3), and  

  contribute to addressing health inequalities and the promotion of equity and solidarity 

(General Objective). 

  The respondents were slightly more uncertain on the relevance of EU support for the 
protection  of  citizens  from  serious  cross-border  health  threats  (Specific  Objective  2) 
and for the facilitation of access to better and safer healthcare for EU citizens (Specific 
Objective 4).  

64 

 
 
 
 
 
Figure 7: Relevance of Programme objectives

Q 2.1. Do you think the EU should provide funding for actions in order to...? 

...promote health, prevent diseases, and foster
supportive environments for healthy lifestyles

…contribute to addressing health inequalities and the 
promotion of equity and solidarity 

…contribute to innovative, efficient and sustainable 
health systems 

…facilitate access to better and safer healthcare for EU 
citizens 

…protect citizens from serious cross-border health 
threats (Zika and Ebola outbreaks) 

74% 

64% 

59% 

53% 

17% 

7% 
2% 

2% 
1% 

27% 

7% 

2% 

1% 

33% 

38% 

6% 
3% 

6% 
2% 

2% 

2% 

45% 

35% 

15% 

Strongly agree

Agree

Neither agree nor disagree

Disagree

Strongly disagree

Don’t know 

0%

20%

40%

60%

80%

100%

The  majority of respondents agreed that  the Programme’s objectives and priorities  are clear 
and easy to understand, as well as consistent with wider EU policy objectives (more than 60% 
agreed  or  strongly  agreed  with  these  statements).  Similar  numbers  were  also  of  the  opinion 
that  the  Programme’s  objectives  and  priorities  are  in  line  with  the  main  health  needs  in 
Europe and are appropriate for addressing the key issues and challenges. Just over one in two 
respondents  agreed  or  strongly  agreed  with  the  remaining  statements.  Overall,  the  outlook 
was  slightly  less  positive  when  it  came  to  the  definition  of  the  Programme  objectives  and 
priorities and their consistency with national health policy objectives. 

Figure 2: Opinions on the Programme's objectives and priorities, n=133 

The OPC respondents were then invited to summarise any concerns about the relevance and 
coherence  of  the  Programme  and  its  objectives.  A  total  of  46  respondents  opted  to  provide 
comments. Often, the comments were very specific and it was difficult to identify trends or 

65 

 
 
 
 
shared concerns. Examples of comments are provided in the full synthesis report in Annex A 
of the Study Report52. 

The OPC respondents were invited to select up to five priorities that they consider to be the 
most important, and up to five that they consider to be not relevant.  

Figure 3 presents the answers provided by the OPC respondents on the respective importance 
of  the  Programme’s  23  thematic  priorities,  which  are  gathered  under  its  four  specific 
objectives.  

Figure 3: Respective importance of the Programme’s 23 thematic priorities, n=133 

Not relevant

Most important

1

e
v
i
t
c
e
j
b
O

2

e
v
i
t
c
e
j
b
O

3

e
v
i
t
c
e
j
b
O

4

e
v
i
t
c
e
j
b
O

1.1 Risk factors

1.2 Drugs-related health damage

1.3 HIV / AIDS, tuberculosis & hepatitis

1.4 Chronic diseases

1.5 Implementation of tobacco legislation

1.6 Health information

5% 

2.1. Additional capacities for risk assessment

2.2. Capacity-building against health threats

2.3. Implementation of EU legislation

2.4. Health information

4% 

8% 

9% 

3% 

2% 

25% 

26% 

15% 

18% 

17% 

12% 

8% 

7% 

28% 

23% 

21% 

48% 

60% 

43% 

3.1. Health Technology Assessment

3.2. Innovation and e-health

3.3. Health workforce forecasting and planning

3.4. Setting up a mechanism for pooling expertise

3.5. EIP on Active & Healthy Ageing

3.6. Implementation of EU legislation (medical … 

3.7. Health information

4.1. European Reference Networks

4.2. Rare diseases

13% 

17% 

8% 

23% 

14% 

16% 

10% 

21% 

11% 

17% 

5% 

17% 

11% 

5% 

11% 

18% 

11% 

13% 

4.3. Patient safety and quality of healthcare

4.4. Measures to prevent AMR etc.

1% 

5% 

40% 

32% 

4.5. Implementation of EU legislation  (SoHO)

4.6. Health information

11% 
11% 

8% 

17% 

52   Mid-term  Evaluation  of  the  Third  Health  Programme  (2014  –  2020)  -  Final  report  (Annex  A),  written  by 
ISBN  978-92-79-68450-0 

International  Development,  SQW  and  Economisti  Associati, 

Coffey 
doi: 10.2875/292289 

66 

 
 
 
                                                           
 
 
 
 
The  respondents  were  also  asked  to  list  any  other  important thematic  priorities  they believe 
the  Programme  should  support  in  the  future,  or  to  suggest  amendments  to  the  existing 
priorities. As many as 51 open replies were given, but the responses were very diverse. They 
included  areas  such  as  diabetes,  hepatitis  C  and  cancer  prevention  were  cited  which  are 
anyway  covered  by  the  Programme.  In  addition,  they  have  provided  as  additional  items  for 
inclusion in the Programme the endocrine disruptors in air, soil and water pollution. 

3.3 Respondents’ views on the implementation of the Programme 

The OPC respondents were then asked to share their opinions in relation to statements about 
the implementation of the Programme (2014 – 2020) to date (i.e. 2014 – 2016).  

  Opinions on the Programme’s funding mechanisms 

Figure 4 summarises the respondents’ level of agreement with three statements regarding the 
suitability of the Programme’s funding mechanisms. About a quarter of the OPC respondents 
were neutral on the three statements. Half of the respondents agreed that the types of funding 
mechanisms  used  by  the  Programme  are  appropriate  to  achieve  the  objectives  of  the 
programme. They were slightly less numerous to think that prioritised actions in the Annual 
Work  Programme  permit  the  optimal  involvement  of  health  actors  and  stakeholders'  groups 
by  making  appropriate  use  of  the  different  funding  mechanisms.  Less  than  one  in  three 
respondents agreed that the level of financial support that the Programme offers is appropriate 
to  address  its  objectives  and  the  respondents  were  more  numerous  to  disagree  with  this 
statement.  

One in 10 participants was unsure how to respond to the question.  

Figure 4: Opinions on the funding mechanisms of the Programme, n=133 

The level of financial support that the 3HP offers
is appropriate to address its objectives

The prioritised actions in the Annual Work
Programme permit the optimal involvement of
health actors and stakeholders' groups by…
The types of funding mechanisms used by the
3HP are appropriate to achieve the objectives of
the programme

2% 

26% 

25% 

25% 

13% 

10% 

3% 

39% 

20% 

24% 

4% 

11% 

6% 

45% 

24% 

12% 

3% 

10% 

Strongly agree

Agree

Neither agree nor disagree

Disagree

Strongly disagree

Don’t know 

The  participants  were  also  asked  whether  they  agreed  or  disagreed  that  the  Programme 
includes  appropriate  measures  to  involve  all  Member  States,  including  those  with  lower 
incomes.  Figure  51  summarises  the  responses  of  the  participants,  by  low-GNI  (Bulgaria, 
Croatia,  Czech  Republic,  Greece,  Hungary,  Latvia,  Lithuania,  Poland,  Slovakia,  Slovenia, 
Romania and Ukraine) and high-GNI countries (Cyprus, EU-15 except Greece and associated 
countries Norway and Switzerland).  

The  opinions  of  respondents  from  low-  and  high-GNI  countries  were  broadly  similar.  A 
majority of respondents from low-GNI countries agreed with the statement, and one in four 
were  neutral.  Only  less  than  10%  of  low-GNI  country  respondents  considered  that  the 
Programme does not include appropriate measures to involve lower income Member States. A 

67 

 
 
 
quarter of the respondents from high-GNI countries were unsure how to answer the question, 
but  almost  half  thought  that  the  Programme  includes  appropriate  measures  to  involve  all 
Member States, including those with lower incomes.  

Figure 5: Suitability of Programme measures to involve all MS, including those with lower incomes, 
n=133 

High-GNI countries, n=111

6% 

39% 

20% 

10% 

3% 

22% 

Low-GNI countries, n=22

4% 

61% 

26% 

4% 

4% 

Strongly agree

Disagree

 

0%

20%

Agree

40%

60%

80%

100%

Neither agree nor disagree

Strongly disagree

Don’t know 

  Additional  concerns  of  respondents  about  the  Programme  and  its 

implementation 

The participants were invited to share any additional concerns about the Programme and the 
way in which it is implemented, in the areas suggested in Figure 6. The respondents were able 
to  select  all  the  areas  of  concern  relevant  to  them.  Over  a  third  of  the  participants  did  not 
provide an answer. 

The  participants  to  the  OPC  answering  this  question  appeared  to  be  mainly  concerned  with 
the administrative burden, as well as the eligibility and funding arrangements. 

Figure 6: Areas of additional concern on the implementation of the Programme, n=133 

Administrative burden

No Answer

Eligibility / funding arrangements

Application process

Dissemination of results

56 

53 

37 

32 

29 

Other

21 

The  respondents  were  able  to  briefly  summarise  any  concerns  other  than  the 
suggested areas in an open response. A few examples of relevant quotes from the 21 
open replies received are provided below:  

68 

 
 
 
 
 
 

 

 

“More  specialised  programmes  are  needed,  mainly  by  disease  and  another,  a 
general one for rare diseases.” 

“Opportunities for young public health professionals” 

“Civil  society  participation  must  be  more  explicit,  even  further  -  each  member 
state  should  have  one  civil  society  organization  working  on  national  and 
international  level  included  in  the  joint  action.  Only  that  will  ensure  greater 
performance and a greater added value for the EU citizens.” 

  Dissemination of results from actions funded by the Programme 

The respondents were also asked whether they agree that the results of actions funded by the 
Programme are sufficiently disseminated and promoted to those who might be able to make 
use of them. Figure 72 presents a summary of the responses collected. The same proportion of 
respondents  agreed  and  disagreed  with  the  statement,  which  suggests  that  their  views  are 
ambivalent  on  the  promotion  of  the  Programme  to  potential  beneficiaries.  This  is  also 
reflected in the fact that a third of the respondents remained neutral. 

Figure 72: Opinions on adequacy of dissemination of results, n=133 

Q 3.3: To what extent do you agree with the following statement about the 
level of awareness of the Programme? 
"The results of actions funded by the Programme are sufficiently 
disseminated and promoted to those who might be able to make use of 
them." 
Neither agree nor disagree

Strongly disagree

Disagree

Agree

Strongly agree

Don’t know 

5% 

26% 

31% 

28% 

7%  5% 

  Final comments 

Finally, the respondents to the OPC were able to share views on issues they considered had 
not  been  covered  by  the  consultation.  44  respondents  provided  further  details  in  their  open 
replies.  Often,  the  comments  were  very  specific  and  it  was  difficult  to  identify  trends  or 
shared  concerns.  A  snapshot  (and  relevant  quotes)  of  the  key  aspects  discussed  in  the  open 
replies is provided in the full synthesis report in Annex B of the Evaluation Report. . 

Comments were also made on the most relevant areas for EU action: 

“EU added value is clearest in a focus on internal market measures to tackle cross-border 
health threats, not only including communicable diseases and pandemics, and especially 
a  stronger  focus  on  AMR,  but  also  recognising  that  all  member  states  face  the  same 
threats  from  cross-border  determinants  including  unhealthy  dietary  environments, 
alcohol,  environmental  health  threats  (including  air  quality,  climate  change),  consumer 
safety, tobacco, sedentary work and mobility, and the impacts of employment and social 
precarity on (mental) health.” 

69 

 
 
 
 
 
“The importance of prevention (of chronic diseases, communicable diseases and AMR) in 
economic  and  health  system  sustainability  should  be  highlighted  in  Health  contributions 
to  EU  economic,  budget,  taxation,  social  and  employment  policies  and  programmes. 
Much closer coordination is needed to respect the Health in all Policies requirement” 

  Summary of main findings  

Overall, the respondents to the open public consultation provided a positive feedback on the 
EU’s  Health  Programme  and  appeared  largely  supportive  of  the  activities  funded  under  the 
Programme. EU support is critical to EU level cooperation between actors of the health sector 
and respondents emphasised the role of the Programme for: 

  The  collection  of  best  practices,  dissemination  of  results  and  development  of 

guidelines 

  The provision of the necessary funding for campaigns and research 

  The support to address systemic challenges such as health inequality. 

The  Programme’s  objectives  are  broadly  agreed,  in  line  with  the  main  health  needs  and 
consistent  with  wider  EU  policy  objectives  and  other  EU  programmes.  But  respondents 
suggested  reconsidering  the  scope  of  the  programme  –  to  include  such  issues  as  diabetes, 
hepatitis C and cancer, but also endocrine disruptors in air, soil and water pollution – as well 
as the coordination and coherence with national health policy objectives.   

From an implementation perspective, respondents raised a number of issues which should be 
monitored and addressed to enhance the cost-effectiveness of the Programme: 

1.  Administrative burden appears to remain a concern, especially when co-funding rates 

are not systematically considered attractive enough for individual organisations; 

2.  Awareness  and  information  on  the  Programme  remain  relatively  limited,  even 
amongst  stakeholders:  more  emphasis  should  be  put  on  the  communication  and 
dissemination  on  the  Programme,  the  opportunities  created  and  its  results,  while  the 
support to the networking of stakeholders could also be strengthened. 

3. FEEDBACK TO STAKEHOLDERS  

and 

data 

The  results  of  the  open  public  consultation  have  been  published  with  a  synthesis  report  on 
factual 
on 
received  made 
contributions 
http://ec.europa.eu/health/programme/consultations/midterm_evaluation_en.  
A full synthesis report is annexed to the Study Report as Annex B53 and will also be publicly 
available after the adoption of the Commission Report on Mid-term evaluation.  

public 

the 

all 

The participants to the open consultation and targeted consultation have been informed about 
the nature and set-up of the evaluation, and that the findings would be used for analysis, and 
that the EU would communicate about the results of the study.  

53  

Mid-term Evaluation of the Third Health Programme (2014 – 2020) - Final report (Annex B) 
written by Coffey International Development, SQW and Economisti Associati;  
ISBN 978-92-79-68449-  4 doi: 10.2875/16031 

70