Communication from the Commission to the European Parliament, the Council, the European Economic and Social Committee and the Committee of the Regions - Interim evaluation of the implementation of the Public Health Programme (2003-2008)
Date: 2008-07-24
Institutions: European Commission
Subjects: EU programme, health care system, health policy, public health, report

EN 

EN 

   EN 

 
 
COMMISSION OF THE EUROPEAN COMMUNITIES 

Brussels, 24.7.2008 
COM(2008) 484 final 

COMMUNICATION FROM THE COMMISSION TO THE EUROPEAN 
PARLIAMENT, THE COUNCIL, THE EUROPEAN ECONOMIC AND SOCIAL 
COMMITTEE AND THE COMMITTEE OF THE REGIONS 

Interim evaluation of the implementation of the Public Health Programme (2003-2008) 

EN 

   EN 

 
 
 
  
 
 
COMMUNICATION FROM THE COMMISSION TO THE EUROPEAN 
PARLIAMENT, THE COUNCIL, THE EUROPEAN ECONOMIC AND SOCIAL 
COMMITTEE AND THE COMMITTEE OF THE REGIONS 

Interim evaluation of the implementation of the Public Health Programme (2003-2008) 

(Text with EEA relevance) 

1. 

INTRODUCTION 

The Decision of the European Parliament and of the Council adopting a programme 
of Community action in the field of public health (2003-2008)1, hereinafter referred 
to as “the Programme Decision”, established a six-year programme on public health 
to be implemented from 1 January 2003 to 31 December 2008. 

The purpose of this communication is to present the conclusions of the evaluation of 
the  implementation  of  the  first  three  years  of  the  Public  Health  Programme (2003-
2008), hereinafter referred to as “the programme”. 

The  need  to  carry  out  the  evaluation  stems  from  Article  12(3)  of  the  aforesaid 
Decision, which also requires the European Commission to communicate the results 
of the evaluation to European Parliament, the Council, the European Economic and 
Social Committee, and to the Committee of the Regions. 

The evaluation was carried out by independent experts from a consultancy company, 
RAND Europe, selected through a tendering procedure. 

The full evaluation report is available online on the DG Sanco Europa website at the 
following address: http://ec.europa.eu/health/ph_programme/eval2003_2008_en.htm. 

2. 

BACKGROUND 

Under Article 152 of the Treaty establishing the European Community, a high level 
of human health protection must be ensured in the definition and implementation of 
all Community policies and activities. 

This  programme,  which  complements  national  policies,  is  designed  to  help  to 
achieve the target set in the Treaty. 

The general objectives of the programme are to: 

–  improve health information and knowledge for the development of public health; 

–  enhance  the  capability  of  responding  rapidly  and  in  a  coordinated  fashion  to 

threats to health; 

–  promote health and prevent disease through action on health determinants across 

all Community policies and activities. 

The programme is thus designed to: 

1 

Decision  No 1786/2002/EC  of  the  European  Parliament  and  of  the  Council  of  23 September 2002 
adopting  a  programme  of  Community  action  in  the  field  of  public  health  (2003-2008),  OJ L 271, 
9.10.2002, p. 1. 

EN 

2 

   EN 

 
                                                 
–  ensure  a  high  level  of  human  health  protection  in  all  Community  policies  and 

activities, by way of an integrated and intersectoral health strategy; 

–  tackle inequalities in health; 

–  encourage  cooperation  between  Member  States  in  the  areas  covered  by 

Article 152 of the Treaty. 

3. 

METHODOLOGY 

The evaluation was based on several data collection methods, namely desk research, 
a survey of project leaders, interviews with a variety of stakeholders, analysis of data 
archives and analysis of the projects database. 

It  focused  mainly  on  projects  co-funded  in  the  period  2003-2005  under  the  annual 
calls  for  proposals,  which  represented  the  main  activity  of  the  programme.  The 
budget  allocated  for  the  calls  for  proposals  accounted  for  most  of  the  operating 
budget. 

As this was an interim evaluation, it was carried out when most of the projects were 
at  an  early  stage  of  development.  This  meant  that  the  evaluators  were  unable  to 
measure exactly to what extent the objectives of the programme were met by projects 
co-funded under the Calls for Proposals for 2003-2005. 

Taking  these  limitations  into  account,  the  following  issues  regarding  the  Public 
Health Programme (2003-2008) were addressed in the evaluation. 

4. 

EVALUATION ISSUES AND QUESTIONS 

The evaluation assessed the following specific issues:  

Relevance: pertinence of the programme objectives to the needs of its stakeholders. 

Effectiveness:  extent  to  which  the  objectives  set  in  the  Programme  Decision  and 
annual  work  plans  are  achieved;  degree  to  which  the  priorities  described  in  the 
annual work plans and the Programme Decision have been duly implemented by the 
programme;  extent  to  which  the  programme  manages  to  complement  national 
policies  (by  way  of  European  added  value)  to  protect  human  health  and  improve 
public health at reasonable cost. 

Efficiency: timeliness of resource allocation, appropriate quantity, quality and cost-
effectiveness;  analysis  of  the  programme’s  organisational  set-up  and  structure, 
related  responsibilities  and  overall  management  capacity  in  relation  to  the  results 
achieved; methods used to implement the programme. 

complementarity: 

Consistency  and 
and 
complementarity  is  ensured  between  measures  implemented  under  the  programme 
and  other  EU  policies  and  activities  (such  as  regional  policy,  research,  statistical 
programmes, labour policy, etc.). 

consistency 

to  which 

extent 

Acceptability:  extent  to  which  stakeholders  accept  public  health  policy  in  general 
and the way in which the programme implements this policy. 

Monitoring: adequacy of the current monitoring system of programme activities. 

Sustainability:  extent  to  which  the  results  of  the  programme  are  sustainable  in  the 
medium and long term, i.e. after the funding granted by the EU has ceased. 

EN 

3 

   EN 

 
5. 

FINDINGS 

The key findings of this evaluation are presented below. 

5.1. 

Relevance 

A  survey  of  project  leaders  and  interviews  with  different  stakeholder  groups  has 
shown  that  there  is  a  clear  perception  among  stakeholders  that  the  programme 
delivers  benefits  for  stakeholders  and  EU  citizens  alike.  Significant  benefits 
mentioned  were:  network  creation,  exchange  of  knowledge  and  good  practices, 
Europe-wide cooperation and capacity building. 

5.2. 

Effectiveness 

The  survey  showed  that  projects  are  considered  to  support  the  delivery  of  the 
objectives of the Programme Decision and the work plans. “The selection of projects 
is  perceived  to  have  been  managed  through  clear  and  transparent  evaluation 
procedures published in the Online Journal, reinforcing the sense that only relevant 
projects  have  been  funded.  In  this  process,  evaluators  appear  to  have  applied 
appropriate criteria transparently and this impression is supported by evidence from 
the surveys and interviews”.2 

5.3. 

Efficiency 

A  significant  number  of  project  leaders  found  the  project  administration  to  be 
complex and bureaucratic, while the project submission procedure was considered to 
be time-consuming and burdensome. 

5.4. 

Consistency and Complementarity 

The extent to which the Public Health Programme (2003-2008) interacts with other 
Community  activities  varies  by  area;  in  particular,  interaction  is  perceived  to  be 
particularly close in the fields of bioterrorism, health systems and pharmaceuticals. 

5.5. 

Acceptability 

Interviewees and stakeholders widely accepted and supported European intervention 
in  public  health.  The  added  value  of  the  programme  derives  from  the  fact  that  it 
could  be  an  effective  link  between  the  health  policies  of  national  governments  and 
the EC, and thus help to support and coordinate the respective measures. 

5.6.  Monitoring 

Of  all  the  programme’s  activities,  the  evaluation  focused  on  the  monitoring  of 
projects  co-funded  under  the  calls  for  proposals.  The  survey  and  interviews 
suggested that the monitoring of projects is adequate. 

5.7. 

Sustainability 

Almost  half  the  project  leader  respondents  (47%)  thought  that  the  project  and  its 
results  would  be  sustainable  once  financial  support  for  the  programme  is  stopped. 
Respondents  commented  that  sustainability  could  be  achieved  through  the  use  of 
project outputs; dissemination (e.g. public health reporting); continued collaboration 
and  networking  with  partners  and  other  organisations;  training  and  additional 
funding.  The  evaluation  concluded  that  it  is  clear  that  at  least  some  parts  of  the 
programme are expected to have sustainable consequences. 

2 

RAND Interim Evaluation of the Public Health Programme (2003-2008), Final Report, pages 100-101. 

EN 

4 

   EN 

 
                                                 
6. 

RECOMMENDATIONS 

The  overall  assessment  of  the  Public  Health  Programme  (2003-2008)  is  positive. 
However,  eighteen  recommendations  were  issued  in  three  areas:  programme, 
management and projects. 

6.1. 

Programme 

The  evaluation  defined  the  programme  priorities  as  complicated.  Even  if  the 
priorities  may  be  entirely  justified,  they  might  not  reflect  the  actual  needs  of 
stakeholders  and/or  of  EU  citizens.  Greater  attention  should  be  paid  to  developing 
sharper priorities through a more needs-driven prioritisation process. 

It was recommended that the focus be on areas with high impact and of high concern. 

6.2.  Management  

Efforts should be made to develop new funding methods in order to encourage more 
innovative proposals. 

An “inner circle” of stakeholders are very aware of the work of the programme, but 
there  is  a  wider  potential  audience  that  may  be  being  missed.  The  evaluation 
suggested  that  the  benefits  produced  by  the  co-funded  projects  should  be 
disseminated through tailored messages to interest groups.  

Considering that successful delivery of the programme depends, among other factors, 
on  cooperation  with  international  organisations,  it  was  recommended  that  existing 
links with international bodies be maintained and strengthened. 

6.3. 

Projects 

It was recommended that the application procedure for submitting projects under the 
call for proposals be simplified. 

The  evaluation  suggested  managing  and  limiting  the  number  of  proposals  to  be 
assessed  by  considering  indirect  competition,  closed  competition  and  informal 
competition. 

The  evaluation  concluded  that  projects  should  monitor  their  progress  against  the 
programme priorities. 

7. 

COMMISSION’S COMMENTS ON THE EVALUTION CONCLUSIONS 

7.1. 

Programme 

The  evaluation  regarded  the  programme  priorities  as  being  complicated.  Even  if 
these  priorities  may  have  been  entirely  justified,  they  might  not  have  reflected  the 
actual needs of stakeholders and/or EU citizens. 

Several initiatives have been launched to streamline and prioritise the Public Health 
Programme activities, such as mapping of the Public Health Programme (2003-2008) 
and  multi-annual  planning.  These  will  help  to  establish  priorities  for  each  year  of 
new Health Programme, i.e. the second programme of Community action in the field 
of  health (2008-2013)3,  identify  stakeholders’  needs  and  define  specific  objectives. 

3 

Decision  No 1350/2007/EC  of  the  European  Parliament  and  of  the  Council  of  23  October  2007 
establishing  a  second  programme  of  Community  action  in  the  field  of  health  (2008-13),  OJ L 301, 
20.11.2007, p. 3. 

EN 

5 

   EN 

 
                                                 
This  will  give  the  new  Health  Programme  a  sharper  definition  by  reducing  the 
number of activities and making priorities more clear-cut. 

In addition to this, a broader and innovative consultation process will be pursued in 
an effort to define the priorities of the annual work plans, address the real needs of 
EU citizens and increase stakeholders’ awareness of the programme. Several groups 
of stakeholders will be consulted to identify what for them are the relevant areas of 
activity. 

7.2.  Management 

As  regards  the  possibility  of  introducing  new  funding  methods,  it  should  be  noted 
that  a  wider  variety  of  financing  mechanisms  are  offered  in  the  second  Health 
Programme (2008-2013). These include: 

–  co-financing an action intended to achieve a programme objective (up to 60% of 

project costs); 

–  co-financing  the  operational  costs  of  a  non-governmental  organisation  or  a 

specialised network (up to 60% of eligible costs); 

–  joint  financing  a  public  body  or  non-governmental  organisation  by  the 

Community and one or more Member States (up to 50% of costs); 

–  joint actions with other Community programmes. 

As regards project results, these will be promoted through adequate channels tailored 
to the different kinds of audience. 

Further  attention  will  be  paid  to  project  sustainability.  Output  indicators  could  be 
used to ensure closer follow-up of projects. At the end of project implementation, an 
assessment should be made of the real output produced by the beneficiary. The aim is 
to check to what extent the final reports reflect expectations at the beginning of the 
project. 

Programme  activities  will  be  communicated  more  widely  than  in  the  past.  For  this 
purpose,  a  comprehensive  strategy  has  been  developed.  New  initiatives  have  been 
promoted and others which are already in place have been expanded, such as the use 
of e-newsletters and the web (DG SANCO website, the EU public health portal and 
the  Health-EU  e-newsletter,  Public  Health  Executive  Agency  website),  Info  Days 
(based in Luxembourg and National Info Days), further development of the national 
focal points network, and new health-related publications. 

The  Commission  will  step  up  cooperation  with  international  organisations,  such  as 
the WHO and OECD through direct grant agreements. 

7.3. 

Projects 

The application procedure for submitting projects under the call for proposals should 
be simplified. This has already been done for the 2007 Call for Proposals and will be 
further pursued under the second Health Programme (2008-2013).  

Furthermore,  evaluation  criteria  in  use  for  the  call  for  proposals  were  revised  in 
2007. Following the evaluation carried out by the Public Health Executive Agency in 
2006, a Commission Decision laying down new general principles and criteria for the 

EN 

6 

   EN 

 
selection and funding of actions under the Public Health Programme4 was adopted on 
12 February 2007. 

The award criteria were revised with the aim of making the evaluation process more 
transparent  and  providing  more  guidance  to  applicants  on  what  is  expected  from 
projects. The award criteria were clearly detailed and some aspects were made more 
specific than was previously the case, e.g. what is meant by geographical coverage, 
the  quality  and  role  of  partnerships  in  the  context  of  calls  for  proposals,  European 
added value, the budget and financial management. 

As regards limiting the number of proposals under the calls for proposals through the 
means proposed by the evaluation, the Commission would point out that the financial 
regulation  prescribes  direct  competition.  Using  closed  competition  in  the  public 
health area is not an advantage, and informal competitions do not seem to guarantee 
impartiality and equal treatment. 

Beneficiaries will be invited to describe how their proposals will improve the health 
of  European  citizens,  as  measured  where  possible  by  appropriate  indicators, 
including  the  Healthy  Life  Years  indicator,5  and  how  they  will  reduce  health 
inequalities in and between EU Member States and regions. 

8. 

CONCLUSIONS 

This  interim  evaluation  drew  largely  positive  conclusions,  while  highlighting  a 
number of areas that it recommended for consideration in the further implementation 
of the Public Health Programme (2003-2008) and for the development of the second 
Health Programme (2008-2013). 

It  was  an  extensive  exercise,  covering  a  long  period,  from  the  planning  stage  to 
finalisation of the report. It ended too late in the policy cycle for any decision to be 
made on a substantial revision of the Public Health Programme (2003-2008), which 
was repealed on 1 January 2008 with the entry into force of the second programme of 
Community action in the field of health (2008-2013). 

However,  the  results  of  the  evaluation  were  immediately  taken  into  account, 
whenever  possible,  in  efforts  to  improve  the  remainder  of  the  Programme  (2003-
2008). The Commission will pay the utmost attention to the recommendations of this 
evaluation during implementation of the second Health Programme (2008-2013). 

4 
5 

OJ L 46, 16.2.2007, p. 45-50. 
http://ec.europa.eu/health/ph_information/indicators/lifeyears_en.htm  

EN 

7 

   EN