Communication from the Commission to the Council and the European Parliament - Independent external monitoring and evaluation of community activities in the area of research and technological development
Date: 1996-05-22
Institutions: European Commission
Subjects: EU activity, activity report, project evaluation, research and development, technological change

\s 

COMMISSION  OF THE  EUROPEAN  COMMUNITIES 

Brussels,  22.05.1996 
COM(96)220  final 

COMMUNICATION  FROM  THE  COMMISSION 
TO THE  COUNCIL  AND  THE EUROPEAN  PARLIAMENT 

INDEPENDENT  EXTERNAL  MONITORING  AND EVALUATION  OF 
COMMUNITY  ACTIVITIES  IN  THE AREA  OF RESEARCH  AND 
TECHNOLOGICAL  DEVELOPMENT 

SUMMARY 

The  Commission  is  continually  seeking  to  demonstrate  and  enhance  the  credibility  and 
efficiency  of  its efforts  in relation  to evaluation  of  Community  RTD  actions.  In the  light of 
the recent legislative Decisions on the fourth  Framework  Programme and the related  specific 
programmes, which prescribe a complex evaluation scheme and an active role for  independent 
external  experts  in this scheme,  the Commission  decided  to take a fresh  look at  Community 
RTD  evaluation  efforts.  The  Scientific  and  Technical  Research  Committee  (CREST)  was 
consulted  for  advice;  subsequently,  CREST established  an  Evaluation  Sub-Committee.  The 
Council's Research Group and Atomic Questions Group as well as the European  Parliament's 
Committee on Research, Technological Development and Energy (CERT) have been  informed 
of  this  development. 

This Communication summarizes the evaluation required by legislative Decisions and outlines 
a  rationalized  approach,  taking  into account  the CREST advice.  The  approach  will  produce 
results  in  a  timely,  efficient  and  cost-effective  manner.  It  concentrates  on  the  continuous 
monitoring  of  specific  programmes  and  the  Framework  Programme  and  their  five-year 
assessment.  Regular  reporting  will  be  made once a year,  correlated  with  the publication  of 
the  Annual  Report  requested  under  Article  130p  of  the  Treaty  on  European  Union.  The 
approach  has  been  introduced  on an  experimental  basis  in  1995 and  will  be  reviewed  after 
three  years  of  implementation. 

The  approach  builds  upon  the  solid  foundation  of  experience  already  acquired  by  the 
Commission  from past evaluations and takes into account a recent  initiative on concrete steps 
towards  best evaluation  practice  in the  Commission.  The  main  feature  of  the  approach  is a 
further  development towards coherent monitoring and evaluation of Community RTD actions. 
In particular,  the  approach: 

(i) 

(ii) 
(iii) 
(iv) 

introduces continuous monitoring and develops through programme evaluation to S&T 
policy  evaluation; 
spells  out  the  implementation  principles; 
involves  appropriate  assistance  from  independent  external  experts;  and 
should  produce  results  in time  for  discussion  on  future  programme  Decisions. 

The Commission is responsible for the monitoring and evaluation of Community  RTD efforts. 
However,  the involvement of outside experts and bodies and the openness of procedures  will 
enhance the independence and transparency of monitoring and evaluation.  In addition, a new 
level  of  accountability  is  introduced  through  publication  of  the  Commission's  responses  to 
recommendations produced by the five-year  assessment panel of the Framework  Programme. 
This  will  all  enhance  the  credibility of  Community  RTD  evaluation. 

Through  this  Communication,  the  Commission  invites  the  Council  and  the  European 
Parliament  to  take  note  of  this  rationalized  approach  which  takes  account  of  the  current 
legislative  requirements  for  monitoring  and  evaluation. 

1. 

INTRODUCTION 

The Commission's White Paper  "Growth, Competitiveness, Employment", COM(93)700 
final,  identifies  increased  RTD effort  and  its adaptation  to new  market conditions as one 
of  the  key  means  to  improve  the  industrial  and  technological  performance  of  Europe. 
Evaluation  -  both  the  evaluation  of  the  programmes  (their  management,  monitoring 
quality,  efficiency  of  internal  procedures, etc.) and  the evaluation of results  (increase in 
scientific  and  technical  knowledge,  dissemination  and  optimization  of results,  industrial 
exploitation  of  RTD  results  and  their  relation  to  innovation  processes,  measurement  of 
long-term  economic  and  social  effects,  etc.)  - plays an  important  role  in this  regard  by 
contributing  to: the efficient  implementation of RTD efforts;  coherence with all  relevant 
policies;  and  best  use of  public  resources. 

The  legislative  Decisions  on  the  fourth  Framework  Programme1  and  its  specific 
programmes,  require  the Commission  to  implement  a complex  evaluation  scheme  (see 
Point 3 below). In this Communication the Commission presents a rationalized  approach 
which  will  produce  results  in a timely,  efficient  and  cost-effective  manner. 

2.  EVOLUTION  OF COMMUNITY  RTD EVALUATION  EFFORTS 

The systematic evaluation of Community RTD activities was confirmed  with the Plan of 
Action  of  19832 and  has since evolved  further3.  The use of  external  experts  has been a 
key  feature  in promoting  the  independence  of  evaluation. 

In  1993, the Council  and  the  European  Parliament  invited  the  Commission  to continue 
to  improve  the  credibility  of  the  evaluation  of  Community  RTD  programmes  while 
ensuring  its  independence.  Subsequently,  the  legislative  Decisions  on  the  fourth 
Framework  Programme  included  additional  requirements  concerning  evaluation. 

In  1994,  the  Commission  Services  involved  in  the  implementation  of  the  Framework 
Programme, prepared a reflection  document as a first step in response to the Council and 
the European Parliament in this regard. The document was transmitted to CREST, CERT 
and  the  European  Science and  Technology  Assembly  (ESTA). 

In  1995,  CREST  produced  advice4  which  included  the  establishment  of  a  CREST 
Evaluation  Sub-Committee.  The  Research  Group  and  the  Atomic  Questions  Group 
(Council), as well as CERT (Parliament),  have been kept informed  of this development. 

On 21  March  1995 the Commission  approved  a programme5  for  "Sound  and  Efficient 
Financial Management (SEM 2000)", divided into three phases: consolidation within the 

In this document, the term  "fourth  Framework  Programme" comprises Community activities resulting  from two Decisions: J^ Decision  N° 
1110/94/EEC concerning  the 4th  Framework  Programme of the  European  Community  activities in the field of  research and  technological 
development and demonstration  (1994-1998);  and,  2L Decision  N°  94/268/Euratom  concerning  a  framework  programme of  Community 
activities  in  the  field  of  research  and  training  for  the  European  Atomic  Energy  Community  (1994-1998). 

Council  Resolution  of  28 June  1983 

Communication  to  the  Council,  concerning  a  Community  Plan  of  Action  relating  to  the  Evaluation  of  Community  Research  and 
Development  Activities  for  the  years  1987 to  1991,  COM  (86) 660 final, 20  November  1986 

CREST advice to Council  and the Commission on the monitoring and evaluation  procedures  for Community  research  programmes; 
adopted  at  its  meeting  of  18 May  1995. 

Recommendation  n° 4 of the Memorandum  from  the  President,  Mrs.  Gradin and Mr.  Liikanen,  SEC (95)  1301/4 of 22 July  1995; 
and  Communication  to the Commission on  Evaluation  (concrete  steps  towards  best  practice  in  the  Commission).  SEC  ... 

present  framework,  reform  of  financial  management  culture,  and  partnership  with 
Member  States.  In  adopting  the  second  phase  of  this  programme  the  Commission 
introduced,  among  other  things,  the  requirement  for  systematic  evaluation  for  all 
Community  programmes  and actions and clarified  that  it  is primarily  the  responsibility 
of  operational  Directorate  Generals  to carry  out  the  task  of  evaluation. 

3. 

EXISTING  SITUATION 

The  legislative  Decisions  on  the  fourth  Framework  Programme  and  their  specific 
programmes6  require  the Commission  (see Table  1): 

- 

- 

For  the  Framework  Programme: 
- 

to  continually  and  systematically  monitor,  with  appropriate  assistance  from 
independent external experts, the progress in relation to its initial objectives  (Articles 
4.1);  and 
to  have an external  assessment conducted  by  independent  qualified  experts  into the 
management  of and progress  with  Community  activities  carried  out during  the  five 
years  preceding  this assessment  and  communicate  the assessment,  accompanied  by 
the  Commission's  comments,  to  the  European  Parliament,  the  Council  and  the 
Economic  and  Social  Committee  prior  to  presenting  a  proposal  for  the  fifth 
Framework  Programme  (Article  4.2); 

For  the  Specific  Programmes: 
- 

to  continually  and  systematically  monitor,  with  appropriate  assistance  from 
independent  external  experts,  the progress  within  the programme  (Article  4.1); 
to  have  an  external  assessment  conducted  by  independent  qualified  experts  of  the 
activities  carried  out  within  the  domains  covered  by  the  programme  and  their 
management  during  the  five  years  preceding  this assessment  (Article  4.2); 

-  and,  on  completion  of  each  specific  programme,  to  provide  an  independent  final 
evaluation  of  the  results  achieved  compared  with  objectives  and  to  forward  this 
evaluation  to  the  European  Parliament,  the  Council  and  the  Economic  and  Social 
Committee  (Article  4.3). 

To  implement  these legislative requirements  with  the existing  methodology  would  lead 
to: 

i)  a  need  to  conduct  2  major  evaluation  exercises  (one  mid-way  through  the  fourth 
Framework  Programme  and  the  other  on  completion),  in  each  case  for  about  20 
programmes; 

ii)  too short a time  interval  between  these 2 evaluations  to be able to detect  significant 

progress; 

iii)  final evaluations being required before significant S&T results are achieved and their 

impacts  become  apparent;  and 

iv)  a requirement to clarify  the roles of independent external experts who would provide 

'appropriate  assistance'. 

Council  Decisions: 94/801/EC or 23 November  1994; 94/572/EC  of 27 July  1994; 94/802/EC  of 23 November  1994;  94/571/EC 
of 27 July  1994; 94/803/EC of 23 November  1994; 94/911/EC ofl5  December  1994; 94/804/EC of 23 November  1994; 94/912/EC 
or  15  December  1994;  94/9I3/EC  of.  15 December  1994;  94/805/EC  of 23  November  1994;  94/806/EC  of  23  November  1994; 
94/914/ECof  15 December  1994; 94/915/EC of  15 December  1994; 94/807/EC of 23 November  1994: 94/9 IT/EC of  15 December 
1994;  94/916/IIC  of  15 December  1994;  94/920/1-ura loin  of  15 December  1994; 94/799/Euratom  of  8  December  1994;  and 
Council  Decisions  implemented  by the Joint  Research  Centre: 94/918/EC or  15 December  1994; 94/919/Euratom of  15 December 
1994. 

4. 

APPROACH  FOR  RATIONALIZATION 

The  following  outlines  a  rationalised  approach  which  the  Commission  intends  to 
implement.  It  takes  into  account  the  CREST  advice  and,  where  appropriate,  the 
principles  of  the  SEM  2000  initiative. 

4.1  How  to  rationalize 

The  CREST  advice  recognises  that  the complex  evaluation  requirements  above  can  in 
practice  be  satisfied  by  a  rationalized  approach,  while  avoiding  bureaucracy  and 
evaluation  for  its  own  sake  (see  Table  1).  Programme  monitoring  and  evaluation 
requirements  will  be  satisfied  by  two  kinds  of  actions  relating  to  the  Framework 
Programme  and  its specific  programmes: 

i)  continuous  monitoring,  with  appropriate  assistance  from  independent  external 

experts,  involving  annual  reporting;  and 

ii)  five-year  assessment  mid-way  through  implementation.  This  assessment  will  be 
conducted  by  independent external experts and  will  incorporate  the final  evaluation 
under  the previous  Framework  Programme.  The  final  evaluation  for  any  ongoing 
specific  programmme  will  be  incorporated  in the subsequent  five-year  assessment. 
Thus,  the  final  evaluation  (Article  4.3)  will  be  independent  from  the  five-year 
assessment  (Article 4.2)  for  any  one  programme. 

These  two  activities,  monitoring  and  evaluation,  are  distinct  and  separate exercises. 

the  Framework  Programme, 

For 
the 
Commission's comments, will be communicated to the European Parliament, the Council 
and  the  Economic  and  Social  Committee  prior  to  presenting  a  proposal  for  the  next 
Framework  Programme  (Article  4.2). 

the  five-year  assessment, 

together  with 

Moreover,  the  CREST advice  suggests  that  the  issues  to be addressed  will  include: 

the  consistency  of  the  selection  of  projects  with  the  objectives  of  the  specific 
programme  and  the work  programme; 
the extent  to  which  selected  projects  or  clusters of  projects  are  fulfilling  the  wider 
policy  objectives  of  the  Community,  in  particular  in  areas  of  relevance  to  the 
specific  programme  concerned; 
the progress  and  output  of  projects  against  the original  targets  set; 
cases  where  the  independent  monitoring  teams  consider  the  results  will  have  a 
important  impact,  or  where  poor  performance  requires  further 
particularly 
examination; 
the efficiency  and transparency of the management of the programme,  including  the 
development of calls, the assessment and selection process, contract negotiation and 
disbursement  of  funds,  as  well  as  internal  Commission  co-ordination; 
the  use of  special  measures  and  support  activities  (e.g.  to support  SMEs,  improve 
dissemination  of  information,  etc); 
the changes  that  may be  needed  to the balance of  the programme  or  to the  strategy 
for  implementation  in the light of experience and changes  in the wider  environment. 

The CREST advice further  suggests a set of input and output performance  indicators  for 
the  monitoring  and  evaluation  processes  (see  Point  5.1 below). 

4.2  How  to  maximize  benefit 

An  integrated  monitoring  and  evaluation  approach,  with  the  assistance  of  external 
experts,  is  outlined  below  where  continuous  monitoring  reports  will  in  time  provide 
significant  supplementary  information  to the annual  report  required  under Article  130p 
of the Treaty on European Union7. The five-year  assessment of programmes will provide 
a  major  input  to discussions  on  future  Community  RTD  activities. 

In this move to continuous monitoring and five-year  assessment,  results will be available 
at  the  time  when  decisions  have  to  be  taken  and  it  will  be  possible  to  have  a  global 
overview of the state of implementation at regular  intervals. In addition, evaluations will 
form  a  'coherent ensemble'.  Lessons from  evaluations will be able to be drawn not only 
for  specific  programmes,  but  also  for  the  Framework  Programme  and  the  whole 
Community  S&T  policy. 

Appropriate involvement of outside experts and bodies (notably CREST and Programme 
Committees),  and  openness  of  procedures,  will  enhance  the  independence  and 
transparency  of  evaluations  whilst  recognizing  the  need  to  respect  confidentiality  of 
certain  data.  The  external  experts  will  discuss  their  reports  with  the  Programme 
Managers and the Programme Committees. This process should facilitate  implementation 
of the experts' recommendations.  In addition, a new level of accountability is introduced 
through  publishing  the  Commission's  responses  to  recommendations  produced  by  the 
five-year  assessment  panel  of  the  Framework  Programme. 

5. 

EXTERNAL  MONITORING 

5.1  Specific  Programmes 

The  .primary  functions  of  external  monitoring  will  be  to  assess  the  progress  of 
programmes  in order  to assist  Programme  Managers  in adapting  specific  programmes 
to  developments,  and  to  contribute  to  the  five-year  assessment  of  such  programmes. 
Monitoring  will  be  a  quick  response  mechanism  providing  an  annual  picture  of 
programme  development. 

Monitoring will be based on continuous and systematic collection of data by  Programme 
Managers about the progress and results achieved  in the specific  programmes.  With the 
assistance of  CREST, a set of  suggested  performance  programme  indicators have been 
identified  which  primarily  relate  to  shared-cost  programmes  under  Activity  1 of  the 
fourth  Framework  Programme.  The  other  specific  programmes,  such  as 
the 
dissemination and optimization of results and the thermonuclear  fusion,  follow  different 
implementation  procedures  for  which  different  performance 
indicators  may  be 
appropriate.  The Joint Research  Centre (JRC) participation  in specific  programmes  will 
be  considered  like any  other  participant,  while  the JRC  direct-action  activities  will  be 
reported  on  through  the  "Observations  of  the  Board  of  Governors  on  the JRC  Annual 
Report"  which  will  directly  constitute  an  input  to  the  overall  Framework  Programme 
monitoring  exercise. 

Treaty  on  European  Union,  Article  130p:  "At  the  beginning  of  each  year  the  Commission  shall  send  a  report  to  the  European 
Parliament  and  the  Council.  The  report  shall  include  information  on  research  and  technological  development  activities  and  the 
dissemination  of  results  during  the  previous  year,  and  the  work  programme  for  the  current  year." 

The  relevant  and  available  data  for  each  specific  programme  contained 
in  the 
performance  indicators will be analyzed by a small panel of independent experts for each 
specific  programme  who would  provide  an external  view. 

The  monitoring  panels will  not duplicate  the functions  of  Programme  Committees,  nor 
will they conflict  with them, but add an extra dimension,  i.e. of independent  assessment 
concerning  the overall performance  and achievements of each specific  programme.  The 
monitoring  panels  will  assess  programme  output  against.programme  objectives  and 
against  the  progress  in  implementation. 

Each  panel  will  present  the  results  of  its  analysis  to  the  relevant  Programme 
Management  which  will  take appropriate  actions. 

5.2  Framework  Programme 

The Framework  Programme  monitoring will assess the year-on-year  implementation of 
the overall  programme and will  contribute  to its five-year  assessment.  An  independent 
external  experts  panel  will  review  the  outcome  of  examinations  of  the  specific 
programmes  and  assess  the  overall  progress  in  relation  to  objectives,  priorities  and 
progress  in implementation of the entire Framework Programme.  In assessing the whole 
Framework  Programme, the contribution to the objective of strengthening  the  scientific 
and  technological  bases  of  Community  industry  and  encouraging  competitiveness  at 
international level, as well as the contribution to the implementation of other Community 
policies  shall  be taken  into  consideration. 

The  panel  will  present  the  results  of  its  analysis  to  the  Framework  Programme 
Management  which  will take appropriate  actions. 

6. 

EXTERNAL FIVE-YEAR ASSESSMENTS  OF SPECIFIC PROGRAMMES  AND 
FRAMEWORK  PROGRAMME 

For  specific  programmes and  the entire Framework  Programme,  five-year  assessments 
timed to provide input to discussions on.future legislative decisions on Community  RTD 
activities,  will  be  produced  by  panels  of  independent  external  experts.  The  annual 
monitoring  reports,  detailed  above,  will  provide  an  input  to  the  five-year  assessment 
which will  include a more strategic dimension.  (For example,  long-term projects  which 
have  been  developed  in  successive  Framework  Programmes  may  be  assessed  over  a 
longer  time  scale.)  These  assessments  will  be  ready  when  the  proposal  for  the  next 
Framework  Programme  is discussed. 

The  five-year  assessments,  incorporating  the  final  evaluations  under  the  previous 
Framework  Programme,  will  include three principal elements, each  looking at distinct, 
but  inter-related  features: 

i) 

ii) 

assessing  relevance,  i.e.  whether  the  initial  objectives  are  still  valid  against 
evolving  S&T,  industrial  and  socio-economic  conditions; 
assessing  efficiency,  i.e.  whether  the  objectives  have  been  pursued  in  a  cost-
effective  manner;  and, 

iii)  assessing  effectiveness,  i.e.  whether  the  initial  objectives  have  been  achieved: 
contributions  to  strengthening  the  scientific  and  technological  bases  and  the 
competitiveness  of  Community  industry,  as  well  as  contributions  to  all  relevant 
Community  policies. 

Special  attention  will  be  paid  to  the  development  of  reliable  methodologies  for  the 
evaluation  of  RTD  project  results.  To  this  end,  an  ad  hoc  Working  Group  has  been 
created  which  includes representatives  from  industry.  It will  recommend,  with  the help 
of external contributions, a common methodology to be used for  the evaluation of results 
of  RTD  projects,  particularly  those  with  industrial  relevance. 

The  five-year  assessments  might  pay  particular  attention  to  the  coherence  between 
Community  and  national  S&T  policies  with  a  view  to  enhancing  their  mutual 
consistency8. 

For  the  specific  programmes,  as  the  five  year  assessment  will  incorporate  the  final 
evaluation of the previous specific programme,  the assessment reports will be  forwarded 
to  the  European  Parliament,  the  Council  and  the  Economic  and  Social  Committee,  as 
required  in Article 4.3  of  the  relevant  Decisions.  For  the  Framework  Programme  the 
five-year  assessment  report,  accompanied  by  the  Commission's  comments,  will  be 
forwarded  to  the  same  bodies  as  above  prior  to  presenting  a  proposal  for  the  next 
Framework  Programme,  as  required  in  Article  4.2  of  the  relevant  Decision. 

7. 

IMPLEMENTATION 

The  Commission  is responsible  for  the  monitoring  and  evaluation  of  Community  RTD 
efforts.  Appropriate consultation of CREST, Programme Committees and other relevant 
bodies  will  be sought.  Independent  external  experts  will be contracted  to undertake  the 
tasks  of  monitoring  and  five-year  assessment  and  preparing  reports. 

Monitoring  panels  for  the specific  programmes  will  normally  comprise  3 experts  with 
appropriate experience and stature: one from  industry, one from academia, and one with 
experience  of  RTD  programme  evaluation.  Monitoring  panels  for  the  Framework 
Programme  will normally comprise 6-7 experts,  the larger  number  reflecting  the wider 
scope of their work. A similar balance of industrial, academic and programme evaluation 
experience  will  be sought. 

For  the  five-year  assessments,  the  panels,  while  maintaining  a  minimum  number  of 
experts,  will  be  slightly  larger  than  for  monitoring.  The  size  of  the  panels  for  the 
specific  programme  assessments  will  be  determined  in  relation  to  the  objectives  and 
scope  of  the  respective  programmes.  A  panel  of  6-8  experts  will  be  required  for  the 
Framework Programme assessment. The five-year  assessment panels might include a few 
experts from the related monitoring panels to ensure continuity between the two  different 
exercises. The experts will  be drawn  from  a variety  of backgrounds,  appropriate  to the 
programme  under  consideration,  and  will  be expected  to have extensive  experience  in 
order  to  fully  examine  horizontal  issues. 

Special  attention  will  be  paid  to  ensure  the  coherence  of  monitoring  and  five-year 
assessment and  to maintain  the highest possible degree of  transparency  for  the exercise. 

Regular  reporting  will  be  made  once  a  year,  correlated  with  the  publication  of  the 
Annual  Report  requested  under  Article  130p of  the Treaty  on  European  Union. 

Treaty  on  European  Union,  Article  I30h:  "The  Community  and  the  Member  States  shall  co-ordinate  their  research  and  technological 
development  activities  so  as  to  ensure  that  national  policies  and  Community  policy  are  mutually  consistent". 

8. 

TIMING 

Every  effort  will  be made to  implement this scheme as quickly  as possible. 

The  1995 monitoring exercise has been completed. The external experts' reports on the 
specific  programmes and on the overall  Framework  Programme are available. 

In respect of  the forthcoming  five-year  assessment,  the first  panels  will  be expected to 
start  their  work  during  Spring  1996  with  specific  programme  reports  expected  by 
Summer/Autumn 1996 and the report on the overall Framework  Programme before the 
end of  the year. 

The  rationalized  approach,  suggested  above,  will  be  reviewed  after  three  years  of 
implementation. 

9. 

CONCLUSIONS 

The series  of  regular  annual  monitoring  reports and five-year  assessment  reports  will 
provide  a  comprehensive  overview  of  the  management  of  programmes  (Framework 
Programme and specific programmes),  their implementation and benefits  derived. The 
involvement  of  external  experts  and  the  openness  of  procedures  will  enhance  the 
independence  and  transparency  of  monitoring  and  evaluation.  The  publication  of 
Commission's responses to recommendations produced by the five-year assessment panel 
of  the  Framework  Programme  will  introduce a new  level  of  accountability.  Meetings 
between the experts and  Programme Managers and the Programme Committees on the 
results of  monitoring and evaluation will ensure mutual understanding and facilitate the 
implementation of appropriate recommendations. All these features will further enhance 
the credibility  of  Community  RTD evaluation  efforts. 

The five-year  assessment reports will be given to the European Parliament,  the Council 
and  the  Economic  and  Social  Committee  thus  providing  input  to  future  legislative 
decisions  on Community  RTD activities. 

Through  this  Communication,  the  Commission  invites  the  Council  and  the  European 
Parliament to take note of this rationalized approach which takes account of  the current 
legislative  requirements  for  monitoring and evaluation. 

TABLE 1: EXISTING SITUATION vs.  SUGGESTED SCHEME 

1990 

1991 

1992 

1993 

1994 

1995 

1996 

1997 

1998 

1999 

2000 

2001 

2002 

Y E AR 

FP 3 : 1990-1994 

1.  LEGISLATIVE  OBLIGATIONS: 

FP3 

Specific programmes 

Review 

! 

FP3 

2) 

FP4 

State of 
implementation 

Specific programmes 

f 

{ 

FP4 

{ 

2. 

SUGGESTED  SCHEME: 

FP 4 : 1994-1998 

FP 5 : 1998-2002 

Final 
evaluation 

Final 
evaluation 

5 year 
assessment 

h-L 

Final 
evaluation 

Continuous monitoring 

State of 

5 year 

implementation  I  assessment n 

(*) 

« 

(*> 

(*> 

,  Continuous monitoring, 
reporting at the beginning 
of each year (*) 

5-year assessment, 
(incl. final evaluation of FP3) 
approx. period  covered:  1991-1995 

5-year assessment 
(incl. final evaluation of FP4) 
approx. period covered:  1995-1999 

JFP3  +  F P4 

|  (+  F P5  + 

...'.) 

{  Specificprogramine, 

j 

(*) 

(*) 

(*) 

(*) 

(*) 

(*) 

Continuous monitoring, 
reporting at the beginning 
of each year (*) 

Note : arrows indicate the approximate periods during which evaluation  reports should be issued. 

ISSN 0254-1475 

COM(96) 220  final 

DOCUMENTS 

EN 

15 

Catalogue number  : CB-CO-96-231-EN-C 

ISBN 92-78-04209-9 

Office  for Official Publications of the European Communities 

L-2985 Luxembourg